Refactored remaining core tables

[dogversioning]

This PR makes the following changes:

• Updates to core templates for handling date fields
• Moves all base core tables to builders
• Moves core-related templates from base templates to core_templates
• moves non-template reading functions from template_sql/templates to template_sql/utils

Checklist

• Consider if documentation (like in docs/) needs to be updated
• Consider if tests should be added
• Run pylint if you're making changes beyond adding studies
• Update template repo if there are changes to study configuration

[mikix]

Not done yet, but I'm going to break for lunch and wanted to dump these on ya first 😄

[mikix]

This is probs fine since the ETL always gives you year-only birthDates. But if we ever want to handle cases where the day might actually fall anywhere in the year, this is the logic I used in quality (works in both athena and duckdb, unlike date_diff):

                year(targetDateTime)
                - year(birthDate)
                - (
                    CASE WHEN
                        extract(doy FROM birthDate) > extract(doy FROM targetDateTime)
                    THEN 1
                    ELSE 0
                    END
                )

[dogversioning]

frankly i just copied this over verbatim from the raw sql without thinking about it.

but - can we talk about this? i assume there's a reason, which is why i didn't touch this, but that feels like an odd behavior.

[mikix]

date_diff works like you want I believe, in Athena. (i.e. it takes day of year into account, is my memory)

But not duckdb - in duckdb, it is naive and just does year-minus-year.

So I used the logic above to get both to work the same. I'm guessing the author of that sql just didn't have duckdb in their brains because why would they. But you might hid oddities during unit testing if you ever feed this a non -01-01 date.

[dogversioning]

so the bit I actually wanted to talk about was truncating the birthdates at the ETL - just wanted to understand more of why that's happening.

[mikix]

Oh that was because of reducing PHI vibes - we keep most datetimes in place, for clinical reasons. But I guess they (Andy & Jamie) felt like birthdates were more sensitive and we should keep truncating them.

For reference, that happens at the MS-deid-tool layer and we could change the config to not do that.

[dogversioning]

OK - I created #169 - we're using that now in a way that's slightly inaccurate and i want to, at some point, just discuss the implications once more.

[mikix]

Dropping another group of comments, before I dig into the tests

[mikix]

OK finished looking at this 😮‍💨 and looks good! Thanks for all this refactoring.

Comments can be resolved as you see fit, but nothing deal breaking.