Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate observation notice, optional fields, sql writing guide #280

Merged
merged 5 commits into from
Aug 20, 2024

Conversation

dogversioning
Copy link
Contributor

This addresses the following tickets via documentation:

#147 (along with smart-on-fhir/cumulus#35)
#214
#216

Checklist

  • Consider if documentation (like in docs/) needs to be updated
  • Consider if tests should be added
  • Update template repo if there are changes to study configuration

nav_order: 9
nav_order: 10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hard-coded nav order system we have does feel less graceful than I would like, but 🤷

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it's not ideal, but we're not dealing with Gravity's Rainbow here or anything

@@ -59,6 +59,38 @@ You can see which encounters were ignored as incomplete by examining the
Usually, you can resolve this by running the ETL process again for the encounters,
making sure to include all associated resources.

## Optional fields
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oof, this is reasonable documentation to include, but so easy to get out of date. And it's not like there's a single spot in the source where you could add a comment like "if you add a field, add it to the docs too"...

Though... maybe you could add a note like that to the five or so resource tables we support now?

But regardless, we can be vigilant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'll add a checkbox item to the PR template - that remains surprisingly effective.

@@ -368,6 +400,7 @@ making sure to include all associated resources.
| Column | Type |Description|
|------------|-------|-----------|
|id |varchar| |
|row |bigint | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... interesting.

I think these got removed by me, because that's what a generate-md did for me, when I pointed the Library at the BCH Cerner data. You also shouldn't have core__medication in your db anymore. I think you should re-run the core study on whatever data source you used (or use the BCH Cerner data)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I actually made some updates in one of the other PRs, i should regen this anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am still seeing this after regen.

I've been running this against a synthetic dataset, which might have different structure? and/or may be out of date?

I'm going to leave this as is for now rather than rebuilding the cerner table before we release breaking changes, but I'm going to add a staging DB at some point that we can use for stuff like this.


FHIR dates/times come across the wires as ISO 8061-formatted strings.
To get them into actual timestamps, use the `from_is08061_timestamp` function as
part of your query, and case to either `date` or `timestamp`.If you want to get a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "case to" is a bit jargony, which might appropriate for a detailed SQL doc, but still - maybe "convert to"? Or use backticks to indicate you mean case as a method.

nit: missing a space between the period and "If"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah lemme do CASE, since SQL is always yelling at you.

Comment on lines 44 to 45
date_trunc('day', date(from_iso8601_timestamp(dr."context"."period"."start")))
AS author_day,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might be an odd example, right - does date_trunc('day', date_value) actually do anything? Maybe truncate to month?

FROM documentreference AS dr
```

## Arrays
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section sounds vaguely familiar to me. Do we have SQL writing guidelines somewhere else too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you thinking of the ticket you filed? 😄

there is some discussion of this in creating-sql-with-python.md, which is maybe a layer of abstraction above this doc.

null values, since these are indistinguishable from the nulls created when cubing outputs.
Instead,
[coalesce](https://trino.io/docs/current/functions/conditional.html#coalesce)
fields to some static value, like 'None', if required.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth mentioning that the frontend handles values like cumulus__none well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably, yeah

Copy link

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
1996 1979 99% 90% 🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 29d8f2c by action🐍

@dogversioning dogversioning merged commit 7f1770f into main Aug 20, 2024
3 checks passed
@dogversioning dogversioning deleted the mg/docs_update branch August 20, 2024 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants