Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add mf timespine behavior change #6408

Merged
merged 11 commits into from
Nov 4, 2024
Merged

add mf timespine behavior change #6408

Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
90102e6
add behavior change
mirnawong1 Nov 4, 2024
39b2959
Merge branch 'current' into add-behavior-flag
mirnawong1 Nov 4, 2024
05788a4
Merge branch 'current' into add-behavior-flag
mirnawong1 Nov 4, 2024
85e5456
Update website/docs/reference/global-configs/behavior-changes.md
nataliefiann Nov 4, 2024
1e6431e
Update website/docs/reference/global-configs/behavior-changes.md
mirnawong1 Nov 4, 2024
eab33f4
Update website/docs/reference/global-configs/behavior-changes.md
mirnawong1 Nov 4, 2024
9c22bb4
Merge branch 'current' into add-behavior-flag
mirnawong1 Nov 4, 2024
c1d128f
Update behavior-changes.md
mirnawong1 Nov 4, 2024
b6586b3
Update behavior-changes.md
mirnawong1 Nov 4, 2024
bbdf38f
Update website/docs/reference/global-configs/behavior-changes.md
mirnawong1 Nov 4, 2024
3a95356
Update website/docs/reference/global-configs/behavior-changes.md
mirnawong1 Nov 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions website/docs/reference/global-configs/behavior-changes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@
require_model_names_without_spaces: False
source_freshness_run_project_hooks: False
restrict_direct_pg_catalog_access: False
require_yaml_configuration_for_mf_time_spines: False
```

</File>
Expand All @@ -66,12 +67,13 @@

| Flag | dbt Cloud: Intro | dbt Cloud: Maturity | dbt Core: Intro | dbt Core: Maturity |
|-----------------------------------------------------------------|------------------|---------------------|-----------------|--------------------|
| require_explicit_package_overrides_for_builtin_materializations | 2024.04 | 2024.06 | 1.6.14, 1.7.14 | 1.8.0 |
| require_resource_names_without_spaces | 2024.05 | TBD* | 1.8.0 | 1.9.0 |
| source_freshness_run_project_hooks | 2024.03 | TBD* | 1.8.0 | 1.9.0 |
| [require_explicit_package_overrides_for_builtin_materializations](#package-override-for-built-in-materialization) | 2024.04 | 2024.06 | 1.6.14, 1.7.14 | 1.8.0 |
| [require_resource_names_without_spaces](#no-spaces-in-resource-names) | 2024.05 | TBD* | 1.8.0 | 1.9.0 |
| [source_freshness_run_project_hooks](#project-hooks-with-source-freshness) | 2024.03 | TBD* | 1.8.0 | 1.9.0 |

Check warning on line 72 in website/docs/reference/global-configs/behavior-changes.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/reference/global-configs/behavior-changes.md#L72 

[custom.Typos] Oops there's a typo -- did you really mean 'source_freshness_run_project_hooks'?
Raw output 
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'source_freshness_run_project_hooks'? ", "location": {"path": "website/docs/reference/global-configs/behavior-changes.md", "range": {"start": {"line": 72, "column": 4}}}, "severity": "WARNING"}
| [Redshift] [restrict_direct_pg_catalog_access](/reference/global-configs/redshift-changes#the-restrict_direct_pg_catalog_access-flag) | 2024.09 | TBD* | dbt-redshift v1.9.0 | 1.9.0 |
| skip_nodes_if_on_run_start_fails | 2024.10 | TBD* | 1.9.0 | TBD* |
| state_modified_compare_more_unrendered_values | 2024.10 | TBD* | 1.9.0 | TBD* |
| [skip_nodes_if_on_run_start_fails](#failures-in-on-run-start-hooks) | 2024.10 | TBD* | 1.9.0 | TBD* |

Check warning on line 74 in website/docs/reference/global-configs/behavior-changes.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/reference/global-configs/behavior-changes.md#L74 

[custom.Typos] Oops there's a typo -- did you really mean 'skip_nodes_if_on_run_start_fails'?
Raw output 
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'skip_nodes_if_on_run_start_fails'? ", "location": {"path": "website/docs/reference/global-configs/behavior-changes.md", "range": {"start": {"line": 74, "column": 4}}}, "severity": "WARNING"}
| [state_modified_compare_more_unrendered_values](#source-definitions-for-state) | 2024.10 | TBD* | 1.9.0 | TBD* |

Check warning on line 75 in website/docs/reference/global-configs/behavior-changes.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/reference/global-configs/behavior-changes.md#L75 

[custom.Typos] Oops there's a typo -- did you really mean 'state_modified_compare_more_unrendered_values'?
Raw output 
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'state_modified_compare_more_unrendered_values'? ", "location": {"path": "website/docs/reference/global-configs/behavior-changes.md", "range": {"start": {"line": 75, "column": 4}}}, "severity": "WARNING"}
| [require_yaml_configuration_for_mf_time_spines](#metricflow-time-spine-yaml) | 2024.10 | TBD* | 1.9.0 | TBD* |

When the dbt Cloud Maturity is "TBD," it means we have not yet determined the exact date when these flags' default values will change. Affected users will see deprecation warnings in the meantime, and they will receive emails providing advance warning ahead of the maturity date. In the meantime, if you are seeing a deprecation warning, you can either:
- Migrate your project to support the new behavior, and then set the flag to `True` to stop seeing the warnings.
Expand Down Expand Up @@ -152,3 +154,13 @@
- '{{ ... if flags.WHICH != 'freshness' }}'
```
</File>


### MetricFlow time spine YAML
The flag is `False` by default.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

Set the `require_yaml_configuration_for_mf_time_spines` flag to `True` to require YAML configuration for MetricFlow time spine file for dbt Cloud Versionless or dbt Core 1.9 and later. In previous versions (dbt Core 1.8 and earlier), the MetricFlow time spine configuration was stored in a `metricflow_time_spine.sql` file.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

When the flag is set to `True`, dbt will raise a deprecation warning if it detects a MetricFlow time spine configured in a SQL file. When the flag is set to `False`, dbt will continue to support the SQL file configuration.
Copy link
Contributor

@DevonFulcher DevonFulcher Nov 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating these docs, but this is the other way around. See my note in the PR description here

I added this to dbt_project.yaml in jaffle-sl-template and saw no warning for dbt parse:
flags:
require_yaml_configuration_for_mf_time_spines: True

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh right sorry, i missed the tests. thanks for clarifying @DevonFulcher ! so the flag allows users to continue to support the SQL file configuration. I'll update!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated it to this @DevonFulcher :

MetricFlow time spine YAML

The require_yaml_configuration_for_mf_time_spines flag is set to False by default.

Set the flag to True to require YAML configuration for MetricFlow time spine file for dbt Cloud Versionless or dbt Core 1.9 and later. In previous versions (dbt Core 1.8 and earlier), the MetricFlow time spine configuration was stored in a metricflow_time_spine.sql file.

When the flag is set to True, dbt will continue to support the SQL file configuration. When the flag is set to False, dbt will raise a deprecation warning if it detects a MetricFlow time spine configured in a SQL file.

The MetricFlow YAML file should be named models/_models.yml and should be located in the models directory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! So, both configurations are accepted for now, regardless of the value of require_yaml_configuration_for_mf_time_spines, so this line isn't totally accurate:

Set the flag to True to require YAML configuration for MetricFlow time spine file for dbt Cloud Versionless or dbt Core 1.9 and later.

I think we can just delete that sentence since you describe the True/False behavior later in this section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good flag, removed it now @DevonFulcher !


The MetricFlow YAML file should be named `models/_models.yml` and should be located in the `models` directory.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the model can be named anything as long as it has the time_spine: field. Maybe we just link to the v1.9 docs here?

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
Loading