Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DO NOT MERGE #6440

Closed
wants to merge 10 commits into from
1 change: 1 addition & 0 deletions website/docs/docs/cloud/about-cloud/about-dbt-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ id: "dbt-cloud-features"
sidebar_label: "dbt Cloud features"
description: "Explore dbt Cloud's features and learn why dbt Cloud is the fastest way to deploy dbt"
hide_table_of_contents: true
unlisted: true
pagination_next: "docs/cloud/about-cloud/architecture"
pagination_prev: null
---
Expand Down
1 change: 1 addition & 0 deletions website/docs/docs/cloud/about-cloud/architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: "dbt Cloud Architecture"
id: "architecture"
unlisted: true
description: "Information about the architecture, communication, and security of dbt Cloud"
---

Expand Down
1 change: 1 addition & 0 deletions website/docs/docs/cloud/about-cloud/tenancy.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: Tenancy
id: tenancy
description: "Information about single tenant and multi-tenant dbt Cloud instances"
unlisted: true
---

import AboutCloud from '/snippets/_test-tenancy.md';
Expand Down
1 change: 1 addition & 0 deletions website/docs/docs/get-started-dbt.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: "dbt Quickstarts"
id: get-started-dbt
unlisted: true
hide_table_of_contents: true
pagination_next: null
pagination_prev: null
Expand Down
1 change: 1 addition & 0 deletions website/docs/reference/global-configs/cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ There are two ways to optionally modify this behavior:
- `CACHE_SELECTED_ONLY` (default `False`): Whether to limit cache population to just the resources selected in the current run. This can offer significant speed improvements when running a small subset of a large project, while still providing the benefit of caching upfront.

For example, to quickly compile a model that requires no database metadata or introspective queries:

```text

dbt --no-populate-cache compile --select my_model_name
Expand Down
112 changes: 112 additions & 0 deletions website/docs/vpc/test-doc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
---
title: "Upgrading to v1.8 (latest)"
id: test-doc
description: New features and changes in dbt Core v1.8
displayed_sidebar: "docs"
unlisted: true
---

## Resources

- Changelog (coming soon)
- [dbt Core CLI Installation guide](/docs/core/installation-overview)
- [Cloud upgrade guide](/docs/dbt-versions/upgrade-dbt-version-in-cloud)

## What to know before upgrading

dbt Labs is committed to providing backward compatibility for all versions 1.x, except for any changes explicitly mentioned on this page. If you encounter an error upon upgrading, please let us know by [opening an issue](https://github.com/dbt-labs/dbt-core/issues/new).

Check warning on line 17 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L17

[custom.Typos] Oops there's a typo -- did you really mean '1.x'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean '1.x'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 17, "column": 76}}}, "severity": "WARNING"}

## Keep on latest version

With dbt Cloud, you can get early access to many new features and functionality before they're in the Generally Available (GA) release of dbt Core v1.8 without the need to manage version upgrades. Refer to the [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version) setting for more details.

Check warning on line 21 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L21

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 21, "column": 148}}}, "severity": "WARNING"}

## New and changed features and functionality

Features and functionality new in dbt v1.8.

Check warning on line 25 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L25

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 25, "column": 39}}}, "severity": "WARNING"}

### New dbt Core adapter installation procedure

Before dbt Core v1.8, whenever you would `pip install` a data warehouse adapter for dbt, `pip` would automatically install `dbt-core` alongside it. The dbt adapter directly depended on components of `dbt-core`, and `dbt-core` depended on the adapter for execution. This bidirectional dependency made it difficult to develop adapters independent of `dbt-core`.

Check warning on line 29 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L29

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 29, "column": 17}}}, "severity": "WARNING"}

Beginning in v1.8, [`dbt-core` and adapters are decoupled](https://github.com/dbt-labs/dbt-adapters/discussions/87). Going forward, your installations should explicitly include _both_ `dbt-core` _and_ the desired adapter. The new `pip` installation command should look like this:

Check warning on line 31 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L31

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 31, "column": 14}}}, "severity": "WARNING"}

```shell
pip install dbt-core dbt-ADAPTER_NAME
```

For example, you would use the following command if you use Snowflake:
```shell
pip install dbt-core dbt-snowflake
```

For the time being, we have maintained install-time dependencies to avoid breaking existing scripts in surprising ways; `pip install dbt-snowflake` will continue to install the latest versions of both `dbt-core` and `dbt-snowflake`. Given that we may remove this implicit dependency in future versions, we strongly encourage you to update install scripts **now**.

### Unit Tests

Check warning on line 44 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L44

[custom.SentenceCaseHeaders] 'Unit Tests' should use sentence-style capitalization. Try '' instead.
Raw output
{"message": "[custom.SentenceCaseHeaders] 'Unit Tests' should use sentence-style capitalization. Try '' instead.", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 44, "column": 5}}}, "severity": "WARNING"}

Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure.

In v1.8, we're introducing native support for [unit testing](/docs/build/unit-tests). Unit tests validate your SQL modeling logic on a small set of static inputs __before__ you materialize your full model in production. They support a test-driven development approach, improving both the efficiency of developers and the reliability of code.

Check warning on line 48 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L48

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 48, "column": 4}}}, "severity": "WARNING"}

Starting from v1.8, when you execute the `dbt test` command, it will run both unit and data tests. Use the [`test_type`](/reference/node-selection/methods#the-test_type-method) method to run only unit or data tests:

Check warning on line 50 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L50

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 50, "column": 15}}}, "severity": "WARNING"}

```shell

dbt test --select "test_type:unit" # run all unit tests
dbt test --select "test_type:data" # run all data tests

```

Unit tests are defined in YML files in your `models/` directory and are currently only supported on SQL models. To distinguish between the two, the `tests:` config has been renamed to `data_tests:`. Both are currently supported for backward compatibility.

#### New `data_tests:` syntax

The `tests:` syntax is changing to reflect the addition of unit tests. Start migrating your [data test](/docs/build/data-tests#new-data_tests-syntax) YML to use `data_tests:` after you upgrade to v1.8 to prevent issues in the future.

Check warning on line 63 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L63

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 63, "column": 197}}}, "severity": "WARNING"}

```yml

models:
- name: orders
columns:
- name: order_id
data_tests:
- unique
- not_null


```

#### The `--empty` flag

The [`run`](/reference/commands/run#the-`--empty`-flag) and [`build`](/reference/commands/build#the---empty-flag) commands now support the `--empty` flag for building schema-only dry runs. The `--empty` flag limits the refs and sources to zero rows. dbt will still execute the model SQL against the target data warehouse but will avoid expensive reads of input data. This validates dependencies and ensures your models will build properly.

### Deprecated functionality

The ability for installed packages to override built-in materializations without explicit opt-in from the user is being deprecated.

- Overriding a built-in materialization from an installed package raises a deprecation warning.
- Using a custom materialization from an installed package does not raise a deprecation warning.
- Using a built-in materialization package override from the root project via a wrapping materialization is still supported. For example:

```sql
{% materialization view, default %}
{{ return(my_cool_package.materialization_view_default()) }}
{% endmaterialization %}
```

### Managing changes to legacy behaviors

dbt Core v1.8 has introduced flags for [managing changes to legacy behaviors](/reference/global-configs/legacy-behaviors). You may opt into recently introduced changes (disabled by default), or opt out of mature changes (enabled by default), by setting `True` / `False` values, respectively, for `flags` in `dbt_project.yml`.

Check warning on line 98 in website/docs/vpc/test-doc.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/vpc/test-doc.md#L98

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/vpc/test-doc.md", "range": {"start": {"line": 98, "column": 10}}}, "severity": "WARNING"}

You can read more about each of these behavior changes in the following links:

- (Mature, enabled by default) [Require explicit package overrides for builtin materializations](/reference/global-configs/legacy-behaviors#require_explicit_package_overrides_for_builtin_materializations)
- (Introduced, disabled by default) [Require resource names without spaces](https://docs.getdbt.com/reference/global-configs/legacy-behaviors#require_resource_names_without_spaces)
- (Introduced, disabled by default) [Run project hooks (`on-run-*`) in the `dbt source freshness` command](/reference/global-configs/legacy-behaviors#source_freshness_run_project_hooks)

## Quick hits

- Custom defaults of [global config flags](/reference/global-configs/about-global-configs) should be set in the `flags` dictionary in [`dbt_project.yml`](/reference/dbt_project.yml), instead of in [`profiles.yml`](/docs/core/connect-data-platform/profiles.yml). Support for `profiles.yml` has been deprecated.
- New CLI flag [`--resource-type`/`--exclude-resource-type`](/reference/global-configs/resource-type) for including/excluding resources from dbt `build`, `run`, and `clone`.
- To improve performance, dbt now issues a single (batch) query when calculating `source freshness` through metadata, instead of executing a query per source.
- Syntax for `DBT_ENV_SECRET_` has changed to `DBT_ENV_SECRET` and no longer requires the closing underscore.

Loading