Skip to content

Commit

Permalink
Adds ls troubleshooting tips (#5266)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?
The `dbt ls` command has been a personal goldmine find in helping to
debug selectors and complex state:modified problems.

I wanted to get this documented for others to see examples of how it can
be used for this purpose. We do call out the command as an option in the
table of commands, but there aren't any examples of how and why you
might want to use it, and I believe it's arguably the best tool for
troubleshooting this stuff, as it works in both the IDE and in jobs for
logging.

## Checklist
<!--
Uncomment when publishing docs for a prerelease version of dbt:
- [X] Add versioning components, as described in [Versioning
Docs](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-entire-pages)
- [X] Add a note to the prerelease version [Migration
Guide](https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/docs/docs/dbt-versions/core-upgrade)
-->
- [X] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [X] For [docs
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning),
review how to [version a whole
page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
and [version a block of
content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content).
- [X] Add a checklist item for anything that needs to happen before this
PR is merged, such as "needs technical review" or "change base branch."
  • Loading branch information
mirnawong1 authored Apr 11, 2024
2 parents 7a82e76 + a139818 commit 314664f
Showing 1 changed file with 20 additions and 8 deletions.
28 changes: 20 additions & 8 deletions website/docs/reference/node-selection/syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Examples:
```bash
dbt run --select "my_dbt_project_name" # runs all models in your project
dbt run --select "my_dbt_model" # runs a specific model
dbt run --select "path.to.my.models" # runs all models in a specific directory
dbt run --select "path/to/my/models" # runs all models in a specific directory
dbt run --select "my_package.some_model" # run a specific model in a specific package
dbt run --select "tag:nightly" # run models with the "nightly" tag
dbt run --select "path/to/models" # run models contained in path/to/models
Expand Down Expand Up @@ -103,9 +103,21 @@ As your selection logic gets more complex, and becomes unwieldly to type out as
consider using a [yaml selector](/reference/node-selection/yaml-selectors). You can use a predefined definition with the `--selector` flag.
Note that when you're using `--selector`, most other flags (namely `--select` and `--exclude`) will be ignored.

### Troubleshoot with the `ls` command

Constructing and debugging your selection syntax can be challenging. To get a "preview" of what will be selected, we recommend using the [`list` command](/reference/commands/list). This command, when combined with your selection syntax, will output a list of the nodes that meet that selection criteria. The `dbt ls` command supports all types of selection syntax arguments, for example:

```bash
dbt ls --select "path/to/my/models" # Lists all models in a specific directory.
dbt ls --select "source_status:fresher+" # Shows sources updated since the last dbt source freshness run.
dbt ls --select state:modified+ # Displays nodes modified in comparison to a previous state.
dbt ls --select "result:<status>+ state:modified+ --defer --state ./<dbt-artifact-path>" # Lists nodes that match certain [result statuses](/reference/node-selection/syntax#the-result-status) and are modified.
```

<Snippet path="discourse-help-feed-header" />
<DiscourseHelpFeed tags="node-selection"/>


## Stateful selection

One of the greatest underlying assumptions about dbt is that its operations should be **stateless** and **<Term id="idempotent" />**. That is, it doesn't matter how many times a model has been run before, or if it has ever been run before. It doesn't matter if you run it once or a thousand times. Given the same raw data, you can expect the same transformed result. A given run of dbt doesn't need to "know" about _any other_ run; it just needs to know about the code in the project and the objects in your database as they exist _right now_.
Expand Down Expand Up @@ -179,14 +191,14 @@ dbt run --select "result:<status> --defer --state path/to/prod/artifacts"

The available options depend on the resource (node) type:

| | model | seed | snapshot | test |
| `result:\<status>` | model | seed | snapshot | test |
|----------------|-------|------|------|----------|
| `result:error` | | | ||
| `result:success` | | | | |
| `result:skipped` | | | | |
| `result:fail` | | | | |
| `result:warn` | | | | |
| `result:pass` | | | ||
| `result:error` |||||
| `result:success` |||| |
| `result:skipped` || || |
| `result:fail` | | | | |
| `result:warn` | | | ||
| `result:pass` | | | ||

### Combining `state` and `result` selectors

Expand Down

0 comments on commit 314664f

Please sign in to comment.