Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add variable to allow users to exclude all results columns #382

Merged
merged 2 commits into from
Sep 18, 2023

Conversation

glsdown
Copy link
Contributor

@glsdown glsdown commented Sep 13, 2023

Overview

As can be seen in #369, there are issues where the BQ length gets too large. This has started happening since introducing this column to return all results available in the objects. Therefore, I have added a new variable dbt_artifacts_exclude_all_results which can be set to true to disable this column from being populated.

Ideally, we'd want the default to be to 'not' include this field, but as it's been released with the field included (i.e. opt-in), it doesn't feel right to do that, so I have built it as an opt-out feature.

Update type - breaking / non-breaking

  • Minor bug fix
  • Documentation improvements
  • Quality of Life improvements
  • New features (non-breaking change)
  • New features (breaking change)
  • Other (non-breaking change)
  • Other (breaking change)
  • Release preparation

What does this solve?

Outstanding questions

What databases have you tested with?

  • Snowflake
  • Google BigQuery
  • Databricks
  • Spark
  • N/A

@glsdown glsdown added the enhancement New feature or request label Sep 13, 2023
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 15:54 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 15:54 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 15:54 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 16:01 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 16:01 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 16:01 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 16:01 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 16:01 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 16:01 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 16:02 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 13, 2023 16:02 — with GitHub Actions Inactive
@stevenkoppenol
Copy link

Great proposal. I hope this will solve the "query too long" error.

@glsdown glsdown merged commit 532edbb into main Sep 18, 2023
11 checks passed
@glsdown glsdown deleted the feat/exclude_all_results_columns branch September 18, 2023 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants