Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable crossdb column metadata with ra3 nodes #738

Closed
wants to merge 2 commits into from
Closed

Conversation

dataders
Copy link
Contributor

@dataders dataders commented Mar 25, 2024

resolves #736

Problem

R3-node users cannot fetch column metadata for tables & views outside of the current database.

Solution

Similar to @jiezhen-chen's approach in #428: use svv_all_columns instead of information_schema."columns".

While we're here, I have a hunch that @aiguofer's dbt-labs/dbt-core#2754 and it's usage of pg_catalog.svv_external_columns can be dropped. Still to be verified.

Questions

  1. Does this fix things for RA3 node cross-db scenarios?
  2. Does it break things for non-RA3 scenarios?
  3. What is the performance impact of such a change?
  4. Is it correct to assume that svv_all_columns includes the results of svv_external_columns?
  5. To ameliorate 2 and 3, should this svv_all_columns only be used if the relation-to-be-fetched's database does not match the one to which dbt is currently connected? (similar: [CT-1122] [CT-1116] [Bug] adapter.get_relation for relations in other databases [RA3] #179 (comment))

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@VolkerSchiewe
Copy link

  1. Is it correct to assume that svv_all_columns includes the results of svv_external_columns?

I can confirm that svv_all_columns contains column information of external schemas as well.

Copy link
Contributor

This PR has been marked as Stale because it has been open with no activity as of late. If you would like the PR to remain open, please comment on the PR or else it will be closed in 7 days.

@github-actions github-actions bot added the Stale label Sep 23, 2024
Copy link
Contributor

Although we are closing this PR as stale, it can still be reopened to continue development. Just add a comment to notify the maintainers.

@github-actions github-actions bot closed this Sep 30, 2024
@mikealfare mikealfare deleted the ra3_all_columns branch November 9, 2024 01:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] adapter.get_columns_in_relation does not work cross database
2 participants