Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds profile_config for Docker #1347

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

andrewhlui
Copy link

@andrewhlui andrewhlui commented Nov 27, 2024

Description

Adds ProfileConfig to Docker operators so that people can use TaskGroups.

Related Issue(s)

Closes #1346

Breaking Change?

Checklist

  • I have made corresponding changes to the documentation (if required)
  • I have added tests that prove my fix is effective or that my feature works

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Nov 27, 2024
@dosubot dosubot bot added area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc area:profile Related to ProfileConfig, like Athena, BigQuery, Clickhouse, Spark, Trino, etc labels Nov 27, 2024
Copy link

codecov bot commented Nov 28, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.02%. Comparing base (06f242a) to head (5cfdce3).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1347   +/-   ##
=======================================
  Coverage   96.02%   96.02%           
=======================================
  Files          67       67           
  Lines        4025     4027    +2     
=======================================
+ Hits         3865     3867    +2     
  Misses        160      160           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@pankajastro pankajastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add a unit test to avoid regression?

Copy link
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @andrewhlui, thanks a lot for your contribution!

Some requests:

  1. Please add a test for this feature
  2. If possible, update the docs, mentioning any limitations of this feature (comment inline)

If you can address this before 20 December, we can release this feature as part of Cosmos 1.8

@@ -42,8 +43,10 @@ class DbtDockerBaseOperator(AbstractDbtBaseOperator, DockerOperator): # type: i
def __init__(
self,
image: str, # Make image a required argument since it's required by DockerOperator
profile_config: ProfileConfig | None = None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this profile work with ProfileMapping or only with a given profiles.yml file?

@tatiana tatiana added this to the Cosmos 1.8.0 milestone Dec 10, 2024
@tatiana tatiana requested a review from pankajastro December 12, 2024 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc area:profile Related to ProfileConfig, like Athena, BigQuery, Clickhouse, Spark, Trino, etc size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Docker Execution doesn't work with Task Groups
3 participants