Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dag randering for taskflow + DbtTaskGroup combo #1360

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

pankajastro
Copy link
Contributor

@pankajastro pankajastro commented Dec 2, 2024

Description

Screenshot 2024-12-03 at 2 30 23 AM

DAG Code

from datetime import datetime

from airflow.decorators import task, dag
from cosmos import DbtTaskGroup, ProjectConfig
from include.constants import jaffle_shop_path, venv_execution_config, manifest_path
from include.profiles import airflow_db

@task(task_id="build_partial_dbt_env_vars_operator")
def build_partial_dbt_env():
    # some code
    # This return is for demonstration purposes only
    return {
        "ENV_VAR_NAME": "value",
        "ENV_VAR_NAME_2": False
    }


# partial_dbt_env = build_partial_dbt_env()



@dag(
    schedule_interval="@daily",
    start_date=datetime(2023, 1, 1),
    catchup=False,
    tags=["simple"],
)
def simple_task_group1() -> None:
    DbtTaskGroup(
        group_id="transform_task_group",
        project_config=ProjectConfig(
        dbt_project_path=jaffle_shop_path,
        manifest_path=manifest_path,
        env_vars=build_partial_dbt_env()
    ),
        profile_config=airflow_db,
        execution_config=venv_execution_config,
    )

simple_task_group1()
#partial_dbt_env >> transform_task_group

Related Issue(s)

closes: #1218

Breaking Change?

Checklist

  • I have made corresponding changes to the documentation (if required)
  • I have added tests that prove my fix is effective or that my feature works

Copy link

cloudflare-workers-and-pages bot commented Dec 2, 2024

Deploying astronomer-cosmos with  Cloudflare Pages  Cloudflare Pages

Latest commit: 50d7c56
Status: ✅  Deploy successful!
Preview URL: https://f2800f4a.astronomer-cosmos.pages.dev
Branch Preview URL: https://fix-dag-rendering.astronomer-cosmos.pages.dev

View logs

Copy link

codecov bot commented Dec 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.06%. Comparing base (a735b16) to head (50d7c56).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1360   +/-   ##
=======================================
  Coverage   96.06%   96.06%           
=======================================
  Files          67       67           
  Lines        4042     4042           
=======================================
  Hits         3883     3883           
  Misses        159      159           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pankajastro pankajastro changed the title Fix dag randering for config using from upstream task Fix dag randering for taskflow + DbtTaskGroup combo Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Tried to set relationships between tasks in more than one DAG
1 participant