Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] To support task display_name #1277

Open
1 task done
t0momi219 opened this issue Oct 23, 2024 · 2 comments · May be fixed by #1278
Open
1 task done

[Feature] To support task display_name #1277

t0momi219 opened this issue Oct 23, 2024 · 2 comments · May be fixed by #1278
Labels
area:rendering Related to rendering, like Jinja, Airflow tasks, etc enhancement New feature or request stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed triage-needed Items need to be reviewed / assigned to milestone
Milestone
Cosmos 1.8.0

Comments

@t0momi219
Copy link
Contributor

t0momi219 commented Oct 23, 2024 •
edited
Loading

Description

When running models that have names containing multibyte characters, runtime errors occur in Airflow environments where statsd is enabled (MWAA uses this statsd metric for collecting metrics in Cloudwatch).

Related Issue: apache/airflow#18010

To address this, Airflow 2.9 introduced the ability to render tasks using display_name, which allows task names to be rendered separately from their task_id.

Reference: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/example_dags/example_display_name.html

It would be ideal to have a mechanism that allows users to set a different task_id while using the model name as the display_name when working with models that contain multibyte characters.

Use case/motivation

The following error occurs in MWAA environment:

[2024-10-17, 10:27:03 UTC] {taskinstance.py:2865} INFO - Starting attempt 1 of 1
[2024-10-17, 10:27:03 UTC] {validators.py:135} ERROR - Invalid stat name: dag.run_cosmos_dbt.日本語名モデル_run.queued_duration.
Traceback (most recent call last):
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/metrics/validators.py", line 132, in wrapper
    stat = handler_stat_name_func(stat)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/metrics/validators.py", line 220, in stat_name_default_handler
    raise InvalidStatsNameException(
airflow.exceptions.InvalidStatsNameException: The stat name (dag.run_cosmos_dbt.日本語名モデル_run.queued_duration) has to be composed of ASCII alphabets, numbers, or the underscore, dot, or dash characters.

Related issues

No response

Are you willing to submit a PR?

  • Yes, I am willing to submit a PR!
@t0momi219 t0momi219 added enhancement New feature or request triage-needed Items need to be reviewed / assigned to milestone labels Oct 23, 2024
@t0momi219 t0momi219 linked a pull request Oct 23, 2024 that will close this issue
Open
2 tasks
@dosubot dosubot bot added the area:rendering Related to rendering, like Jinja, Airflow tasks, etc label Oct 23, 2024
@tatiana
Copy link
Collaborator

tatiana commented Oct 29, 2024

Thanks for logging and proposing a fix for this, @t0momi219 ! We're planning to release Cosmos 1.8 in mid-November, and it will be great to have this feature in it.

@tatiana tatiana added this to the Cosmos 1.8.0 milestone Oct 29, 2024
@github-actions GitHub Actions
Copy link

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:rendering Related to rendering, like Jinja, Airflow tasks, etc enhancement New feature or request stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed triage-needed Items need to be reviewed / assigned to milestone
Projects
None yet
Milestone
Cosmos 1.8.0
Development

Successfully merging a pull request may close this issue.

To support task display_name t0momi219/astronomer-cosmos
2 participants
@tatiana @t0momi219