Support building DAGs from topologically unsorted YAML files

[tatiana]

Any YAML files that declare upstream tasks after downstream tasks, regardless of using dynamic task mapping, would fail.

Example of DAG that would fail:

test_expand:
  default_args:
    owner: "custom_owner"
    start_date: 2 days
  description: "test expand"
  schedule_interval: "0 3 * * *"
  default_view: "graph"
  tasks:
    process:
      operator: airflow.operators.python_operator.PythonOperator
      python_callable_name: expand_task
      python_callable_file: $CONFIG_ROOT_DIR/expand_tasks.py
      partial:
        op_kwargs:
          test_id: "test"
      expand:
        op_args:
          request.output
      dependencies: [request]
    request:
      operator: airflow.operators.python.PythonOperator
      python_callable_name: example_task_mapping
      python_callable_file: $CONFIG_ROOT_DIR/expand_tasks.py

In this example, the upstream (parent) task "request" is defined after the downstream (child) task "process". Before this change, this DAG would fail.

I implemented a solution to solve the problem that uses Kahn's algorithm to sort the tasks topologically:
https://en.wikipedia.org/wiki/Topological_sorting#Kahn's_algorithm

It has asymptotic complexity O(N + D), where N is the total number of tasks, and D is the total number of dependencies. This complexity seems acceptable.

An alternative to the current approach would be to create all the tasks without dependencies as a starting point and add the dependencies once all tasks were made - similar to what we did in https://github.com/astronomer/astronomer-cosmos. However, this approach would require a bigger refactor of the DAG factory and may have issues with dynamic task mapping.

Closes: #225

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.70%. Comparing base (2e9fd4e) to head (0ef4f66).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #307      +/-   ##
==========================================
+ Coverage   92.45%   92.70%   +0.25%     
==========================================
  Files          10       10              
  Lines         702      727      +25     
==========================================
+ Hits          649      674      +25     
  Misses         53       53              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.