Select and complete Apache Airflow Tutorial

[krlberry]

Note: This is essentially a sub-task of #9

Objective

Set up Apache Airflow for local use, complete a set of Airflow tutorials to understand key concepts, and document key take-aways from the tutorials.

---

Requirements

1. Explore and decide on suitable Apache Airflow tutorials from the official Airflow documentation or community resources.
2. Team members set up Airflow for local use (possibly as part of the tutorial)
3. Complete the selected tutorials to understand key concepts, including:
   • Tasks and DAGs (Directed Acyclic Graphs)
   • Task dependencies
   • Scheduling
   • Error handling and retries

---

Definition of Done

• Airflow tutorials are identified, agreed upon, and completed by all team members.
• Airflow is installed locally for all team members
• Key learnings and notes from the tutorials are shared with the team via a GitHub Wiki page

---

Key Decision Points

1. Tutorial Selection:
   • Choose relevant tutorials from Apache Airflow documentation or community guides.

---

Artifacts

• Notes and resources from completed tutorials.

---

Success Criteria

• The team has identified and completed a set of Apache Airflow tutorials that cover its core features including reliability and fault tolerance.
• Team members can run simple workflows from the tutorials locally
• Key take aways from the tutorials have been documented on a GitHub wiki page.

---

Potential Challenges

• Identifying tutorials that are both comprehensive and time-efficient.
• There is always the possibility that setting up Apache Airflow to work locally for all team members could be time-consuming or difficult in unexpected ways.