Test DAGs
Effectively testing DAGs requires an understanding of their structure and their relationship to other code and data in your environment. In this guide, you'll learn about DAG validation testing, unit testing, and data and pipeline integrity testing.
Custom XCom backends
Creating a custom XCom backend with Airflow 2.0.
Airflow logging
An introduction to Airflow logging.
KubernetesPodOperator
Use the KubernetesPodOperator in Airflow to run tasks in Kubernetes Pods
Data quality
Check the quality of your data using Airflow.
SQL check operators
Executing queries in Apache Airflow DAGs to ensure data quality.
Dynamically generate DAGs
In Airflow, DAGs are defined as Python code. Airflow executes all Python code in the dags_folder and loads any DAG objects that appear in globals(). The simplest way to create a DAG is to write it as a static Python file.
Dynamic tasks
How to dynamically create tasks at runtime in your Airflow DAGs.
Datasets and data-aware scheduling
Using datasets to implement DAG dependencies and scheduling in Airflow.
Deferrable operators
Implement deferrable operators to save cost and resources with Airflow.