airflow

a DAG describes how you want to carry out your workflow; but notice that we haven’t said anything about what we actually want to do

When searching for DAGs, Airflow only considers python files that contain the strings “airflow” and “DAG” by default. To consider all python files instead, disable the DAG_DISCOVERY_SAFE_MODE configuration flag.

Scope

Airflow will load any DAG object it can import from a DAGfile. Critically, that means the DAG must appear in globals(). Consider the following two DAGs. Only dag_1 will be loaded; the other one only appears in a local scope.

dag_1 = DAG('this_dag_will_be_discovered')
 
def my_function():
    dag_2 = DAG('but_this_dag_will_not')
 
my_function()

Task Lifecycle