Ad

How To Change The Dag Bag Folder For Airflow Web Ui?

Airflow scheduler is picking up the dags from the correct folder as per set in the airflow.cfg file. However, Airflow ui webserver is picking the dags from wrong folder. I realised this via looking at the stdout of the terminal in which airflow webserver is running: [2018-10-08 14:26:48,519] [15535] {models.py:167} INFO - Filling up the DagBag from /home/aviral/airflow/dags

Note, my airflow.cfg file has:

# The folder where your airflow pipelines live, most likely a
# subfolder in a code repository
# This path must be absolute
dags_folder = /home/aviral/dev/airflow_learning/airflow_home/dags
Ad

Answer

It looks to me like you have multiple Airflow installations. I think you have to check the UI in the Admin -> Configuration menu.

That will tell you which Airflow is active and set as AIRFLOW_HOME and AIRFLOW_CONFIG.

If the AIRFLOW_CONFIG environment variable is not set, it will always default to your home directory. You'll have to set it to something like this: AIRFLOW_CONFIG=$AIRFLOW_HOME/airflow.cfg.

Ad
source: stackoverflow.com
Ad