Airflow on Windows with Docker – unable to install or load my own dags

  airflow, docker, windows

I’ve spent an incredible amount of time on this and I believe I’ve now read every single thread or post on this but nothing seems to be helping.

I am trying to run Airflow on my (work) Windows machine using Docker. I’ve tried the following two approaches but without success:

  1. Follow the guide by Airflow, get the yaml file and run compose-up (https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html) – I get the following error:

`Creating airflow_airflow-init_1 … error

ERROR: for airflow_airflow-init_1 Cannot create container for service airflow-init: TCGA-FS01UsersMyNameDocumentsAirflowlogs%!(EXTRA string=is not a valid Windows path`

Nothing that I’ve tried seems to fix this error and I just cannot initiate the actual airflow image. I’m confused as to what the error is referring to because I do not have a "logs%!" folder/file in my Airflow folder – the Airflow folder only has the yaml file and a two dags that I was hoping to load.

  1. Run the puckel version of Airflow. I am able to start this successfully without any issues. If I run the "airflow scheduler" command in the CLI or using bash, I am able to load all of the example dags that come with the puckel airflow container (none will show in the webserver until I ran this).
    Following this guide: http://www.marknagelberg.com/getting-started-with-airflow-using-docker/
    I’ve tried mounting my own local folder to the usr/local/airflow/dags and it seems to mount it successfully again but my own dags never show up. I thought this was an issue with file sharing (and it might be) as my work machine might have some permission set up blocking this but I did get a notification from windows upon mounting that asked me to check that I’m ok with sharing the folder with docker/WSL, making me think that might not be the problem.

So in summary, Option 1 – I get the windows string error and I am unable to proceed with the start up. Option 2 – I cannot get my load my own dags.

I am absolutely lost and if anyone could help me, I would be eternally grateful. Thank you folks.

Source: Windows Questions

LEAVE A COMMENT