Executing Airflow Task on Windows based Apache Aiflow worker

  airflow, python, windows

is it possible to execute Apache Airflow tasks on Windows server?

Our already existing infrastructure heavily depends on Windows and we would rather not recreate the wheel. Me and the team are in the early stages of research and looking for options. The ultimate goal is to replace Alteryx used as a scheduler and Python API.

We are aware that Airflow is not natively supported on Windows. I tried several times to install Airflow on my PC but eventually succeeded using Docker. Is it possible to use server side WSL2 as an Airflow Worker to execute Python scripts in Windows?

Engineers at r/dataengineering suggested:

  1. DockerOperators for windows apps, then push to fargate
  2. BatchOperators
  3. Dagster

Other non-airflow options we have:

  • Azure scheduler
  • VisualCron
  • Jenkins

Thanks for any suggestion!

Source: Windows Questions

LEAVE A COMMENT