Lead Apache Airflow Engineer

Description

We are seeking a Lead Apache Airflow Engineer to drive the development and maintenance of Airflow-based workflows, guide technical delivery, and mentor team members while collaborating with cross-functional and distributed teams. Responsibilities Develop new and maintain existing Airflow Operators, Hooks and Sensors Integrate new Airflow versions with Cloud Composer Fix bugs in Airflow and implement improvements Implement tests for Airflow, including Unit Testing and Solution Testing Create internal documentation for the work artifacts Coordinate development activities within the team and support delivery planning Collaborate with cross-functional and distributed teams to support delivery and resolve technical issues Support junior team members through knowledge sharing, technical guidance, and code review activities Participate in technical discussions with client, prioritization, and coordination of development activities within the team Requirements 7+ years of experience with strong Python programming skills Experience with Cloud APIs (REST/RPC) Knowledge of DevOps/SysAdmin practices, Linux, and Kubernetes, including building container images Understanding of Cloud platforms, especially Google Cloud Platform Expertise in Airflow or similar experience in building workflows based on Luigi, Oozie, Argo, Azkaban, or Apache NiFi Experience working with open-source communities Proficiency in English communication, both spoken and written Nice to have Understanding of cloud architecture, including scalability and IaaS principles Google Cloud Certification (e.g., GCP Architect or Data Engineer)

Requirements

• 7+ years of experience with strong Python programming skills

• Experience with Cloud APIs (REST/RPC)

• Knowledge of DevOps/SysAdmin practices, Linux, and Kubernetes, including building container images

• Understanding of Cloud platforms, especially Google Cloud Platform

• Expertise in Airflow or similar experience in building workflows based on Luigi, Oozie, Argo, Azkaban, or Apache NiFi

• Experience working with open-source communities

• Proficiency in English communication, both spoken and written

Responsibilities

• Develop new and maintain existing Airflow Operators, Hooks and Sensors

• Integrate new Airflow versions with Cloud Composer

• Fix bugs in Airflow and implement improvements

• Implement tests for Airflow, including Unit Testing and Solution Testing

• Create internal documentation for the work artifacts

• Coordinate development activities within the team and support delivery planning

• Collaborate with cross-functional and distributed teams to support delivery and resolve technical issues

• Support junior team members through knowledge sharing, technical guidance, and code review activities

• Participate in technical discussions with client, prioritization, and coordination of development activities within the team

Seniority

• Lead

Nice to have

• Understanding of cloud architecture, including scalability and IaaS principles

• Google Cloud Certification (e.g., GCP Architect or Data Engineer)