Lead Apache Airflow Engineer
Gdańsk, Pomeranian Voivodeship, Polska, Wrocław, Lower Silesian Voivodeship, Polska
Key offer highlights
Min. 1 year of experience
Backend: Java / .NET / Node / Python
DevOps / Cloud: AWS, Azure, Docker, Kubernetes
Employment: contract of employment
Hybrid model - partly remote
Description
We are seeking a Lead Apache Airflow Engineer to spearhead the development and enhancement of our workflow orchestration platform. In this role, you will drive the evolution of Airflow components, integrate new versions with Cloud Composer and lead a talented team of engineers in building robust data pipelines. Please note that working from the office in Gdansk or Wroclaw 3 days per week is required. Responsibilities Develop and maintain Airflow Operators, Hooks and Sensors Integrate new Airflow versions with Cloud Composer Fix bugs in Airflow and implement improvements Implement tests for Airflow, including Unit Testing and Solution Testing Create internal documentation for work artifacts Lead and mentor a team of Airflow engineers Collaborate with Open Source communities to contribute improvements Requirements 5+ years of experience in Python 1+ years of relevant leadership experience Programming experience with Cloud APIs (REST/RPC) Basic knowledge of DevOps/SysAdmin Basic understanding of Linux and Kubernetes, including how to build container images Familiarity with Cloud platforms Expertise in Airflow or similar experience in building workflows (e.g. based on Luigi, Oozie, Argo, Azkaban, Apache HiFi) Experience working with Open Source communities Communication skills in English, spoken and written (B2+) Nice to have Understanding of Cloud Architecture, including scalability and IaaS principles
Requirements
5+ years of experience in Python
1+ years of relevant leadership experience
Programming experience with Cloud APIs (REST/RPC)
Basic knowledge of DevOps/SysAdmin
Basic understanding of Linux and Kubernetes, including how to build container images
Familiarity with Cloud platforms
Expertise in Airflow or similar experience in building workflows (e.g. based on Luigi, Oozie, Argo, Azkaban, Apache HiFi)
Experience working with Open Source communities
Communication skills in English, spoken and written (B2+)
Zakres obowiązków
Develop and maintain Airflow Operators, Hooks and Sensors
Integrate new Airflow versions with Cloud Composer
Fix bugs in Airflow and implement improvements
Implement tests for Airflow, including Unit Testing and Solution Testing
Create internal documentation for work artifacts
Lead and mentor a team of Airflow engineers
Collaborate with Open Source communities to contribute improvements
Seniority
Lead
Mile widziane
Understanding of Cloud Architecture, including scalability and IaaS principles
Keywords / Skills