Senior Apache Airflow Engineer

Description

We are searching for a Senior Apache Airflow Engineer to strengthen and grow our data orchestration ecosystem. Within this role, you will take part in crafting fresh Airflow components, guaranteeing their smooth alignment with Cloud Composer and providing dependable answers for sophisticated data pipelines. Please note that working from the office in Gdansk or Wroclaw 3 days per week is required. Responsibilities Construct and sustain Airflow Operators, Hooks and Sensors Incorporate the latest Airflow releases into Cloud Composer Resolve defects within Airflow and deploy refinements Craft and run tests for Airflow across Unit Testing and Solution Testing Compose internal documentation that describes delivered work artifacts Partner with Open Source communities to exchange contributions and upgrades Requirements 3+ years of practical work with Python Hands-on programming background involving Cloud APIs (REST/RPC) Foundational grasp of DevOps/SysAdmin practices Working command of Linux and Kubernetes, such as assembling container images Familiarity with Cloud platforms Expertise in Airflow or equivalent background in assembling workflows (e.g. based on Luigi, Oozie, Argo, Azkaban, Apache HiFi) Experience of contributing alongside Open Source communities Solid proficiency in English communication, spoken and written (B2+) Nice to have Understanding of Cloud Architecture principles, covering scalability and IaaS concepts

Requirements

• 3+ years of practical work with Python

• Hands-on programming background involving Cloud APIs (REST/RPC)

• Foundational grasp of DevOps/SysAdmin practices

• Working command of Linux and Kubernetes, such as assembling container images

• Familiarity with Cloud platforms

• Expertise in Airflow or equivalent background in assembling workflows (e.g. based on Luigi, Oozie, Argo, Azkaban, Apache HiFi)

• Experience of contributing alongside Open Source communities

• Solid proficiency in English communication, spoken and written (B2+)

Zakres obowiązków

• Construct and sustain Airflow Operators, Hooks and Sensors

• Incorporate the latest Airflow releases into Cloud Composer

• Resolve defects within Airflow and deploy refinements

• Craft and run tests for Airflow across Unit Testing and Solution Testing

• Compose internal documentation that describes delivered work artifacts

• Partner with Open Source communities to exchange contributions and upgrades

Seniority

• Senior

Mile widziane

• Understanding of Cloud Architecture principles, covering scalability and IaaS concepts