Senior Python Data Engineer

Description

We are seeking a hands-on Senior Python Data Engineer to join the Regulatory Reporting team in Poland within the Investment Bank's Digital Operations Stream, working to transform how the client does business by simplifying and improving the products and platforms relied on by Regulatory and Data Clients. Please note that working from the client's office is required 3 days per week. Responsibilities Design, implement and maintain new and existing data-driven applications Develop and deliver solutions end to end Engineer reliable data pipelines for sourcing, processing, distributing and storing data Design data models that can support ever-growing volumes Contribute to all stages of the software development lifecycle Provide technical expertise, recommendations and innovative solutions Collaborate with the product owner and other technologists Share expertise and best practice with colleagues and contribute regularly to the engineering culture Requirements Hold a Bachelor's degree in Computer Science or equivalent in the IT field with 5+ years of experience in a similar role Background in Big Data, Data warehouse and Data lake Showcase of hands-on experience building scalable data pipelines, processing high volume data in batch and stream for near real-time analytics Expertise in Databricks, Python and Spark Familiarity with Delta Lake, DLT and GitLab Knowledge of Cloud technologies such as Azure, Unix/Linux and shell scripting Expertise designing applications considering NFRs around high throughput and low latency for data processing Expertise building automated functional testing and CI/CD pipelines using industry standard tools Capability to quickly implement prototypes and iterate to harden the product with good engineering principles Hands-on usage of AI in engineering SDLC English proficiency at B2 level or higher Nice to have Proficiency in database development including SQL, PL/SQL, PostgreSQL and Azure SQL

Requirements

• Hold a Bachelor's degree in Computer Science or equivalent in the IT field with 5+ years of experience in a similar role

• Background in Big Data, Data warehouse and Data lake

• Showcase of hands-on experience building scalable data pipelines, processing high volume data in batch and stream for near real-time analytics

• Expertise in Databricks, Python and Spark

• Familiarity with Delta Lake, DLT and GitLab

• Knowledge of Cloud technologies such as Azure, Unix/Linux and shell scripting

• Expertise designing applications considering NFRs around high throughput and low latency for data processing

• Expertise building automated functional testing and CI/CD pipelines using industry standard tools

• Capability to quickly implement prototypes and iterate to harden the product with good engineering principles

• Hands-on usage of AI in engineering SDLC

• English proficiency at B2 level or higher

Responsibilities

• Design, implement and maintain new and existing data-driven applications

• Develop and deliver solutions end to end

• Engineer reliable data pipelines for sourcing, processing, distributing and storing data

• Design data models that can support ever-growing volumes

• Contribute to all stages of the software development lifecycle

• Provide technical expertise, recommendations and innovative solutions

• Collaborate with the product owner and other technologists

• Share expertise and best practice with colleagues and contribute regularly to the engineering culture

Seniority

• Senior

Nice to have

• Proficiency in database development including SQL, PL/SQL, PostgreSQL and Azure SQL