Python Technical Lead

Description

We are seeking a highly skilled Python Technical Lead to join our dynamic team, driving the design and development of distributed backend solutions while leading a team of 5+ data engineers to deliver API backends for deploying and querying various models on Azure Databricks. Responsibilities Lead design, development and deployment of backend platform capabilities in a restricted Azure Greenfield environment that will host the models Design an API to allow querying of the models by users Leverage Azure Databricks to host and deploy models Establish process and guidelines for model deployment, including how to wrap the model in an MLflow wrapper and what the API should look like Implement best practices for CI/CD using GitLab, ensure code quality, testing, and deployment automation Lead, mentor, and manage a team of 5+ engineers, foster a collaborative and high-performance environment Requirements 8+ years of experience in data engineering, with at least 2+ years in a leadership role Expertise in Python and key data engineering libraries, including Python Core, Multithreading, Async, REST API, Exceptions, and Pydantic Strong experience with Azure cloud services, including Azure Databricks, Azure App Service, and IAM/RBAC Proficiency in CI/CD workflows, version control (GitLab), and DevOps practices Familiarity with containerization and orchestration, including Docker and Kubernetes Strong problem-solving skills and ability to work in a fast-paced, agile environment Excellent communication, leadership, and stakeholder management skills

Requirements

• 8+ years of experience in data engineering, with at least 2+ years in a leadership role

• Expertise in Python and key data engineering libraries, including Python Core, Multithreading, Async, REST API, Exceptions, and Pydantic

• Strong experience with Azure cloud services, including Azure Databricks, Azure App Service, and IAM/RBAC

• Proficiency in CI/CD workflows, version control (GitLab), and DevOps practices

• Familiarity with containerization and orchestration, including Docker and Kubernetes

• Strong problem-solving skills and ability to work in a fast-paced, agile environment

• Excellent communication, leadership, and stakeholder management skills

Responsibilities

• Lead design, development and deployment of backend platform capabilities in a restricted Azure Greenfield environment that will host the models

• Design an API to allow querying of the models by users

• Leverage Azure Databricks to host and deploy models

• Establish process and guidelines for model deployment, including how to wrap the model in an MLflow wrapper and what the API should look like

• Implement best practices for CI/CD using GitLab, ensure code quality, testing, and deployment automation

• Lead, mentor, and manage a team of 5+ engineers, foster a collaborative and high-performance environment

Seniority

• Lead