Senior Data Engineer

Description

The primary goal of the project is the modernization, maintenance and development of an eCommerce platform for a big US-based retail company, serving millions of omnichannel customers each week. Solutions are delivered by several Product Teams focused on different domains - Customer, Loyalty, Search and Browse, Data Integration, Cart. Current overriding priorities are new brands onboarding, re-architecture, database migrations, migration of microservices to a unified cloud-native solution without any disruption to business.

What we offer

• Global Relocation - (Relocation options; Experience in an international environment; Cross-cultural experience)

• Recognition and Evaluation - (Feedback culture; Regular appraisals)

• Time Off - (Annual holiday - 20 or 26 days. The duration of the leave depends on the overall seniority; Occasional leave - 1 or 2 days/ depending on the circumstances; Child care leave - 2 days or 16 hours per year; Absence due to force majeure - 2 days or 16 hours per year; Maternity Leave - 20 weeks; Parental Leave - 41 weeks; Paternity Leave - 14 days)

• Luxoft Training Center - (Expert-led tech courses covering basic to advanced topics; Internal instructor-led soft skills courses; Comprehensive in-house self-learning resources for both soft and hard skills; Access to external self-learning libraries like ProQuest eBook and Udemy for Business; Cloud Programs: MS Cloud Academy, AWS Partner Academy, Google Cloud Academy; Custom Learning Programs: upskilling, reskilling, technical mentorship; Leadership Programs for Managers)

• Well-being and Work-life Balance - (Multisport card; Possibility to order Multisport card at the corporate rate for family members; LuxGood Program: wellbeing seminars, contests, relaxation sessions, yoga sessions, etc.; One Team Program: Buddy for each New Joiner; seminars, meeting and workplace space to support integration with local community and culture; “Hire me” workshops for partners; Preferential banking offer; Preferential car leasing offer; Cafeteria program discounts for shops, cinema tickets, holiday offers; Luxoft Social Benefit Fund: sport and recreation benefits, the possibility to receive financial support)

• Health Care - (Private Healthcare Insurance with unlimited access to specialists; Full dental support; Travel Insurance; Possibility to add private healthcare coverage for family members at the corporate rate; Life insurance at the corporate rate for employees and family members, including payment of the basic package for the employee by the employer; Reimbursement for corrective glasses)

• Company Events and Friendly Environment - (Many fun social activities organized by the Luxoft team offline in your city; Online entertainment events for whole company and local team events; A workplace where you’re treated with respect within a multicultural team)

• Internal Mobility - (Rotation between projects and accounts; New career opportunities)

• Self-Learning Library

• CSR Projects

Other

• Languages: English: B2 Upper Intermediate

• Seniority: Lead

Requirements

• Readiness to work until 8.00 pm CET (no need to do overtimes)

• Overall years of experience required 8+ (at least 1+ year in a Lead/Architect position)

• Strong, recent hands-on expertise with Azure Data Factory and Synapse is a must (3+ years).

• Strong expertise in designing and implementing data models, including conceptual, logical, and physical data models, to support efficient data storage and retrieval.

• Strong knowledge of Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks, pySpark for building scalable and reliable data solutions.

• Extensive experience with building robust and scalable ETL/ELT pipelines to extract, transform, and load data from various sources into data lakes or data warehouses.

• Ability to integrate data from disparate sources, including databases, APIs, and external data providers, using appropriate techniques such as API integration or message queuing.

• Proficiency in designing and implementing data warehousing solutions (dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, Data Vault)

• Proficiency in SQL to perform complex queries, data transformations, and performance tuning on cloud-based data storages.

• Experience integrating metadata and governance processes into cloud-based data platforms

• Certification in Azure, Databricks, or other relevant technologies is an added advantage

• Experience with cloud-based analytical databases.

• Experience with Azure MI, Azure Database for Postgres, Azure Cosmos DB, Azure Analysis Services, and Informix.

• Experience with Python and Python-based ETL tools.

• Experience with shell scripting in Bash, Unix or windows shell is preferable.

• Demonstrated ability to lead cross-functional engineering teams, define technical strategy and architecture, drive delivery of complex data platforms, mentor engineers, and effectively communicate with stakeholders at all organizational levels.

• Experience with Elasticsearch

• Familiarity with containerization and orchestration technologies (Docker, Kubernetes).

• Troubleshooting and Performance Tuning: Ability to identify and resolve performance bottlenecks in data processing workflows and optimize data pipelines for efficient data ingestion and analysis.

• Collaboration and Communication: Strong interpersonal skills to collaborate effectively with stakeholders, data engineers, data scientists, and other cross-functional teams.

• Ability to plan, estimate and track progress of implementing features

• Computer Science and data science academic and education credentials

Responsibilities

• We are looking for Data Engineer who will be responsible for designing a solution for a big retail company. The main focus is to support processing of big data volumes and integrate solution to current architecture.