Lead Data Engineer

Lead Data Engineer | Location: London (Hybrid - 1-2 day a month in office) | Salary: £90,000-£110,000

The Opportunity

Are you passionate about building cutting-edge data platforms that drive business growth? Our client is seeking a skilled and motivatedLeadData Engineerto play a key role in the creation of abrand-new data platformwithin theAzure ecosystemandDatabricks.

This is an exciting opportunity to be at the forefront of data innovation, working within a newly formedData & Analyticsteam in a long standing London Market Insurer. You’ll help shape the data strategy, improve data quality, and empower the business to make data-driven decisions.

As a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize ahigh-performance data platformbuilt onDatabricks. This platform will be built to scale, incorporating the latest advancements in data intelligence while supporting strategic business objectives.

Key Responsibilities

🔹Build & Develop– Design and maintain a robustDatabricksData Platform, ensuring performance, scalability, and availability.

🔹Data Pipelines– Connect APIs, databases, and data streams to the platform, implementing ETL/ELT processes.

🔹Data Integrity– Embed quality measures, monitoring, and alerting mechanisms.

🔹CI/CD & Automation– Create deployment pipelines and automate workflows.

🔹Collaboration– Work with stakeholders acrossGlobal IT, Data, and Brokingteams to translate business requirements into technical solutions.

🔹Futureproofing– Drive the evolution of the data platform, ensuring adaptability for new data sources, analytical models, and emerging technologies.

What You’ll Bring

✅Extensive hands-on experiencewithDatabricks, andMicrosoft Azure data tools(must-have: Azure Data Factory, Azure Synapse, or Azure SQL).

✅Dimensional modellingexpertise for analytics use cases.

✅ StrongETL/ELTdevelopment skills.

✅Python scriptingexperience for data automation.

✅ Experience withCI/CD methodologiesfor data platforms.

✅ Knowledge ofMS SQL Server, SSIS, Visual Studio, and SSDT projects.

✅ Previous London Market Insurance experience

Why Join?

🚀Greenfield Project– Work on an all-new data platform, shaping its architecture from the ground up.

🌍Collaborative Culture– Engage with global teams in an agile, innovative environment.

📈Career Growth– Play a pivotal role in driving data excellence within a forward-thinking business.

💡Cutting-Edge Tech– Work with the latest advancements inAzure, Databricks, and Data Engineering.

This is a fantastic opportunity for aLead Data Engineerlooking to make a tangible impact. If you’re ready to take on a challenging and rewarding role, apply today!