Technical Lead; Data Engineering - London/Hybrid - Energy/Trading

London
6 months ago
Applications closed

Related Jobs

View all jobs

Data Engineering Lead

Head of Data Engineering

Staff/Lead Data Engineer

Head of Data Engineering

Senior Data Engineer

Data Engineering Lead

Job Title: Technical Lead - Data Engineering
Location: London/Hybrid- Onsite 2-3 days per week, with flexibility to work remotely once up and running
Salary/Rate: Up to £580 per day INSIDE IR35
Start Date: As soon as possible
Job Type: 12 month contract initially + extensions

Company Introduction

Coltech is partnered with a prestigious consultancy client who are looking for an experienced Technical Lead in Data Engineering to lead the development of the data Lakehouse platform

Job Responsibilities

  • Lead the design, development, and implementation of data engineering solutions for our data Lakehouse platform.

  • Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.

  • Provide technical leadership, mentoring, and coaching to the data engineering team.

  • Ensure best practices in data engineering, including data modelling, ETL processes, and data quality.

  • Optimize big data workloads in Spark and other big data technologies.

  • Manage CI/CD pipelines using Azure DevOps or Git.

  • Develop and maintain event-driven pipelines using .NET and other relevant technologies.

  • Implement and manage Databricks Unity Catalog and other data governance tools.

  • Deliver projects in the streaming data world using Kafka, KSQL DB, and similar technologies.

  • Utilize reporting tools such as Power BI and Qlik for data visualization and reporting.

    Required Skills/Experience:

  • Extensive experience with Databricks, Apache Spark, and data lakes.

  • Proficiency in Python and SQL.

  • Experience with CI/CD processes using Azure DevOps or Git.

  • Knowledge of streaming data technologies such as Kafka and KSQL DB.

  • Familiarity with reporting tools like Power BI and Qlik.

  • Software engineering experience with .NET for event-driven pipelines and automation testing.

  • Experience with Databricks Unity Catalog.

  • Strong skills in data modelling, including Snowflake modelling, star schema modelling, and other techniques.

  • Ability to optimize big data workloads in Spark.

    Desired Skills/Experience:

  • Experience in delivering projects in the streaming data world.

  • Understanding of data governance and data quality best practices.

  • Familiarity with automation testing frameworks.

  • Experience working in Energy trading related projects.

    Apply now for immediate consideration

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Careers Education

Top 10 Best UK Universities for Machine Learning Degrees (2025 Guide)

Explore ten UK universities that deliver world-class machine-learning degrees in 2025. Compare entry requirements, course content, research strength and industry links to find the programme that fits your goals. Machine learning (ML) has shifted from academic curiosity to the engine powering everything from personalised medicine to autonomous vehicles. UK universities have long been pioneers in the field, and their programmes now blend rigorous theory with hands-on practice on industrial-scale datasets. Below, we highlight ten institutions whose undergraduate or postgraduate pathways focus squarely on machine learning. League tables move each year, but these universities consistently excel in teaching, research and collaboration with industry.

Jobs

How to Write a Winning Cover Letter for Machine Learning Jobs: Proven 4-Paragraph Structure

Learn how to craft the perfect cover letter for machine learning jobs with this proven 4-paragraph structure. Ideal for entry-level candidates, career switchers, and professionals looking to advance in the machine learning sector. When applying for a machine learning job, your cover letter is a vital part of your application. Machine learning is an exciting and rapidly evolving field, and your cover letter offers the chance to demonstrate your technical expertise, passion for AI, and your ability to apply machine learning techniques to solve real-world problems. Writing a cover letter for machine learning roles may feel intimidating, but by following a clear structure, you can showcase your strengths effectively. Whether you're just entering the field, transitioning from another role, or looking to advance your career in machine learning, this article will guide you through a proven four-paragraph structure. We’ll provide practical tips and sample lines to help you create a compelling cover letter that catches the attention of hiring managers in the machine learning job market.

Jobs Careers

Veterans in Machine Learning: A Military‑to‑Civilian Pathway into AI Careers

Introduction Artificial intelligence is no longer relegated to sci‑fi films—it underpins battlefield decision‑support, fraud detection, and even supermarket logistics. The UK Government’s 2025 AI Sector Deal forecasts an additional £200 billion in GDP by 2030, with machine‑learning (ML) engineers cited as the nation’s second most in‑demand tech role (Tech Nation 2024). The Ministry of Defence’s Defence AI Strategy echoes that urgency, earmarking £1.6 billion for FY 2025–28 to embed ML into planning, logistics, and autonomous systems. If you have ever tuned a radar filter, plotted artillery trajectories, or sifted sensor data for actionable intel, you have already worked with statistical modelling—the backbone of machine learning. This guide shows UK veterans how to reframe military experience for ML roles, leverage MoD transition funding, and land high‑impact positions building the models shaping tomorrow’s defence and commercial landscapes. Quick Win: Bookmark our live board for Machine‑Learning Engineer roles to see who’s hiring today.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.