Lead Data Engineer

Birmingham
2 weeks ago
Create job alert

Lead Data Engineer - Databricks

Salary: £70-£95k (Dependant on Experience)

Location: Can be based anywhere in the UK (Largely remote, must be willing to travel weekly)

Are you an experienced Data Engineer with a passion for leading teams and delivering high-quality solutions?

We are looking for a Lead Data Engineer to join our dynamic consultancy customer.

This role is perfect for someone with a strong technical background in Databricks and Azure Analytics, combined with excellent leadership skills.

Key Responsibilities:

  • Databricks Expertise: Implement and optimize workloads in Databricks, with a full understanding of features, access controls, security, and networking. Experience with Unity Catalog is essential.

  • Team Leadership: Lead and mentor a team of internal and client engineers, setting high standards in coding, performance, and solution design.

  • Solution Architecture: Design and implement scalable data solutions using Azure Data Factory, Storage, Key Vault, Databricks, and/or Fabric Engineering.

  • Customer Engagement: Communicate complex technical solutions to non-technical stakeholders with clarity and confidence.

  • Technical Community Presence: Stay active in the data engineering community by contributing to events, blogs, or open-source initiatives.

  • On-Site Collaboration: Willingness to travel to client sites one day per week (expenses covered).

    Required Skills & Experience:

  • Expert-level proficiency in Python and Apache Spark.

  • Proven experience in Databricks, with a strong understanding of Unity Catalog, ingestion methods, and CI/CD.

  • Strong hands-on experience with Azure Data Factory, Key Vault, Storage, networking concepts, and Databricks (Microsoft Fabric a plus!)

  • Leadership experience with the ability to shape best practices and develop engineering teams.

  • Excellent communication and stakeholder management skills, able to bridge the gap between technical and non-technical audiences.

  • Ability to lead and run projects, and engage with key business stakeholders externally and internally.

    Join a team that values innovation, collaboration, and professional growth.

    Apply now to take the next step in your career

Related Jobs

View all jobs

Lead Data Engineer

Lead Data Engineer

Lead Data Engineer

Lead Data Engineer

Lead Data Engineer

Lead Data Engineer

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Top 10 Machine Learning Career Myths Debunked: Key Facts for Aspiring Professionals

Machine learning (ML) has become one of the hottest fields in technology—touching everything from recommendation engines and self-driving cars to language translation and healthcare diagnostics. The immense potential of ML, combined with attractive compensation packages and high-profile success stories, has spurred countless professionals and students to explore this career path. Yet, despite the boom in demand and innovation, machine learning is not exempt from myths and misconceptions. At MachineLearningJobs.co.uk, we’ve had front-row seats to the real-life career journeys and hiring needs in this field. We see, time and again, that outdated assumptions—like needing a PhD from a top university or that ML is purely about deep neural networks—can mislead new entrants and even deter seasoned professionals from making a successful transition. If you’re curious about a career in machine learning or looking to take your existing ML expertise to the next level, this article is for you. Below, we debunk 10 of the most persistent myths about machine learning careers and offer a clear-eyed view of the essential skills, opportunities, and realistic paths forward. By the end, you’ll be better equipped to make informed decisions about your future in this dynamic and rewarding domain.

Global vs. Local: Comparing the UK Machine Learning Job Market to International Landscapes

How to evaluate opportunities, salaries, and work culture in machine learning across the UK, the US, Europe, and Asia Machine learning (ML) has rapidly transcended the research labs of academia to become a foundational pillar of modern technology. From recommendation engines and autonomous vehicles to fraud detection and personalised healthcare, machine learning techniques are increasingly ubiquitous, transforming how organisations operate. This surge in applications has fuelled an extraordinary global demand for ML professionals—data scientists, ML engineers, research scientists, and more. In this article, we’ll examine how the UK machine learning job market compares to prominent international hubs, including the United States, Europe, and Asia. We’ll explore hiring trends, salary ranges, workplace cultures, and the nuances of remote and overseas roles. Whether you’re a fresh graduate aiming to break into the field, a software engineer with an ML specialisation, or a seasoned professional seeking your next challenge, understanding the global ML landscape is essential for making an informed career move. By the end of this overview, you’ll be equipped with insights into which regions offer the best blend of salaries, work-life balance, and cutting-edge projects—plus practical tips on how to succeed in a domain that’s constantly evolving. Let’s dive in.

Machine Learning Leadership for Managers: Strategies to Motivate, Mentor, and Set Realistic Goals in Data-Driven Teams

Machine learning (ML) has become an indispensable force in the modern business world, influencing everything from targeted marketing campaigns to advanced medical diagnostics. As industries integrate predictive algorithms and data-driven decision-making into their core operations, the need for effective leadership in machine learning environments has never been greater. Whether you’re overseeing a small team of data scientists or spearheading an enterprise-scale ML project, your leadership style must accommodate rapid innovation, complex problem-solving, and diverse stakeholder expectations. This guide provides actionable insights into how you can motivate, mentor, and establish achievable goals for your machine learning teams—ensuring they thrive in data-driven environments.