Data Engineer Manager

Canary Wharf
1 month ago
Applications closed

Related Jobs

View all jobs

Data Engineer Manager

Azure Data Manager

Data Engineer - Python & Azure

Data Engineer - Unique NFP

Data Engineer (Construction)

Data Manager - New Team - Unique NFP

Data Engineer Manager

Perm

London

£75,000pa - £85,000pa

Role Summary

The Data Engineer Manager is responsible drive the design, development, and optimization of data solutions in the data infrastructure. In addition to fostering the growth of a skilled team, you will play a pivotal role in delivering the data applications, infrastructure, and services, ensuring they align with organizational goals and industry best practices.

As part of the Technology Hub the Data Engineer Manager will work very closely with all teams across the business. The role is instrumental in defining and upholding a clear vision for the integrity of data life cycle management aligning the strategic goal of becoming a centre of expertise. Additionally, it ensures stewardship of business data and technical architecture, fostering innovation and reliability across all data initiatives.

Key Responsibilities



Mentor the data engineering team to design and implement complex, tailored data solutions that support processing of high volumes of data across all schemes and applications.

*

Establish and support the technical vision and strategy for a robust data architecture that aligns with the overall strategy, with a strong focus on ensuring security for all structured data.

*

Establish and maintain robust operational support and monitoring systems to ensure the reliable performance of critical systems in live environments.

*

Facilitate the adoption and implementation of continuous delivery practices while advocating for the use of cloud solutions.

*

Design, implement, and optimize end-to-end data pipelines and solutions on Azure, ensuring data quality, reliability, and security throughout. Oversee the integration of both structured and unstructured data sources.

*

Oversee project timelines, scope, and deliverables to ensure successful execution, while actively monitoring progress and addressing risks proactively.

*

Implement best practices for process improvements, cost optimization and monitoring.

*

Continuously evaluate and improve the Azure data platform to enhance performance and scalability.

*

Collaborate with stakeholders to understand business requirements and translate them into technical solutions.

*

Develop and implement a comprehensive data governance framework to ensure data quality, security, and compliance across the data applications.

*

Design, evaluate impacts, perform technical design reviews, and approve technical designs as part of the design authority process.

*

Establish and maintain comprehensive documentation for all data engineering processes, pipelines, and systems.

*

Implement best practices for maintaining version control and traceability of documentation.

*

Foster continuous learning and adoption of the latest technologies while mentoring and training the data engineering team.

Key Requirements

Essential:

*

Minimum 6 years’ experience in Data Engineering, Data Architecture & Governance frameworks.

*

Minimum 4 years' experience with Python, preferably PySpark.

*

Experience leading small teams of Engineers.

*

Excellent communication and stakeholder management abilities.

*

Strong expertise in Azure: ADLS, Databricks, Stream Analytics, SQL DW, Synapse, Databricks, Azure Functions, Serverless Architecture, ARM T emplates, DevOps.

*

Hands-on experience with ETL/ELT processes and data warehousing.

*

Solid understanding of data security and compliance standards.

*

Experience with DevOps practices and tools (e.g., CI/CD pipelines, Azure DevOps).

*

The ability to simplify complex technical issues for a non-technical stakeholder audience.

*

Capable of understanding business needs and requirements while providing valuable, insightful recommendations.

*

Skilled in delivering presentations and technical reports clearly and persuasively

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Tips for Staying Inspired: How Machine Learning Pros Fuel Creativity and Innovation

Machine learning (ML) continues to reshape industries—from personalised e-commerce recommendations and autonomous vehicles to advanced healthcare diagnostics and predictive maintenance in manufacturing. Yet behind every revolutionary model lies a challenging and sometimes repetitive process: data cleaning, hyperparameter tuning, infrastructure management, stakeholder communications, and constant performance monitoring. It’s no wonder many ML professionals can experience creative fatigue or get stuck in the daily grind. So, how do machine learning experts keep their spark alive and continually generate fresh ideas? Below, you’ll find ten actionable strategies that successful ML engineers, data scientists, and research scientists use to stay innovative and push boundaries. Whether you’re an experienced practitioner or just breaking into the field, these tips can help you fuel creativity and discover new angles for solving complex problems.

Top 10 Machine Learning Career Myths Debunked: Key Facts for Aspiring Professionals

Machine learning (ML) has become one of the hottest fields in technology—touching everything from recommendation engines and self-driving cars to language translation and healthcare diagnostics. The immense potential of ML, combined with attractive compensation packages and high-profile success stories, has spurred countless professionals and students to explore this career path. Yet, despite the boom in demand and innovation, machine learning is not exempt from myths and misconceptions. At MachineLearningJobs.co.uk, we’ve had front-row seats to the real-life career journeys and hiring needs in this field. We see, time and again, that outdated assumptions—like needing a PhD from a top university or that ML is purely about deep neural networks—can mislead new entrants and even deter seasoned professionals from making a successful transition. If you’re curious about a career in machine learning or looking to take your existing ML expertise to the next level, this article is for you. Below, we debunk 10 of the most persistent myths about machine learning careers and offer a clear-eyed view of the essential skills, opportunities, and realistic paths forward. By the end, you’ll be better equipped to make informed decisions about your future in this dynamic and rewarding domain.

Global vs. Local: Comparing the UK Machine Learning Job Market to International Landscapes

How to evaluate opportunities, salaries, and work culture in machine learning across the UK, the US, Europe, and Asia Machine learning (ML) has rapidly transcended the research labs of academia to become a foundational pillar of modern technology. From recommendation engines and autonomous vehicles to fraud detection and personalised healthcare, machine learning techniques are increasingly ubiquitous, transforming how organisations operate. This surge in applications has fuelled an extraordinary global demand for ML professionals—data scientists, ML engineers, research scientists, and more. In this article, we’ll examine how the UK machine learning job market compares to prominent international hubs, including the United States, Europe, and Asia. We’ll explore hiring trends, salary ranges, workplace cultures, and the nuances of remote and overseas roles. Whether you’re a fresh graduate aiming to break into the field, a software engineer with an ML specialisation, or a seasoned professional seeking your next challenge, understanding the global ML landscape is essential for making an informed career move. By the end of this overview, you’ll be equipped with insights into which regions offer the best blend of salaries, work-life balance, and cutting-edge projects—plus practical tips on how to succeed in a domain that’s constantly evolving. Let’s dive in.