National AI Awards 2025Discover AI's trailblazers! Join us to celebrate innovation and nominate industry leaders.

Nominate & Attend

Senior Data Engineer

Insight Global
Manchester
1 month ago
Applications closed

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer - Azure - Leeds

Senior Data Engineer Consultant

Senior Data Engineer - Clayton-le-Moors

Senior Data Engineer - Clayton-le-Moors

Senior Data Engineer

Insight Global is seeking a Azure Data Engineers to join a prestigious energy client based in London. The successful candidates will be responsible for designing, implementing, and managing live data streaming pipelines for the client’s Energy Trading team. This role involves designing data pipelines for Crude and Products trading data from on-premise DB2 databases to an Azure Cloud environment.


This is an Inside IR35 contract role and the individual must sit out of the UK.


Key Responsibilities:

  • Pipeline Management: Design, implement, and manage live data streaming pipelines using Azure Databricks to ensure seamless data flow and real-time processing.
  • Process Evaluation: Assess and optimize on-premise to cloud data exchange processes for accuracy, efficiency, and scalability.
  • Code Review and Debugging: Conduct thorough code reviews and debugging sessions, providing guidance and mentorship to junior data engineers to ensure high-quality code and best practices.
  • Problem Solving: Develop innovative solutions to address computing and cost challenges, leveraging advanced technologies and methodologies.


Must Haves:

  • Data Engineering experience
  • Deep expertise with Azure Databricks (DLT, Data Streaming , Unity Catalogue etc.)
  • Prove experience designing high volume, live data streaming solutions using Azure DLT (Delta Live Tables)
  • Expert with Apache Spark and PySpark (ability to review quality of code and debug issues)
  • Background in Data warehousing (SAP Hana, BI/BW, Oracle etc.)
  • Proficient with SQL
  • Proven ability to problem solve and think outside the box

Plusses:

  • Background in Trading domain, specifically energy commodities trading
National AI Awards 2025

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

How to Find Hidden Machine Learning Jobs in the UK Using Professional Bodies like BCS, Turing Society & More

Machine learning (ML) continues to transform sectors across the UK—from fintech and retail to healthtech and autonomous systems. But while the demand for ML engineers, researchers, and applied scientists is growing, many of the best opportunities are never posted on traditional job boards. So, where do you find them? The answer lies in professional bodies, academic-industry networks, and tight-knit ML communities. In this guide, we’ll show you how to uncover hidden machine learning jobs in the UK by engaging with groups like the BCS (The Chartered Institute for IT), Turing Society, Alan Turing Institute, and others. We’ll explore how to use member directories, CPD events, SIGs (Special Interest Groups), and community projects to build connections, gain early access to job leads, and raise your professional profile in the ML ecosystem.

How to Get a Better Machine Learning Job After a Lay-Off or Redundancy

Redundancy in machine learning can feel especially frustrating when your role was technically advanced, strategically important, or AI-facing. But the UK still has strong demand for machine learning professionals across fintech, healthtech, retail, cybersecurity, autonomous systems, and generative AI. Whether you're a research-oriented ML engineer, production-focused MLOps developer, or applied scientist, this guide is designed to help you bounce back from redundancy and find a better opportunity that suits your goals.

Machine Learning Jobs Salary Calculator 2025: Figure Out Your True Worth in Seconds

Why last year’s pay survey is useless for UK ML professionals today Ask a Machine Learning Engineer wrangling transformer checkpoints, an MLOps Lead firefighting drift alarms, or a Research Scientist training diffusion models at 3 a.m.: “Am I earning what I deserve?” The honest answer changes monthly. A single OpenAI model drop doubles GPU demand, healthcare regulators release fresh explainability guidance, & a fintech unicorn pays six figures for vector‑search expertise. Each shock nudges salary bands. Any PDF salary guide printed in 2024 now looks like an outdated Jupyter notebook—missing the gen‑AI tsunami, the surge in edge inference, & the UK’s new Responsible‑AI framework. To give ML professionals an accurate benchmark, MachineLearningJobs.co.uk distilled a transparent, three‑factor formula that estimates a realistic 2025 salary in under a minute. Feed in your discipline, UK region, & seniority; you’ll receive a defensible figure—no stale averages, no guesswork. This article unpacks the formula, highlights the forces driving ML pay skyward, & offers five practical moves to boost your value inside the next ninety days.