Data Engineer

We are TXP. We help businesses and organisations move forward, at pace and at scale. We believe in the transformative power of combining technology and people. By providing consulting expertise, development services and resourcing, we work closely with organisations to solve their most complex business problems.

Our work transforms organisations – and we take that responsibility seriously. We focus on success, pursue excellence and take ownership of everything we do.

We are seeking a Microsoft Fabric Data Engineer to join our technology function. You'll be joining a collaborative team, co-creating tools, supporting each other, providing governance, and building a community. The successful candidate will partner with Public and Private sector clients to deliver end‑to‑end data engineering solutions on Microsoft Fabric, from ingesting and transforming raw data to shaping curated Lakehouse layers that power analytics and reporting. You will transform complex data estates into clean, governed and high‑performing platforms that give clients confidence in the insights they rely on.

We are proud of our culture and values, which guide everything we do:

• Client Focus – We put our clients at the heart of every decision.
• Adaptability – We embrace change and thrive in dynamic environments.
• Responsibility – We take ownership and deliver with integrity.
• Excellence in Delivery – We are committed to delivering outstanding results.
• Success & Celebration – We celebrate achievements and learn from every experience.

As a Data Engineer you will embody these values in your leadership, decision‑making, and client interactions.

Key Responsibilities:

ETL and Data Pipeline Development

• Design and build scalable ETL and ELT pipelines in Microsoft Fabric using PySpark in Fabric Notebooks and Dataflows Gen2 to ingest and transform data from ERP and business applications.
• Implement robust ingestion patterns and orchestration using Fabric Data Factory capabilities, handling varied formats and refresh frequencies while writing curated outputs to Lakehouse tables.
• Develop transformation logic that standardises, cleanses and harmonises data across business units, publishing to managed Delta tables for downstream analytics.
• Apply incremental load and near real time replication strategies, including Mirroring where appropriate, to optimise runtime and latency for operational reliability.

Integration and Quality

• Integrate data from diverse sources into OneLake and Lakehouse, enforcing data quality checks, validation rules and reconciliation steps in pipelines and notebooks to maintain accuracy and integrity.
• Build error handling, logging and monitoring into pipelines, and document lineage using Fabric’s item lineage and the OneLake catalogue to support operational reliability and auditability.
• Collaborate with stakeholders to translate business data requirements into transformation rules and curated models that meet reporting and analytics needs across Fabric workloads.

Lakehouse Development and Performance

• Design and maintain medallion architecture in the Microsoft Fabric Lakehouse on OneLake with bronze, silver and gold layers, promoting clear contracts and progressive data quality.
• Optimise storage formats, partitioning and table maintenance for Delta performance, applying practices such as V‑Order and efficient file sizing to improve query speed and cost effectiveness.
• Prepare gold layer datasets for downstream analytics with Direct Lake connected semantic models to deliver high performance BI without heavy refresh cycles.
• Create reusable frameworks and templates for common ingestion and transformation patterns to accelerate team delivery in Fabric.

Collaboration, Governance and Continuous Improvement

• Work with data leaders to implement standards for workspace design, security and lifecycle management across Fabric capacities, using OneLake and Fabric governance features.
• Contribute to code reviews and CI/CD practices for Fabric items and notebooks, aligning with Microsoft’s analytics engineering guidance and development lifecycle.
• Support testing activities including unit, integration and user acceptance testing for pipelines, notebooks and data models, ensuring reliable promotion through environments.

Skills & Experience:

• Strong hands‑on experience building ETL and ELT pipelines with PySpark in Fabric Notebooks, including Spark SQL and Python for large‑scale data processing.
• Practical experience with Microsoft Fabric Lakehouse on OneLake, including medallion architecture implementation and curation of bronze, silver and gold layers.
• Proficiency with Dataflows Gen2 for low‑code ingestion and transformation, and familiarity with Fabric Data Factory capabilities for orchestration and scheduling.
• Deep understanding of Delta tables, table optimisation and storage layout for query performance, including V‑Order and efficient Parquet file strategies.
• Experience enabling Direct Lake semantic models for high‑performance analytics over Lakehouse and Warehouse data.
• Strong SQL skills for transformation and validation across Lakehouse and Warehouse experiences within Fabric.
• Familiarity with Mirroring to replicate operational data into OneLake for near real time analytics when required.
• Experience integrating data from enterprise systems and APIs into Fabric, and shaping curated datasets for analytics with clear definitions and data contracts.
• Understanding of dimensional modelling and semantic model best practices to support BI and analytics in Fabric
• Experience working in Agile or fast‑paced project environments
• Experience working as part of a technology or professional services consultancy.
• 25 days annual leave (plus bank holidays)
• An additional day of paid leave for your birthday (or Christmas eve)
• 4% Matched employer contributed pension (salary sacrifice)
• Life assurance (3x)
• Access to an Employee Assistance Programme
• Private medical insurance through our partner Aviva.
• Cycle to work scheme
• Access to an independent financial advisor
• 2 x social value days per year to give back to local communities.

Grow with us:

Work on exciting new projects

If you want to avoid getting stuck with the mundane, you’re in the right place. We work in many sectors with fantastic clients, so you’ll always be working on something exciting and challenging.

We recognise that you might have a career path planned out and you might need some support to help you move forward. We’re here to support you and make the most out of your time with us, through challenging work, opportunities to grow and learning and development opportunities.

Be part of the TXP growth journey

We are a high growth, fast paced environment. We currently have 200+ employees and work with clients across the UK. Joining TXP means you’ll be part of that.