Lead Data Engineer required by our market leading, award winning, retail sector client based in Avonmouth. Our client have recently migrated across to Microsoft Dynamics Business Central and require a Lead Data Engineer to lead the development, maintenance, support, and integration across an increasing array of data sources into an Azure-hosted SQL data warehouse and the Power BI platform.

This is a hybrid role with 2-3 days a week onsite but with flexibility.

The Lead Data Engineer will oversee the technical aspects of Reporting & BI solutions within the Technology team, providing robust reporting capabilities to support Commercial, Operational, and Finance functions.

Main Responsibilities

• Analyse, optimise, and refactor existing stored procedures, ETL jobs, and data workflows to improve performance and reduce load times on the current data warehouse.
  
  
• Clean up and enhance the efficiency of the current Power BI Premium (P1) instance, ensuring optimal resource allocation and utilisation as data volumes and complexity grow.
  
  
• Review and maintain the scheduling of data jobs to ensure timeliness and reliability, improving operational processes to meet evolving business needs.
  
  
• Implement data quality checks, governance, and performance monitoring for both ETL processes and reporting outputs.
  
  
• Lead and actively execute the long-term migration of the on-premises data warehouse to Microsoft Fabric, including hands-on development and implementation of a robust strategy for transitioning datasets, ETL pipelines, and reporting solutions to the cloud.
  
  
• Build scalable ETL processes using Azure Data Factory, Azure Synapse, and other Fabric-native tools, ensuring a smooth integration between the on-prem and future cloud environments.
  
  
• Design and implement Azure-based data architecture, leveraging tools like Azure Data Lake, Azure Synapse Analytics, and Fabric’s data capabilities to future-proof the platform.
  
  
• Utilise Azure tools such as Synapse Analytics, Logic Apps, and Azure Functions to streamline data processing and orchestration across the business.
  
  
• Empower stakeholders to build their own reports and dashboards by enhancing data accessibility and developing self-service capabilities.
  
  Required Background
  
  
• Extensive experience in maintaining and optimising on-premises SQL Server data warehouses, focusing on stored procedures, indexing, partitioning, and load performance.
  
  
• Deep understanding of Azure cloud tools such as Azure Synapse Analytics, Azure Data Lake, Azure Data Factory, and Logic Apps
  
  
• Hands-on experience with Microsoft Fabric or similar modern cloud platforms, with a strong focus on data migration strategies and ETL development
  
  
• Proficiency in writing advanced SQL queries, stored procedures, and Python scripts for data manipulation, pipeline development, and automation
  
  
• Experience with ETL tools (both on-prem like SSIS, and cloud-based like Azure Data Factory) for creating, optimising, and maintaining data pipelines
  
  
• Skilled in Power BI Premium, including data modelling, DAX, performance tuning, and advanced reporting features like composite models and incremental refresh
  
  
• Strong understanding of data governance, security best practices, and compliance requirements (GDPR, etc.)
  
  
• Experience working with Agile methodologies
  
  
• Excellent communication skills, capable of collaborating with both technical and non-technical stakeholders to translate business needs into effective data solutions.
  
  The ‘Nice to Haves’
  
  
• Experience with Microsoft Dynamics 365 Business Central
  
  
• Experience of defining and operating within data management best practice frameworks
  
  
• Experience working in a Retail environment
  
  This fantastic opportunity comes with a competitive starting salary which is accompanied by the following benefits: 7-10% annual bonus, Health Cash Plan, Life Assurance, 25 days annual leave and a generous staff discount to name but a few