Data Engineer

Azure/Databricks Data Engineer (D365 F&O Deployment)

£550 - £580 PER DAY

6 month contract

OUTSIDE IR35

REMOTE, but have to be UK based

Our client, a leader in their field, requires a talented Data Engineer, to join their Group technology team, responsible for the global intelligent data platform. You will be joining the business at a key moment in their evolution and will make a key and lasting impact on our technology organisation and landscape. Reporting to the Group Director of Data and Architecture with responsibility for the data workloads delivered against their Azure/Databricks platform.

You will be working within a major transformation programme migrating data to D365 F&O. You will be expected to have a proactive, hands-on approach.

You will be a key contributor to designing, developing, and managing data ingestion processes and transformation pipelines within Azure and Databricks environments.

The role involves utilizing Databrick's medallion architecture to create well-defined and governed data consumption models, adopting a data-as-a-product mindset and implementing key platform governance steps such as master data management and augmentation, governance, observability and exception/quality reporting

The ideal candidate will have experience in cloud data engineering, an understanding of Databricks, and a strong proficiency in Azure data services.

YOUR SKILLS:

• Minimum of 5 years of experience in data engineering with a focus on cloud technologies.
• Proven experience with Azure services (eg, Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Synapse, Azure Blob Storage).
• Extensive experience with Databricks, including the development and management of data pipelines.
• Strong proficiency in SQL, reasonable Python skills.
• Experience with data governance, data quality, and data security best practices.
• Familiarity with data-as-a-product concepts and methodologies.
• Excellent problem-solving skills and ability to work in a fast-paced, dynamic environment.
• Strong communication and collaboration skills.
• Previous experience in a D365 Data Migration role, with experience in Finance & Operations.
• Extensive wider knowledge of the Microsoft D365 application stack.
• Proficiency in F&O standards documentation and best practices.
• Good understanding and solution design experience of technologies. This includes, but is not restricted to, solutions such as:
• Microsoft Dynamics Finance and Operations
• Microsoft Power Platform
• Microsoft Collaboration Platforms (Office365, SharePoint, Azure DevOps)

WHAT YOU WILL BE DOING:

• Design, implement, and maintain scalable and efficient data pipelines for ingestion, processing, and storage of global enterprise datasets using Azure and Databricks.
• Take end-to-end ownership of application ingestion workloads, ensuring all platform steps/runbooks are adopted.
• Utilize Databrick's medallion architecture (bronze, silver, gold layers) to ensure clean, reliable, and organized data flows.
• Ensure strict version control and reproducibility of data transformations using the DBT toolset.
• Develop and maintain ETL processes to transform raw data into structured data sets for analysis and consumption.
• Work within our data governance framework to implement our runbook to provide best practices to ensure data quality, accessibility, security, and compliance.
• Collaborate with identified data stewards to define productised data consumption. models/products and ensure workload datasets map to the target models.
• Ensure master data structures and reference data are correctly augmented to each workload.
• Optimize and troubleshoot data pipelines to ensure high performance and reliability.
• Use best practice implementations for observability, alerting and monitoring to evolve an effective data operation function.
• Align workloads to our master data management function to ensure data can be matched across applications/workloads.

If your profile matches the above, please send your CV for full details: