Senior Data Engineer | Hybrid Newcastle | Databricks

Senior Data Engineer | Databricks |

Hybrid 2 - 3 days per week in Newcastle

IR35 / Daily Rate - Yet to be determined, ideally need applicants that are open to both Inside and Outside IR35

Contract Length - 3 month initial

Key DeliverablesThe Deliverables can be described as follows:

Define, build and test the migration away from our Azure SQL Server, used to host the Power BI models, over to the "Gold" Databricks SQL layer.
Identify and deliver opportunities to improve the performance and stability of the over-night data ingestion processes or further downstream to improve the service level of data availability to the business each day.
Act as an advisor to the Data Platform and BI team to support them in their own deliverables and business-as-usual.

Tasks & ResponsibilitiesTo deliver these [three deliverables] you will need experience in:

Data Infrastructure Development: Design, develop, and maintain scalable data pipelines and ELT/ETL processes to support various business needs.
Data Warehousing: Architect and implement data warehouse solutions that are robust, scalable, and aligned with business requirements.
Data Integration: Integrate data from various sources, ensuring high data quality, consistency, and reliability.
Optimisation: Optimise existing data processes for performance and scalability.
Collaboration: Work closely with data platform engineers, BI developers and business stakeholders to understand data needs and translate them into technical requirements.
Documentation: Maintain comprehensive documentation of all data engineering processes, designs, and data models.
Data Security: Implement and enforce best practices for data security, compliance, and governance.
About youCan demonstrate advanced knowledge of cloud services, preferably the broader Fabric or Databricks suite of products; or Azure Data Lake, Azure Data Factory, Azure Logic Apps, Azure Function Apps, alongside a skillset incorporating data modelling techniques and business data requirement comprehension, documentation and communication.

Are able to work efficiently in a dynamic environment, with a focus on detail and accuracy and have a keenness to problem solve.

Hold a steadfast commitment to data security and ability to guide others in this area

Are keen to understand the commercial drivers of the business and can build data solutions to drive growth.

(url removed) / (phone number removed)