Jobs

Senior Data Engineer - Databricks


Job details
  • Nasstar
  • UK
  • 1 month ago

We are Colibri Digital Born in the cloud, Colibri Digital is one of the fastest-growing Cloud Consultancies in the UK with deep expertise in areas such as big data, data science, machine learning, and cloud computing. We are Premier Tier partners with AWS and our recent attainment of Gold competency with Microsoft proves our excellence in evolving cloud and data technologies. Led by industry-recognised technologists we are a dynamic team with a proven track record of delivering masterclass cloud solutions across all industry verticals. Location: Home-based Job Summary We are seeking a highly skilled Data Engineer to join our team and play a pivotal role in designing, implementing, and managing new data technology infrastructure. You will be responsible for building and maintaining a scalable and robust cloud-based platform that consolidates existing legacy data services. The platform will enable efficient data management, automation of processes, and provide high-quality data services to our users and tooling. Key Responsibilities Design, develop, and implement data pipelines and ETL processes to ingest, transform, and load data into the data platform. Collaborate with the Lead Architect and Cloud Engineer to ensure the seamless integration of data services within the cloud infrastructure. Develop data models and schemas to support efficient data storage and retrieval. Implement data quality and validation processes to ensure the accuracy and consistency of data. Optimize data processes and queries for performance and scalability. Collaborate with business stakeholders to understand their data requirements and provide insights and solutions for data-driven decision-making. Work closely with data scientists to provide them with the necessary data and tooling for developing revenue-generating insights and models. Implement data governance and security controls to ensure data privacy and compliance with regulations. Stay updated with the latest data technologies and trends and make recommendations for technology adoption and improvements. Requirements Proven experience as a Data Engineer, working with Azure data services. Strong understanding of data modeling concepts and relational and non-relational databases. Proficiency in SQL and experience with data integration and ETL tools (e.g., Azure Data Factory, Azure Databricks). Experience with data warehousing concepts and technologies (e.g., Azure Synapse Analytics, Azure Data Lake Storage). Familiarity with big data technologies and frameworks (e.g., Hadoop, Spark) is a plus. Knowledge of data governance and data security best practices. Strong problem-solving skills and attention to detail. Excellent communication and collaboration skills to work effectively within a team environment. Experience with Agile methodologies is a plus.

Sign up for our newsletter

The latest news, articles, and resources, sent to your inbox weekly.

Similar Jobs

Senior Data Engineer

You will join the Service Performance team that monitors the entire suite of DWPs digital services a grand total of 700 software applications and systems. We use Splunk, ServiceNow and Power BI tooling to bring data together and develop quality dashboards which are relied upon and widely used by service...

Department for Work and Pensions Manchester

Senior Data Engineer

Senior Data Engineer needed for AI Startup Do you want to work on genuine greenfield projects? Want to use your skills to aid Education? Want to work with like minded individuals with the latest technologies? The Role Youll play a pivotal role in designing, building and optimising their data systems,...

Ocho Belfast

Senior Data Engineer

Senior/Lead Data EngineerA Senior/Lead Data Engineer is required by a rapidly growing Fintech based in London. The Senior/Lead Data Engineer will have extensive knowledge of modern technologies. This would ideally suit a Data Engineer with a passion for driving excellence and experience in a platform engineering or data engineering team....

Dragonfly People London

Senior Data Engineer web3

Hiring CriteriaMUST have experience setting up A/B testing pipelines & MUST have GCP experience.Experience working within the Crypto industry is a plus.Experience working at a fast moving impressive crypto company or FAANG level company or scale-up.Can work full time in London 5 days week. (West End)

TBC Holdings LLC London

Senior Data Engineer (start up experience)

Senior Data engineer- PermanentWant to help save lives?*Hybrid in Central London**2 stage interview*Did you know a frontline worker loses their life every 10 seconds?Imagine leading a team in developing a risk management platform to help save lives.Your role is crucial in creating and maintaining an innovative mobile application that will...

trg.recruitment London

Senior Data Engineer

Senior Data EngineerTech- Python, Airflow, Kafka, AWS/GCP/AzureSalary –£70,000 - £120,000Location– Full Remote (UK)A growing digital consultancy that specialises in transformation is looking for a highly skilled Senior Data Engineer to join a growing and highly experienced team solely focused on building modern data platforms. You will focus on exciting client...

Formula Recruitment Birmingham