Data Engineer - Analytics/Modelling
Location: Birmingham/London, UK
Mode: Hybrid
Responsibilities
Lead the design and implementation of AWS-based data products that leverage reusable datasets.
Collaborate on creating scalable data solutions using a range of new and emerging technologies from the AWS platform.
Demonstrate AWS data expertise when communicating with stakeholders and translating requirements into technical data solutions.
Manage both real-time and batch data pipelines. Our technology stack includes a wide variety of technologies such as Kafka, AWS Kinesis, Redshift, and DBT.
Design and model data workflows from ingestion to presentation, ensuring data security, privacy, and cost-effective solutions.
Create a showcase environment to demonstrate data engineering best practices and cost-effective solutions on AWS.
Build a framework suitable for stakeholders with low data fluency. This framework should enable easy access to data insights and facilitate informed decision-making.Requirements
Expertise in the full data lifecycle: project setup, data pipeline design, data modelling and serving, testing, deployment, monitoring, and maintenance.
Strong data architecture background in cloud-based architectures (SaaS, PaaS, IaaS).
Proven engineering skills with experience in Python, SQL, Spark, and DBT, or similar frameworks for large-scale data processing.
Deep knowledge of AWS services relevant to data engineering, including AWS Glue, AWS EMR, Amazon S3, Redshift.
Experience with Infrastructure-as-Code (IaC) using Terraform or AWS CloudFormation.
Proven ability to design and optimize data models to address data quality and performance issues.
Excellent communication and collaboration skills to work effectively with stakeholders across various teams.
Ability to create user-friendly data interfaces and visualizations that cater to stakeholders with varying levels of data literacy
The latest news, articles, and resources, sent to your inbox weekly.
This hybrid, 12 month FTC, offers a great balance of home and office working. You’ll join your colleagues in your local office at least 2 days a week.As the UK’s largest fibre-only network, and its only proven wholesale challenger, we’re busy setting new standards for what digital infrastructure can and...
Irlam
Data Engineer Tech Lead (Azure, Databricks Data Lakes)12 month contract, hybrid with weekly visits to office in Central London.Certain Advantage are recruiting for our global commodities trading client in London for a Data Engineer Tech Lead with extensive experience with Databricks, Apache Spark, and data lakes.By taking on the role...
London
Data Engineer with a minimum of 4+ years of experience of data engineering experience. The role will require deep knowledge of data engineering techniques to create data pipelines and build data assets.At least 4+ years of Strong hands on programming experience with Pyspark / Python / Boto3 including Python Frameworks,...
London
An exciting opportunity has arisen for an experienced Data Engineer / Data Warehouse Developer to join a well-established Financial Services organisation in their Central London office - they operate a hybrid working model, and you will spend 3 days per week in their modern office to collaborate with your team.Following...
London
An exciting opportunity has arisen for an experienced Data Engineer / Data Warehouse Developer to join a well-established Financial Services organisation in their Central London office - they operate a hybrid working model, and you will spend 3 days per week in their modern office to collaborate with your team.Following...
City of London
Data Engineer - Circa £650 per day inside ir35 - Remote with occasional travel to Reading - 6-12 months+We are looking for an SC Cleared Data Engineer to work on a major Oracle to PostgreSQL project working within an AWS cloud environment.All applicants must have been involved in an Oracle...
Great Lea Common
