We are a technology solutions provider that specialises in serving the public sector.
Our mission is to help government organisations become data-driven, transform their digital offerings, streamline their processes, improve citizen services and enhance transparency.
We are seeking a highly skilled and experienced Senior Data Engineer to join our team working with our Central Government client on a contract basis. This role requires a deep understanding of data engineering best practices, strong hands-on experience with AWS, Azure, Apache Spark, data warehousing, database modelling and SQL. You’ll play a critical role in designing, building, and maintaining our data infrastructure to support scalable, high-performance data pipelines and analytics platforms.
Responsibilities will include:
- Design, build, and maintain robust, scalable, and secure data pipelines using AWS services and Apache Spark.
- Develop and optimize data models for reporting and analytics in Redshift and other DWH platforms.
- Collaborate with Data Scientists, Analysts, and Business Stakeholders to understand data requirements and deliver clean, validated datasets.
- Monitor, troubleshoot, and optimize ETL/ELT workflows to ensure data quality and pipeline efficiency.
- Implement best practices in data governance, security, and compliance within cloud environments.
- Lead and mentor junior data engineers, promoting a culture of technical excellence and continuous improvement.
- Write clean, maintainable, and efficient code while following best practices in software development.
- Debug and resolve issues and implement solutions to improve the performance and functionality of existing applications.
- Stay up to date with the latest in cloud and big data technologies, evaluating their potential for use in the platform.
- Collaborate with DevOps and Platform teams to deploy and maintain data pipelines and services.
- Stay up-to-date with industry trends, best practices, and emerging technologies to enhance the development process and maintain high coding standards.
- Document code, systems, and processes to facilitate knowledge sharing and maintainability.
Requirements
- Proven work experience as a Senior Data Engineer using cloud platform technologies, alongside experience with a variety of database technologies including Oracle, Postgres and MSSQLServer;
- Strong expertise in AWS services including AWS DMS, S3, Lambda, Glue, EMR, Redshift, and IAM.
- Proficient in Apache Spark (batch and/or streaming) and big data processing.
- Solid experience with SQL and performance tuning in data warehouse environments.
- Hands-on experience with Amazon Redshift or equivalent, including table design, workload management, and implementing Redshift Spectrum.
- Experience building ETL/ELT pipelines using tools like AWS Glue, EMR, or custom frameworks.
- Familiarity with data modeling concepts.
- Excellent problem-solving and communication skills.
- Proficiency in Java and data pipeline development.
- Familiarity with version control systems (e.g., Git) and agile development methodologies.
- Experience working with Public Sector clients
- Consulting experience
Preferred Knowledge/Experience
- Experience with CI/CD and Infrastructure-as-Code (e.g., Terraform, CloudFormation).
- Familiarity with modern data stack components (e.g., dbt, Snowflake, Airbyte).
- Experience working in an Agile/Scrum environment.
- Knowledge of Python or Java/Scala for data engineering.
- Experience with version control systems (eg Git, CVS).
Applicants will be required to obtain SC and therefore 5 year's UK address history is essential.
IR35 status: Outside
Start: ASAP
Duration: 3 months (initially)
Day rate: negotiable
