Data Engineer

London

Hybrid - in office 1/2 days per week.

About the Role

We're looking for a great candidate to join our clients Platform team to help deliver data engineering and technical pipeline solutions that support their growing product portfolio.

This will include orchestration of complex pipelines that bridge software products, frameworks and our own data lake infrastructure.

Responsibilities

• Working within the Platform Team to architect, build and maintain pipeline/data engineering solutions for geospatial and non-spatial datasets.
• Integrate and consume data from WFS/WMS services.
• Support the deployment and maintenance of Geoserver instances.
• Collaborate with analysts, developers, and stakeholders to define data requirements.
• Ensure data integrity, quality, and security across all sources.
• Liaise with other team members, including within the Platform team and wider Make/Sell teams in order to determine their requirements and identify further opportunities for automation.
• Supporting product/project management cycles by helping the product team to prioritise requirements.
• Develop novel solutions to match data transformation requirements, following best practices, prioritising maintainability and performance.
• Keep software, datasets and systems up to date.
• Assist in data and server migrations.
• Automation of processes and systems that are not yet properly implemented.
• Keeping up-to-date with current and emerging automation technologies and trends.
• Manage and optimize PostgreSQL databases for performance and scalability.

Required Skills

• Experience in building third and first party software pipelines and ETL solutions.
• Experience in working across common database implementations and cloud technologies.
• Experience of using, managing and maintaining version control solutions.
• A good understanding of:

Integrations with external systems.

Data structures.

Software architecture, design patterns and algorithms

Agile principles.