Intermediate Data Engineer - Batch / Data Lake
Hybrid Working (6 days per month onsite in Farringdon)
Who are we?
Toyota Connected Europe wants to create a better world through connected mobility for all. We are a new company created to bring big data and a customer focus into all aspects of the mobility experience, so everyone’s experience is more personal, convenient, fun and safe. We create and enable technologies to delight and simplify the lives of everyone who use our products and empower them to think of and use our services in new ways.
You will be joining us at the beginning of Toyota Connected Europe’s journey of building our team and products. We are building teams to inspire, innovate and build technologies and products that are used by millions of people from all walks of life. We want every member of our team to live and breathe the start-up culture of Toyota Connected Europe and feel and act like an owner every day. This is an opportunity to have an immediate impact and voice: what you create today, you will see being used tomorrow.
About the role:
The Data Engineering team enables and manages the ingestion of low latency, high volume car telemetry data that powers our engineering and data science teams to build smart and insightful data products. We are looking for an experienced Senior Data Engineer to join the team who will have a key role in the design, development, implementation and documentation of large-scale, distributed data applications, systems and services. You will engineer data pipelines that will consume vehicle telemetry data to build insightful data services and products. The features you build will power driving experiences across the world.
What you will do:
- Work closely with Data Engineering Lead, Senior Engineers and Product to shape and deliver features to customers.
- Help adoption of modern principles, techniques and technology to the team, raising software quality, value and delivery.
- Ensure engineering practices in accordance with good practice architecture, software engineering and creative thinking to crush expectations.
- Design, implement, and maintain complex data engineering solutions to acquire and prepare data.
- Create and maintain data pipelines to connect data within and between data stores, applications and organisations.
- Carry out complex data quality checking and remediation.
- Design system components and complex software applications and modules using appropriate modelling techniques following agreed architecture and software design standards, guidelines, patterns and methodology.
- Create and communicate multiple design views to balance stakeholders' concerns and to satisfy functional and non-functional requirements.
- Identify, evaluate and recommend alternative design options and trade-offs.
- Model, simulate or prototype the behaviour of proposed software to enable approval by stakeholders and effective construction of the software.
- Verify software design by constructing and applying appropriate methods.
- Design, code, verify, test, document, amend and refactor complex programmes/scripts and integration software services.
- Work side-by-side with other talented engineers in a team-oriented, agile software engineering environment.
- Love writing code and learning to constantly hone your craft as an engineer.
- Work closely with product owners to shape and deliver features to customers.
About you:
- Expertise and experience of Apache Spark, PySpark, Python based pipeline jobs.
- Solid Data Lake/Data Warehouse principles, techniques and technologies - Star Schema, SQL (AWS EMR, Apache Iceberg, parquet).
- Strong database skills and experience is required, we have NoSQL databases as well as relational databases in use often with large data volumes.
- Strong data modelling concepts and principles having extensive experience of building data architectures consolidating multiple complex sources.
- Experience developing and delivering systems on at least one major public cloud provider; preferably AWS.
- Knowledge of and experience working with APIs (designing with OpenAPI is desirable) and web services, CI/CD pipelines (Git-lab desirable) and automated testing (BDD, Performance, Security), Kubernetes and cloud native practices, containerised workloads with tools such as Docker.
- Experience of modern software and data engineering patterns, including those used in highly scalable, distributed, and resilient systems.
- Experience developing microservices-based architectures, including distributed messaging patterns is a plus.
- Experience of high volume IoT domain would be great but not essential.
- Aspiration to take a mentoring role within a team.
- Passion for agile practices, DevSecOps, incremental delivery, continuous improvement and ability to cultivate a strong team culture.
- We would like a self-starter, someone who would reach out to other teams if needed to seek answers and calling out in agile ceremonies blockers.
- Willingness to get involved in problem resolution and initiatives to smooth operational maintenance of production services which might be spread across geographic boundaries.
- We think the knowledge acquired earning a BS in Computer Science, Engineering, Mathematics, or a related field would be of excellent value in this position, but if you are smart and have the experience that backs up your abilities, for us, talent trumps degree every time.
Equal Opportunities, Inclusion & Diversity:We’re committed to building a diverse and inclusive group of talent with a broad range of backgrounds, skills and capabilities and will give full and fair consideration to all applicants. We know that flexibility is key to success and our people work flexibly in many ways, so if this is important to you, please let us know. If you have a disability or any other additional need that requires consideration, accommodation or adjustment to the role or recruitment process, please do let us know.
#J-18808-Ljbffr
