Senior Data Engineer

OVERVIEW

We are looking for a Senior Data Engineer who will be working as part of the Data Products team responsible for building identity segmentation solutions, optimising programmatic advertising campaigns, and measuring their performance and other KPIs.

Search on its own is a highly complex dataset. Captify’s technologies have been built to extract maximum value from search for brands, partners and businesses all over the world and also to innovate & improve the consumer experience.

Captify’s world-class engineers, semantic specialists and product teams are building the future of Search, and as part of our Engineering Team you will play a key part in developing our offering.

KEY RESPONSIBILITIES

What you’ll be doing:

• 
  
  
  
  
  
  
• You will clean, transform,
  and analyse vast amounts of raw data from various systems using Pyspark
  
  
  
  
  
  
• Run large jobs on
  Kubernetes, AWS and Databricks
  
  
  
  
  
  
• Build massive data pipelines
  
  
  
  
  
  
• Produce unit tests
  
  
  
  
  
  
• Code Documentation
  

REQUIREMENTS:

What you need to be successful:

• 
  
• 5 years + professional experience as a Data Engineer
  
• Proficiency in Python/Pyspark.
  
• Apache Spark 3.x: RDD, SQL DataFrame
  
• Spark query tuning and performance optimization
  
• PSQL database integration
  
• Some experience working with Prometheus/Grafana, S3, Athena, Apache Kafka, Airflow and Pandas
  
• Scala/Java experience is a plus
  
• Statistics/ML/AI knowledge is a plus
  

YOU WILL ENJOY:

• Generous holiday + bank holidays + your birthday + volunteer day
  
  
• Regular shutdown days, including the festive period
  
  
• Hybrid or remote working - depending on location
  
• Private healthcare (Depending on location and contract)
  
• Access to SPILL - instant therapy support
  
  
• Access to Mental Health First Aiders
  
• Financial well-being support
  
• Work from any Captify office in the world