Jobs

Data Engineer- AI SaaS


Job details
  • Vortexa
  • London
  • 2 weeks ago
Applications closed

Vortexa is a fast-growing international technology business founded to solve the immense information gap that exists in the energy industry. By using massive amounts of new satellite data and pioneering work in artificial intelligence, Vortexa creates an unprecedented view on the global seaborne energy flows in real-time, bringing transparency and efficiency to the energy markets and society as a whole.

http://www.vortexa.com/

Processing thousands of rich data points per second from many and vastly different external sources, moving terabytes of data while processing it in real-time, running complex prediction and forecasting AI models while coupling their output into a hybrid human-machine data refinement process and presenting the result through a nimble low-latency SaaS solution used by customers around the globe is no small feat of science and engineering. This processing requires models that can survive the scrutiny of industry experts, data analysts and traders, with the performance, stability, latency and agility a fast-moving startup influencing multi-$m transactions requires.

The Data Production Team is responsible for all of Vortexa’s data. It ranges from mixing raw satellite data from 600,000 vessels with rich but incomplete text data, to generating high-value forecasts such as the vessel destination, cargo onboard, ship-to-ship transfer detection, dark vessels, congestion, future prices, etc

The team has built a variety of procedural, statistical and machine learning models that enabled us to provide the most accurate and comprehensive view of energy flows. We take pride in applying cutting-edge research to real-world problems in a robust, long-lasting and maintainable way. The quality of our data is continuously benchmarked and assessed by experienced in-house market and data analysts to ensure the quality of our predictions.

You’ll be instrumental in designing and building infrastructure and applications to propel the design, deployment, and benchmarking of existing and new pipelines and ML models. Working with software and data engineers, data scientists and market analysts, you’ll help bridge the gap between scientific experiments and commercial products by ensuring 100% uptime and bulletproof fault-tolerance of every component of the team's data pipelines.

Requirements

You Are:

  • Experienced in building and deploying distributed scalable backend data processing pipelines that can go through terabytes of data daily using AWS, K8s, and Airflow.
  • With solid software engineering fundamentals, fluent in both Java and Python (with Rust good to have).
  • Knowledgeable about data lake systems like Athena, and big data storage formats like Parquet, HDF5, ORC, with a focus on data ingestion.
  • Driven by working in an intellectually engaging environment with the top minds in the industry, where constructive and friendly challenges and debates are encouraged, not avoided
  • Excited about working in a start-up environment: not afraid of challenges, excited to bring new ideas to production, and a positive can-do will-do person, not afraid to push the boundaries of your job role.
  • Passionate about coaching developers, helping them improve their skills and grow their careers
  • Deep experience of the full software development life cycle (SDLC), including technical design, coding standards, code review, source control, build, test, deploy, and operations

Awesome If You:

  • Have experience with Apache Kafka and streaming frameworks, e.g., Flink,
  • Familiar with observability principles such as logging, monitoring, and tracing
  • Have experience with web scraping technologies and information extraction.

Benefits

  • A vibrant, diverse company pushing ourselves and the technology to deliver beyond the cutting edge
  • A team of motivated characters and top minds striving to be the best at what we do at all times
  • Constantly learning and exploring new tools and technologies
  • Acting as company owners (all Vortexa staff have equity options)– in a business-savvy and responsible way
  • Motivated by being collaborative, working and achieving together
  • A flexible working policy- accommodating both remote & home working, with regular staff events
  • Private Health Insurance offered via Vitality to help you look after your physical health
  • Global Volunteering Policy to help you ‘do good’ and feel better

Sign up for our newsletter

The latest news, articles, and resources, sent to your inbox weekly.

Similar Jobs

Data Engineer - £115,000 - Snowflake - London - Hybrid

Data Engineer - £115,000 - Snowflake - London - HybridCompany Overview:My client is a leading global commodities merchant and infrastructure asset investor, specialising in the physical and financial trading of energy commodities. They leverage advanced data analytics and cutting-edge technology to optimise trading strategies and asset management. With a strong...

City of London

Data Engineer / Architect - £68,500 - Remote

Data Engineer / Architect - £68,500 - RemoteRole Overview:This role would be perfect for a Senior Data Engineer, or junior architect looking to take a huge step up into a more enterprise role. You will have the opportunity to define and implement the enterprise data strategy for the organisation, leading...

Glasgow

Data Engineer Leicester / Hybrid

Data Engineer Leicester HybridMy client based is Leicester is growing! After successfully hiring a number of developers it's the Business Intelligence Teams turn!Working in the BI team this role will see you using modern toolsets within the Microsoft Stack to manage day to day integration, developer ETL processes and enable...

Leicester

Data Engineer

Data EngineerI am looking for a Data Engineer / Data Scientist for a company in the Maidenhead area.Work is Hybrid – 3 days onsite, 2 days remote. You will be leveraging your analytical skills and programming experience to extract insights from complex datasets, develop predictive models and support decision making.KEY...

Maidenhead

Data Engineer (Snowflake) - London - Hybrid - £90k - £100k

Data Engineer (Snowflake) - London - Hybrid - £90k - £100kThis is a great opportunity for a highly experienced data engineer with expertise in Snowflake, AWS, and Python to join a rapidly expanding data team within a well-established company. This role will have senior responsibilities, requiring a candidate who is...

London

Data Engineer 70k

Job Title: Data EngineerLocation: BirminghamSalary: £50,000 - £70,000 per year + BonusBenefits:Private Health-careGenerous Bonus SchemeFlexible Working Options (Remote/Hybrid)About the Role:My client, a leading organisation in the manufacturing, is seeking a skilled Data Engineer to join their growing team. This is an exciting opportunity for a data enthusiast with experience in...

Birmingham