National AI Awards 2025Discover AI's trailblazers! Join us to celebrate innovation and nominate industry leaders.

Nominate & Attend

Databricks Unity Catalog Engineer

Amber Labs
London
9 months ago
Applications closed

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer ›

Senior Data Engineer

Data Engineer

Data Engineer - Operational Data - Shirebrook

Data Engineer - Operational Data

Databricks Unity Catalog Engineer

The Company: 

At Amber Labs, we are a cutting-edge UK and European technology consultancy that prioritises empowering autonomy, promoting experimentation, and facilitating rapid learning to provide exceptional value to our clients. Our company culture is centred around collaboration, where all colleagues, regardless of their role, work together to minimise risk and shorten delivery times. Our team consists of highly-skilled cross-functional consultants, analysts, and support staff.

We are looking for an experienced Databricks Unity Catalog Engineer to join our data engineering team. The ideal candidate will be responsible for designing, implementing, and managing the Databricks Unity Catalog to ensure seamless governance and secure access to data across multiple environments. You will work closely with data architects, data engineers, and cloud administrators to implement governance policies, catalog management, and provide secure data access across the organization's Databricks ecosystem.

As a Unity Catalog Engineer, you will play a key role in enhancing data governance, ensuring compliance, and implementing best practices for managing large-scale data lakes and enterprise data in a multi-cloud environment.

Key Responsibilities:

  • Implement and manage Databricks Unity Catalog, ensuring effective data governance and secure access to sensitive datasets.
  • Design and establish data governance policies, including role-based access control (RBAC), data lineage, data masking, and metadata management within Unity Catalog.
  • Collaborate with data engineering and cloud teams to integrate Unity Catalog with data lakes, ETL pipelines, and data warehouse architectures across AWS, Azure, or GCP environments.
  • Monitor and optimize the performance of Unity Catalog, ensuring that data classification and access policies are applied and enforced correctly.
  • Work closely with security teams to ensure data governance and security policies align with compliance standards such as GDPR, CCPA, HIPAA, or industry-specific regulations.
  • Establish best practices for data discovery and cataloging, enabling data scientists and analysts to quickly access the data they need.
  • Implement data lineage tracking within Unity Catalog to ensure transparency and traceability of data usage across the organization.
  • Manage access control configurations and ensure smooth integration with existing IAM solutions such as Azure AD, AWS IAM, or other identity providers.
  • Develop and maintain detailed technical documentation on the configuration and management of Unity Catalog, including processes for onboarding new datasets.
  • Troubleshoot and resolve issues related to Unity Catalog configurations, data access policies, and integration with other Databricks features such as Delta Lake and MLflow.
  • Provide training and support to data engineering and analytics teams on utilizing Unity Catalog for their data governance and access needs.

Essential Skills and Qualifications:

  • Bachelor's or Master's degree in Computer Science, Information Systems, Data Engineering, or related field.
  • Proven experience with Databricks and hands-on knowledge of implementing Databricks Unity Catalog.
  • Strong experience with cloud platforms (Azure, AWS, GCP), with a focus on Databricks integration.
  • In-depth understanding of data governance principles, including data access control, data lineage, and metadata management.
  • Experience with role-based access control (RBAC), IAM, and cloud security policies to manage secure data access.
  • Proficiency in working with large-scale data lakes and Delta Lake architectures.
  • Solid understanding of ETL/ELT processes, data ingestion, and transformation workflows in cloud environments.
  • Hands-on experience in working with Python, SQL, and other scripting languages for data management and cataloging tasks.
  • Knowledge of data governance and compliance regulations such as GDPR, CCPA, and HIPAA.
  • Experience with Databricks notebooks for developing data pipelines and workflows.
  • Strong problem-solving skills and experience troubleshooting data access issues, performance bottlenecks, and catalog management challenges.

Desirable Skills:

  • Experience with other data cataloging and governance tools, such as AWS Glue, Apache Atlas, or Azure Purview.
  • Certification in Databricks or relevant cloud certifications (e.g., AWS Certified Solutions Architect, Azure Data Engineer).
  • Knowledge of machine learning workflows and experience in supporting data science teams with managed datasets.
  • Experience in multi-cloud environments and data orchestration across Azure, AWS, and GCP.

Benefits:

Join a rapidly expanding start-up where personal growth is a part of our DNA.

Benefit from a flexible work environment focused on deliverable outcomes.

Receive private medical insurance through Aviva.

Enjoy the benefits of a company pension plan through Nest.

25 days of annual leave plus UK bank holidays.

Access Perkbox, a global employee rewards platform offering discounts, perks, and wellness resources.

Participate in a generous employee referral program.

A highly collaborative and collegial environment with opportunities for career advancement.

Be encouraged to take bold steps and embrace a mindset of experimentation.

Choose your preferred device, PC or Mac.

Diversity & Inclusion:

Here at Amber Labs, we are dedicated to fostering an inclusive and equitable workplace for all. Our commitment to diversity, equality, and inclusion includes:

Valuing the unique experiences, perspectives, and backgrounds of all employees and creating an environment where everyone feels welcomed, respected, and valued.

Prohibiting all forms of harassment, bullying, discrimination, and victimisation and promoting a culture of dignity and respect for all.

Educating all new hires on our Diversity and Inclusion policies and ensuring they are aware of their rights and responsibilities to create a safe and inclusive workplace.

By taking these steps, we are dedicated to building a workplace that reflects and celebrates the diversity of our employees and communities.

At Amber Labs, all employees are required to meet the Baseline Personnel Security Standard (BPSS). Please be advised that, at this time, we are unable to consider candidates who require sponsorship or hold a visa of any type.

What Happens Next?

Our Talent Acquisition Team will be in touch to advise you on the next steps. We have a two-stage interview process for most of our consultants. In certain cases, we may include a third and final stage, which is a conversation with the company Partners. This will only be considered if deemed necessary.

National AI Awards 2025

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Machine Learning Jobs Salary Calculator 2025: Figure Out Your True Worth in Seconds

Why last year’s pay survey is useless for UK ML professionals today Ask a Machine Learning Engineer wrangling transformer checkpoints, an MLOps Lead firefighting drift alarms, or a Research Scientist training diffusion models at 3 a.m.: “Am I earning what I deserve?” The honest answer changes monthly. A single OpenAI model drop doubles GPU demand, healthcare regulators release fresh explainability guidance, & a fintech unicorn pays six figures for vector‑search expertise. Each shock nudges salary bands. Any PDF salary guide printed in 2024 now looks like an outdated Jupyter notebook—missing the gen‑AI tsunami, the surge in edge inference, & the UK’s new Responsible‑AI framework. To give ML professionals an accurate benchmark, MachineLearningJobs.co.uk distilled a transparent, three‑factor formula that estimates a realistic 2025 salary in under a minute. Feed in your discipline, UK region, & seniority; you’ll receive a defensible figure—no stale averages, no guesswork. This article unpacks the formula, highlights the forces driving ML pay skyward, & offers five practical moves to boost your value inside the next ninety days.

How to Present Machine Learning Solutions to Non-Technical Audiences: A Public Speaking Guide for Job Seekers

Machine learning is driving change across nearly every industry—from retail and finance to health and logistics. But while the technology continues to evolve rapidly, the ability to communicate it clearly has become just as important as building the models themselves. Whether you're applying for a junior ML engineer role, a research position, or a client-facing AI consultant job, UK employers increasingly expect candidates to explain complex machine learning solutions to non-technical audiences. In this guide, you’ll learn how to confidently present your work, structure your message, use simple visuals, and explain the real-world value of machine learning in a way that makes sense to people without a background in data science.

Machine Learning Jobs UK 2025: 50 Companies Hiring Now

Bookmark this page—we refresh the Hotlist every quarter so you always know who’s really scaling their ML teams. The UK’s National AI Strategy, a £2 billion GenAI accelerator fund and a record flow of private capital have kicked ML hiring into overdrive for 2025. Whether you build production‑grade LLM services or optimise on‑device models for edge hardware, employers need your skills now. Below you’ll find 50 organisations that advertised UK‑based machine‑learning vacancies or announced head‑count growth during the past eight weeks. They’re grouped into five quick‑scan categories so you can jump straight to the type of employer—and mission—that excites you. For each company we list: Main UK hub Example live or recent vacancy Why it’s worth a look (stack, impact, culture) Search any employer on MachineLearningJobs.co.uk to see real‑time adverts, or set a free alert so fresh openings drop straight in your inbox.