Lead AI Test Engineer

October 16, 2026

Hybrid

Lead

London

Indeed

Description

Lead AI Test Engineer role driving quality assurance and testing strategies for AI/ML-powered applications, Generative AI solutions and intelligent automation platforms. The position combines deep expertise in software test engineering with hands-on validation of AI models, LLM-based applications, machine learning systems and data-driven solutions across the full development lifecycle. Based in London with hybrid working across offices, client sites and home.

Responsibilities

Define and implement end-to-end testing strategies for AI, Machine Learning and Generative AI solutions
Lead QA activities across AI product development ensuring quality, reliability, performance, security and compliance
Design and execute test plans for LLM-powered applications, chatbots, copilots, RAG solutions and AI agents
Validate AI model accuracy, bias, fairness, explainability, robustness and performance
Develop and maintain AI test automation frameworks using Python and modern testing tools
Perform functional, integration, regression, API, performance and security testing for AI platforms
Establish test data management practices and synthetic data generation approaches
Define and monitor AI quality metrics: model drift, hallucination rates, response consistency and user experience KPIs
Collaborate with Data Scientists, AI Engineers, Product Owners, Architects and business stakeholders
Support CI/CD integration and quality gates for AI model deployment pipelines
Mentor QA teams and promote best practices in AI testing and automation

Requirements

10 years of experience in Software Testing, Test Automation and Quality Engineering
Hands-on experience testing AI/ML and Generative AI solutions
Strong knowledge of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Prompt Engineering and AI Agents
Experience with AI platforms such as Azure OpenAI, Microsoft Copilot Studio, OpenAI, AWS Bedrock, Vertex AI or similar
Proficiency in Python and test automation frameworks
Experience with API testing tools such as Postman, Rest Assured or similar
Good understanding of Machine Learning model validation techniques and evaluation metrics
Experience with CI/CD pipelines, Git, Azure DevOps, Jenkins or GitHub Actions
Knowledge of security, privacy, ethical AI and responsible AI principles
Familiarity with cloud platforms including Azure, AWS or GCP
Desirable: experience implementing AI testing frameworks and governance controls
Desirable: knowledge of MLOps and model lifecycle management
Desirable: understanding of AI risk management and regulatory compliance frameworks
Desirable: certifications in AI, Cloud, Quality Engineering or Test Automation

Conditions

Hybrid work blending company offices, client sites and home; unable to work at home 100% of the time
Permanent contract
Disability Confident Employer (Level 2) offering an interview to disabled candidates who meet the minimum essential criteria
Access to 250,000 courses and external certifications from AWS, Microsoft and others
Accredited Great Place to Work for Wellbeing with Mental Health Champions and wellbeing apps