QA Automation Engineer

September 18, 2026

Office

Middle

London

Important before applying

This job has conditions that may affect your decision to apply

Citizenship requirement appliesOffice-only work

Description

Anaplan is hiring a QA Automation Engineer focused exclusively on quality assurance for GenAI and agentic systems. The role centres on building testing strategies, evaluation frameworks and quality metrics for LLM-powered applications. You will develop end-to-end automated test coverage in Python and help ensure AI features are reliable, accurate and trustworthy. The position is based in London.

Responsibilities

Develop and execute end-to-end automated test coverage across evaluations, APIs, UIs and performance tests using Python
Implement testing strategies for GenAI features, including conversational AI, agentic systems and LLM-powered workflows
Run and maintain automated test suites for prompt testing, including regression tests to detect unintended model behaviour changes
Support and maintain evaluation frameworks measuring GenAI performance across accuracy, relevance, safety, consistency and latency
Contribute to curating high-quality test datasets and golden examples representing diverse user scenarios and edge cases
Assist in adversarial testing to identify potential model hallucinations or biases and integrate production alerting systems
Utilise internal tools and testing frameworks to help developers validate GenAI code and assist with user acceptance testing
Take accountability for the workstream automated coverage, documenting testing procedures and known issues

Requirements

Solid QA or test engineering experience with exposure to or strong interest in AI/ML systems
Hands-on Python coding experience
Good understanding of GenAI technologies including LLMs, prompt engineering and AI application patterns
Solid experience with test automation frameworks and scripting (Python, Pytest, API and UI testing)
Practical knowledge of software testing methodologies: functional, integration, regression, performance and security testing
Ability to execute test cases and evaluation criteria for non-deterministic systems
Experience with API testing tools (Postman, REST Assured) and backend testing
Familiarity with CI/CD pipelines and automated testing integration

PythonPytestPostmanREST AssuredCI/CDGenAILLMenterprise saas / ai scenario planning

Similar jobs

Picked by AI from the meaning of the description, not keywords

This device does not meet the site requirements

What may not work

QA Automation Engineer

Job summary

Score

Description

Responsibilities

Requirements

Similar jobs

Job summary

Score

Similar jobs

QA Automation Engineer

Job summaryAnalysis and summary of the description [data-chart=chart-_R_b5s5eav5ubsqiuialb_] { --color-score: var(--warning); } .dark [data-chart=chart-_R_b5s5eav5ubsqiuialb_] { --color-score: var(--warning); } 50ScoreAverage

Job summary

Score

Description

Responsibilities

Requirements

Similar jobs

Job summaryAnalysis and summary of the description [data-chart=chart-_R_b5s5eav5ubsqiuialb_] { --color-score: var(--warning); } .dark [data-chart=chart-_R_b5s5eav5ubsqiuialb_] { --color-score: var(--warning); } 50ScoreAverage

Job summary

Score

Similar jobs