Overview
AI QA Specialist Jobs in Lisbon Metropolitan Area at InnoTech
Title: AI QA Specialist
Company: InnoTech
Location: Lisbon Metropolitan Area
A Company you will be happy to grow with!
At InnoTech, it´s all about connecting people with business through technology;
We care about you, your happiness, and your well-being.
We seek people who want to learn and further improve their skills with us – growth is part of our DNA, and that’s why we’re proud to be certified as a Great Place to Work and to be among the Top 50 Best Workplaces in Portugal!
For us, sustainability is not just a word. Integrity and social responsibility lead our behavior.
We dare to be different, ambitious, dream big, and deliver value like no other!
You will have the opportunity to work with top-level partners in the banking area, insurance, retail, telecommunications, financial services, and so much more!
We are 180+ InnoTechers, and we want your talent as an AI QA Specialist.
Your Mission Will Be
- Define, evolve, and maintain evaluation strategies for AI, LLM, and conversational systems;
- Design and manage versioned datasets covering core flows, edge cases, negative scenarios, and safety validations;
- Validate conversational behavior including intent recognition, slot extraction, state transitions, and business rules;
- Detect regressions and evaluation drift across model, data, or prompt changes using reliable quality signals;
- Collaborate with QA and ML teams to embed AI evaluation into CI/CD pipelines and release processes.
You Will Need
- 4+ years of experience in QA, data, ML, or AI-adjacent roles with exposure to production systems;
- Hands-on experience testing AI, LLM, NLU, or conversational systems and handling non-deterministic behavior;
- Strong analytical skills with the ability to design deterministic validations where possible;
- Experience working with structured outputs such as JSON schemas, tool usage, or function calling;
- CPLP nationality or an EU work permit is required to facilitate employment.
Would Be Awesome If You Have
- BSc or MSc in Computer Science/Engineering, Information Technology, or related fields;
- Experience testing voice-based, conversational, or multimodal AI systems;
- Background in data quality, analytics workflows, or ML pipelines;
- Familiarity with observability, monitoring, or evaluation tooling for AI systems;
- Scripting experience, preferably using Python.
With Us You Can Be a
- TechThinker – people who want to re-imagine technology with us;
- TechSolver – strategic, resilient, and a little bit of a puzzle lover;
- InnoMinder – inhale innovation, exhale digital transformation;
- InnoTecher – no words needed. Just happiness, talent, and all the above!.
Are you convinced? Come see it for yourself.
Join our Top-Notch Team!
