Lead QA Engineer (AI Voice)

Description

What you'll be doing

• Design the voice AI test strategy and process, from speech input through speech recognition (ASR), the LLM, tools and integrations, to speech output.
• Define the automation approach and how it scales, including multilingual coverage of around 100 languages.
• Test both predictable behaviour (intents, routing, integrations) and non-deterministic AI behaviour.
• Evaluate prompts and LLM responses, using test datasets and scoring criteria.
• Lead UAT: planning, scenarios, test data, coordination of business testers and sign-off recommendation.
• Gather business acceptance criteria and turn them into measurable targets.
• Own defect and risk management, and give release go/no-go recommendations.
• Replan and communicate options when development is delayed.
• Challenge stakeholders on readiness and quality, and mentor the Regular engineer.

Requirements

• At least 6 years of experience in software QA.
• At least 2 years of experience in testing voice agents or conversational AI in live delivery.
• Has led E2E and UAT, and worked as the lone or lead QA with limited guidance.
• Experience in international or distributed teams.
• Voice AI testing: speech recognition and speech output, latency, interruptions, accents, noise.
• Voice quality metrics such as word error rate, intent accuracy and task completion.
• Practical understanding of how LLMs work and fail. Deep LLM engineering is not needed.
• Test automation in Python or JavaScript/TypeScript, with CI/CD and API testing (e.g. Postman).
• Use of cloud speech services (Azure, Google, AWS or similar) to generate test calls.
• Practical experience with Jira or Azure DevOps, and a test management tool.
• Practical experience with troubleshooting from logs, transcripts and call recordings.
• Voice testing platforms (Hamming, Cekura, Coval, Botium) is a plus.
• Experience with LLM evaluation tools (Promptfoo, DeepEval, LangSmith) is a plus.
• Experience with Telephony basics (SIP, Twilio, Azure Communication Services) is a plus.
• ISTQB certification is a plus.
• Experience in Responsible-AI testing (PII, bias, safety) is a plus.
• Experience validating customer journeys across multiple markets is a plus.
• Structured, recommendation-first communication with technical and business stakeholders.
• English at least B2.

Benefits

• Stable employment. On the market since 2008, 1500+ talents currently on board in 7 global sites.
• “Office as an option” model. You can choose to work remotely or in the office.  
• Workation. Enjoy working from inspiring locations in line with our workation policy. 
• Great Place to Work® certified employer.
• Flexibility regarding working hours and your preferred form of contract.  
• Comprehensive online onboarding program with a “Buddy” from day 1.   
• Cooperation with top-tier engineers and experts.  
• Unlimited access to the Udemy learning platform from day 1.
• Certificate training programs. Lingarians earn 500+ technology certificates yearly. 
• Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.
• Grow as we grow as a company. 76% of our managers are internal promotions.  
• A diverse, inclusive, and values-driven community.   
• Autonomy to choose the way you work. We trust your ideas.  
• Create our community together. Refer your friends to receive bonuses.  
• Activities to support your well-being and health.
• Plenty of opportunities to donate to charities and support the environment. 
• Modern office equipment. Purchased for you or available to borrow, depending on your location.