SQA Engineer, Machine Learning

About the Role
As a Machine Learning SQA Engineer, you will design and execute test automation and validation systems that ensure the reliability, accuracy, and consistency of AI and ML-based applications. You’ll help bridge classic software QA principles with emerging AI quality challenges — validating large language model (LLM) outputs, data pipelines, and AI-driven APIs.

This role is ideal for a QA engineer with strong automation skills and growing experience in AI or data-centric systems, who is eager to apply proven testing discipline to next-generation intelligent products.

Role expectations

• Develop, maintain, and execute automated and manual test cases for AI/ML systems, APIs, and data pipelines.

• Build and enhance test automation frameworks to validate LLM or ML model outputs and integrations.

• Test AI agent workflows and monitor system performance, reliability, and output consistency.

• Collaborate with engineers and data scientists to diagnose, document, and track defects.

• Evaluate LLM and prompt behavior against predefined benchmarks and edge cases.

• Participate in code reviews and provide feedback on testability and quality standards.

• Contribute to continuous integration and delivery (CI/CD) testing processes and pipelines.

• Maintain test documentation, datasets, and metrics to track system health and quality trends.

• Research and propose new QA methodologies for AI and ML systems (“AI testing AI”).

What we're looking for

• 5-8 years of experience as an SQA, Test Automation, or QA Engineer (preferably with exposure to data or ML systems).

• Proficiency in Python and one or more of: JavaScript, Java, or TypeScript for automation scripting.

• Strong understanding of software testing principles, test case design, and defect lifecycle management.

• Experience building or maintaining test automation frameworks (e.g., PyTest, Cypress, Selenium, Playwright).

• Working knowledge of SQL and structured data validation.

• Familiarity with ML/LLM fundamentals and evaluating model outputs.

• Experience testing RESTful APIs and using tools such as Postman or pytest-requests.

• Exposure to cloud environments (AWS, GCP, or Azure) and CI/CD pipelines (e.g., GitHub Actions, Jenkins).

Preferred Attributes

• Curiosity to learn emerging AI technologies and apply traditional QA methods to new paradigms.

• Analytical mindset with strong systems thinking — able to understand and test full workflows.

• Ability to visualize and communicate complex test scenarios.

• Comfort working in dynamic, iterative environments where experimentation drives improvement.