What you will do
- Architect and oversee end-to-end test strategies across different layers in the testing pyramid;
- Define, evaluate, and monitor model validation criteria, including accuracy threshold, confidence scoring, fallback behaviors, etc.;
- Drive the automation of QA processes, ensuring robust, scalable processes for data ingestion, extraction, and classification;
- Lead the validation integration points between AI components and business logic, identifying risks and ensuring seamless operation;
- Own, maintain, and evolve the test infrastructure, specifically the CI / CD quality gates, data generation, and automated validation scripts;
- Establish and monitor key quality metrics aligned with the business goals, including turnaround time, confidence scores, and user corrections;
- Act as a strategic partner to Product and Engineering, and advocate for quality from design through delivery;
- Keep up with the latest AI tech and how to best apply the latest advancements to make the previously impossible possible for our customers;
- Sense of ownership, take ownership of code quality through exploratory and automated tests.
Must haves
Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field experience;5+ years in AQA, Software Test Engineering, or related roles, including leadership responsibilities;Experience in AQA Javascript ;Solid knowledge of software development life cycle (SDLC), Agile methodologies, and Quality Assurance or Quality Engineering methodologies;Proven experience building and scaling test automation frameworks (e.g. Pytest, Cypress, Selenium, Postman, etc.). Strong technical skills with REST APIs, data, pipelines, and backend services;Strong technical background with APIs, CI / CD pipelines, backend services, and cloud environments (AWS, Azure);Exceptional attention to detail, with a mindset for building robust systems that catch issues early and protect the customer experience;Demonstrated fast, proactive learning skills, especially crucial in the rapidly changing field of AI;Uppper-Intermediate English level.Nice to haves
Experience testing AI / ML in production environments;Experience with GenAI model evaluation metrics;Experience setting up proactive monitoring and alerting systems to catch issues like model drift, performance degradation, or workflow failures.AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions.
If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment every day, there’s no better place — guaranteed! : )
35731
J-18808-Ljbffr