Senior QA Engineer ID66279

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards. WHY JOIN US If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you! About The Role We are looking for a Senior QA Engineer to perform manual testing of web applications and validate AI-generated outputs as part of a product team building LLM-integrated features. You will design and execute test cases across UI, API, and AI behavior scenarios using TestRail, Postman, and JIRA, working within an Agile environment. The role requires a detail-oriented engineer who can evaluate model quality metrics and ensure accuracy, completeness, and alignment across prompt–response workflows. What You Will Do Perform manual testing of web applications to identify bugs, glitches, and inconsistencies; Develop, maintain, and execute detailed manual test cases based on product requirements and specifications; Perform thorough cross-browser UI testing to ensure consistent behavior across multiple browsers; Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs; Design and execute test scenarios to verify AI behavior, including prompt-response consistency and contextual relevance; Evaluate AI model quality using defined acceptance metrics such as precision, recall, hallucination rate, and duplication rate; Conduct manual API testing to validate request and response accuracy, error handling, and integration; Use TestRail to create, manage, and track test plans, test cases, and test execution results; Use Jenkins CI/CD pipelines; Report and document defects clearly using JIRA; Participate in daily stand-ups, sprint planning, and other Agile team meetings; Review user stories and acceptance criteria to understand the scope of testing; Estimate own tasks and report risks to the team lead; Collaborate with developers and other QA team members to reproduce issues and verify fixes; Maintain and update test documentation as needed. MUST HAVES 4+ years of experience with manual testing; Basic understanding of methodologies and SDLC and STLC models; Knowledge of testing and quality assurance processes; Strong experience with test design activities to cover a set of product requirements; Knowledge of test design techniques and testing methodologies; Defect analysis skills and the ability to write detailed and useful defect reports; Experience with Postman for API testing and browser developer tools; Experience with UI testing; Experience with TestRail as a test management system; Basic understanding of CI/CD processes; Intermediate English level. NICE TO HAVES Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles; Familiarity with AI and ML concepts, APIs, and frameworks such as OpenAI API, Azure OpenAI, or LangChain; Knowledge of data protection and compliance standards for AI systems, including GDPR, zero-data retention, and PII anonymization; Experience with SSO testing; Experience in performance or load testing. Perks And Benefits Professional growth: Mentorship, TechTalks, and personalized growth roadmaps. Competitive compensation: USD-based pay with education, fitness, and team activity budgets. Exciting projects: Modern solutions with Fortune 500 and top product companies. Flextime: Flexible schedule with remote and office options.