Testing Engineer

Required Skills & Experience

Strong test automation experience. Playwright E2E testing. CI/CD test integration and deployment gating. Experience designing regression test suites for API-driven applications. Ability to create deterministic test fixtures and expected- Well design experience (optional)

Job Description

Build and maintain automated test suites for the Runability workflow. Implement Playwright-based end-to-end tests for key user journeys Build validation-path tests, especially around incomplete data and workflow blocking Build regression tests for AG-UI thread handlers Design and maintain the ground-truth test corpus used to validate AI and analytical behavior. Create golden input/output fixture computations. Build automated CI checks that compare produced results against ground truth. Ensure deliberate mutations or degraded outputs fail the test gate. Contribute to LLM-as-judge evaluation harnesses for AI-generated outputs Define scoring rubrics for AI outputs, including accuracy, completeness, consistency, actionability, and engineering relevance. Maintain eval score reports and support deployment gates based on minimum acceptable AI-quality thresholds. Validate authentication and authorization behavior where it affects testing Support cross-browser smoke tests, but this is secondary to automation and AI validation.