Jobs in the United States

25+ jobs

Save this search

Added 9 mins ago
Enterprise Applications Manager (AI & Platform Modernization)
Dentons US LLP
Indianapolis, IN
…on AI-enabled productivity, platform modernization, process automation, data governance, adoption, measurable business value, and continuous improvement while protecting Firm…
Added 9 mins ago
EG
Cloud Engineer
Eliassen Group
Indianapolis, IN
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 31 mins ago
Consultant, Software Engineer (DevXP)
Nationwide Life and Annuity Insurance Co
OH
Software Engineer - AI-Enabled Requirements & Quality Engineering
Added 39 mins ago
IS
Sr. Engineer
INSPYR Solutions
Fort Worth, TX
- $50 USD / hour
Build and implement Agile workflow automation, including process improvements leveraging approved AI tools.
Added 39 mins ago
Manhattan WMS Software Engineer
INSPYR Solutions
Lithia Springs, GA
Experience with warehouse automation, conveyors, sortation systems, or other material handling technologies.
Added 45 mins ago
EG
Cloud Engineer
Eliassen Group
Juneau, AK
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 55 mins ago
UE
Software Development Engineer I - Colorado
Ulteig Engineers, In
Greenwood Village, CO
- $80,000 to $120,000 USD / year
Ulteig is looking for a Software Engineer who is a Computer Science graduate seeking their first professional software engineering role and who is excited to build with modern AI…
Added 55 mins ago
UE
Substation Lead Electrical Engineer
Ulteig Engineers, In
Greenwood Village, CO
- $123,200 to $159,650 USD / year
Ulteig is seeking qualified candidates for the position of Lead Substation Electrical Engineer to support Renewables/Utility/Power projects. The Lead Electrical Engineer acts as a technical…
Added 1 hour ago
EG
Cloud Engineer
Eliassen Group
Jackson, MS
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 1 hour ago
EG
Cloud Engineer
Eliassen Group
Charleston, WV
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 1 hour ago
EG
Cloud Engineer
Eliassen Group
Lincoln, NE
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 1 hour ago
Senior Electrical Engineer, Robotics
Success Matcher Recruitment
Watertown · Watertown, MA
Our client is a fast-growing AI robotics company changing how industrial automation works. They offer robots-as-a-service that customers can deploy quickly at a fraction of the cost of…
Added 2 hours ago
Lead Distributed Systems Engineer (AI/ML Platform) - Watertown, MA
Kelly Services
Watertown, MA
A fast-growing AI and data platform company is seeking a Lead Distributed Systems Engineer (AI/ML Platform) to architect and scale the infrastructure that powers enterprise AI and machine…
Added 2 hours ago
Principal Identity Security Engineer - Natick, Ma
Kelly Services
Natick, MA
This is an opportunity for a senior-level engineer who enjoys solving complex identity challenges, improving platform resilience, and driving automation across critical enterprise services.…
Added 2 hours ago
C
Customer Success Program Manager - Food
Carrier
Beverly, MA
- $79,000 to $158,000 USD / year
…streaming data, machine learning, and predictive analytics to engineer process efficiencies, maximize customer retention, and build sustainable supply…
Added 2 hours ago
CIAM Full Stack Engineer
Manulife
Boston, MA
Added 2 hours ago
Lead Data Engineer, Scientific Data - Boston, Ma
Kelly Services
Boston, MA
…engineering, and artificial intelligence is seeking a Lead Data Engineer, Scientific Data to join its high-impact engineering…
Added 2 hours ago
EG
Cloud Engineer
Eliassen Group
Boston, MA
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 2 hours ago
EG
Cloud Engineer
Eliassen Group
Columbia, SC
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 2 hours ago
SYSTEMS ENGINEER
City of New York
Brooklyn, NY
…in both Google Cloud and Amazon Web Services by hiring one engineer dedicated to each platform. These positions will work collaboratively with the Systems Engineering Manager on strategic…
Added 2 hours ago
Sr. Computational Scientist/Engr.
Triton Systems, Inc.
Chelmsford, MA
We are seeking a Senior Computational Scientist / Engineer to provide technical leadership on research and development efforts within our Applied Computational Engineering group. This role…
Added 2 hours ago
EG
Cloud Engineer
Eliassen Group
Boise, ID
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 2 hours ago
EG
Cloud Engineer
Eliassen Group
Montgomery, AL
- $70 to $88 USD / hour
Our client seeks a senior Cloud Engineer to own and stabilize a legacy GCP and Azure footprint while assisting with conversions and decommissioning. The role will support automation efforts…
Added 2 hours ago
EG
Staff AI & Data Engineer - Databricks
Eliassen Group
Jefferson City, MO
- $80 to $90 USD / hour
Our client is seeking a highly experienced Staff AI & Data Engineer to design and build enterprise-grade data and AI solutions on Databricks. This hands-on role focuses on strengthening the…
Added 2 hours ago
Sr. Systems Engineer - AI Gateway Platform(Remote)
CrowdStrike
Jefferson City, MO
CrowdStrike is seeking an experienced Systems Engineer to own, scale, and secure our enterprise Kong AI Gateway platform as a core technical pillar of the AI Platform Enablement team. This…

Gradientflo Labs

QA Automation Engineer (Founding Team)

San Francisco, CA, United States

$24,000 to $32,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $24,000 to $32,000 USD / year
Posting language: English

Original job description

Role Overview You’ll be the safety net and enabler of speed at Vibecoderz. In a system where AI agents generate artifacts, mini-apps, and real-time interactions, quality isn’t just about correctness - it’s about trust. As the founding QA Automation Engineer, you’ll design frameworks that validate not just code but also AI outputs. From traditional E2E flows to specialized agent evaluation pipelines, you’ll ensure that everything shipping to production meets our gold standard. This role is about building autonomous, self-healing QA systems that scale with our multi-agent platform. You’ll work hand-in-hand with PM, FE, BE, AI, and DevOps to embed quality into every phase of the pipeline using Linear (execution), Notion (test cases/playbooks), and GitHub (CI/CD integration) as your tools of record.

Key Responsibilities

Automation Framework Ownership
Design and implement end-to-end automated test frameworks for FE, BE, and AI agents.
Cover unit, integration, regression, and smoke tests.
Frontend Testing
Build E2E suites for Next.js UI using Playwright or Cypress.
Simulate real-user flows: onboarding, AI tutor interaction, artifact generation.
Backend Testing
Automate FastAPI endpoint validation.
Ensure <100ms response SLA is consistently met.
AI Agent Evals
Develop golden sets for agent responses (TutorAgent, Mini-App Generator, Quiz Generator).
Automate detection of hallucinations, irrelevant answers, or unsafe outputs.
Continuous Testing Integration
Integrate all test suites into GitHub Actions CI/CD pipelines.
Block merges unless all critical tests pass.
Realtime Testing
Validate WebSocket-based real-time interactions (voice transcription, live artifacts).
Create stress tests for multi-user editing and presence.
Security & Reliability Testing
Build automated checks for prompt injection and unsafe code generation.
Implement sandboxed test environments for mini-app execution.
Performance & Load Testing
Benchmark system under 100K concurrent learners.
Identify latency bottlenecks across Pub/Sub and Cloud Run services.
Defect Tracking & Reporting
Use Linear for bug triage and prioritization.
Create dashboards of defect density, test coverage, and agent accuracy.
Culture of Quality
Define QA best practices, playbooks, and documentation in Notion.
Mentor engineers to write better testable code.

Success Metrics

90 Days (Probation):

Automated framework live for FE (Next.js) and BE (FastAPI).
AI TutorAgent “Text → Course” flow covered by golden set evals.
CI/CD pipeline enforces automated test runs before deploy.

12 Months:

85% test automation coverage across platform.
<1% escaped defects in production.
AI agent eval accuracy >90% on golden set outputs.
System proven under 100K concurrent users with <1% error rate.

Must-Haves

10+ years QA automation in SaaS/product engineering.
Strong expertise with Playwright/Cypress, Pytest, Postman/Newman.
Proven experience integrating test suites into CI/CD pipelines.
Familiarity with real-time systems (WebSockets, CRDT/OT).
Deep understanding of QA methodologies: smoke, regression, load, E2E.

Nice-to-Haves

Experience with LLM/AI output evaluation (evals).
Prior startup or founding engineer experience.
Knowledge of security testing (XSS, prompt injection defense).
Contributions to open-source QA frameworks.

Tech Stack Visibility

Frontend: Playwright, Cypress
Backend: Pytest, Postman/Newman
AI Evals: Custom golden set framework, LangSmith/Langfuse for prompt monitoring
CI/CD: GitHub Actions integration
Observability: OpenTelemetry hooks into test reports
Infra: Cloud Run staging environments for test runs

Assessment Objective: Validate ability to design automated test frameworks that cover FE, BE, and AI outputs.

Challenge (Candidate PoC):

Frontend (FE) E2E Test
Write Playwright/Cypress tests for:
User onboarding (sign-up with Google).
Chatting with TutorAgent mock API.
Rendering a Quiz block with answer validation.
Backend (BE) API Test
Write Pytest suite for FastAPI service:
/generate-course endpoint (input: “Teach React Hooks”).
Assert valid JSON structure for output.
AI Agent Eval
Create a golden set with 5 input prompts and expected TutorAgent responses.
Automate comparison with tolerance for natural language variation.
CI/CD Integration
Configure GitHub Actions to run FE + BE + AI eval tests on each PR.
Block merge if any critical tests fail.

Deliverables:

GitHub repo with FE, BE, and AI test suites.
GitHub Actions workflow file with integrated tests.
Golden set JSON for TutorAgent eval.
README documenting design choices and edge cases.

Evaluation Criteria:

Automation Framework Quality (30%)
AI Evals & Innovation (25%)
CI/CD Integration (20%)
Test Coverage & Reliability (15%)
Documentation & Clarity (10%)

Apply About employer

Report this job