Jobs in the United States

25+ jobs

Save this search

Added just now
Senior Software Engineer, Firmware
NVIDIA Corporation
CA · Santa Clara, CA · Santa Clara
Proficient in test automation and test case design: Strong ability to design comprehensive and efficient test cases, ensuring coverage of edge cases and critical scenarios. Experienced in…
Added just now
Senior Software Engineer, Agentic Systems
NVIDIA Corporation
Remote, United States
We are looking for a Senior Software Engineer to help build NeMo Platform, NVIDIA’s product for developing, evaluating, deploying, and operating AI systems at scale. This role will focus on…
Added just now
Director - Customer Response Engineering
Trellix
Jefferson City, MO
+ Build and run a lab that reproduces customer environments. Use it to reproduce issues quickly and to test releases before customers deploy them.
Added just now
ADAS Embedded SW Engineer
Ford Motor Company
Dearborn, MI
Develop test procedures, plans, automated scripts, and cases
Added just now
ADAS Verification & Validation - System HIL Engineer
Ford Motor Company
Dearborn, MI
Prove out test tools and test benches by performing automated and manual test execution and results analysis.
Added 1 min ago
C
AI Data Engineer
Cognizant
Lansing, MI
- $100,000 to $115,000 USD / year
As an AI Data Engineer, you will make an impact by designing, building, and optimizing scalable cloud-native data engineering solutions that enable advanced AI-driven applications and…
Added 1 min ago
C
AI Data Engineer
Cognizant
Lansing, MI
- $100,000 to $115,000 USD / year
As an AI Data Engineer, you will make an impact by designing, building, and optimizing scalable cloud-native data engineering solutions that enable advanced AI-driven applications and…
Added 18 mins ago
VV
Lead Systems Engineering I Systems Integration Test
Veracity Ventures Inc.
Rochester, NY
- $140,000 to $178,000 USD / year
Experience developing test applications or automated test solutions using:
Added 20 mins ago
AE
Electronic Engineer
Advanced Energy Industries, Inc
Fort Collins, CO
- $90,000 to $120,000 USD / year
Added 20 mins ago
Electrical/Test Engineer II
Lockheed Martin
Littleton, CO
Lead the design, procurement, assembly, test and verification of our automated test systems, including data review and trending.
Added 1 hour ago
Senior Software Engineer - Lab Provisioning & Integration
The Boeing Company
Berkeley, MO
The Boeing Company is currently seeking a Senior Software Engineer - Lab Provisioning & Integration in Berkeley, MO .
Added 1 hour ago
RU
Automation Engineer (Ansible / Infrastructure)
Randstad USA
Remote, United States
- $65 to $70 USD / hour
We are seeking a skilled Ansible Automation Engineer to join our team to transform manual operational workflows into repeatable, automated infrastructure code. In this role, you will work…
Added 1 hour ago
Senior Full Stack Developer, Agentic Engineering
FullBloom
Statewide, PA
…on agent-ready specs, AI-driven implementation and testing, automated quality gates, and human review of everything that ships. This engineer helps stand that pipeline up, codifies what…
Added 2 hours ago
A
Maintenance Mechanic
Aerotek
Plymouth, MA
- $24 USD / hour
Diagnose equipment problems; replace or repair parts; test repaired machinery; and fine-tune settings to optimize performance.
Added 3 hours ago
Test Conductor
Northrop Grumman
Gilbert, AZ
Our Tactical Space Systems Division Integration and Test group is seeking an Test Conductor 2 (Integration and Test) to join our team of qualified and diverse individuals in Gilbert, AZ.…
Added 3 hours ago
Software Engineer, AI Products - MCP Manager
Usercentrics
NY · New York, NY
Write automated tests alongside your code and help keep the test suite fast and reliable.
Added 3 hours ago
Software Engineer (AI Agents)
Fluent, LLC
NY · New York, NY
Fluent is hiring its first dedicated AI Engineer. Over the past year, AI at Fluent has moved from pilots to production: a service desk agent that closes tickets without a human touch, a…
Added 3 hours ago
EA
Software Engineer - AI & Workflow Automation
Elevation Autism Center
Atlanta, GA
- $40 to $60 USD / hour
Software Engineer – AI & Workflow Automation | $40–$60/hour | Hybrid – Alpharetta, GA
Added 3 hours ago
Lead Full Stack Engineer, Wealth Management Technology
Raymond James Financial, Inc.
Saint Petersburg, FL
We are seeking a Lead Full Stack Engineer to drive the design and development of enterprise applications supporting Wealth Management .
Added 3 hours ago
Low Latency Trading Infrastructure Engineer
Citigroup
Jacksonville, FL
The Low Latency Trading Infrastructure Engineer is a rare combination of deep technical specialist and mission-critical operator - a professional whose expertise sits at the intersection of…
Added 3 hours ago
EG
Senior Software Engineer
Eliassen Group
Orlando, FL
- $86.96 to $90.31 USD / hour
Our client seeks a Senior Software Engineer to design, build, and support integrations for an AI-enabled employee assistant that helps users find answers, request services, and complete HR…
Added 4 hours ago
M
Principal Software Engineer
Microsoft
WA · NY · CA · Redmond, WA · Mountain View, CA · Sunnyvale, CA · Redmond · New York · Mountain View · Sunnyvale
- $142,800 to $304,200 USD / year
As a Principal Software Engineer, on the Substrate Application Platform team, you will help build the next generation platform for internal and external developers and agents to access M365…
Added 4 hours ago
M
Principal Hardware Engineer
Microsoft
WA · CA · OR · NC · TX · Redmond, WA · Mountain View, CA · Hillsboro, OR · Raleigh, NC · Austin, TX · Redmond · Mountain View · Hillsboro · Raleigh · Austin
- $142,800 to $304,200 USD / year
…layers, and converting the result into a permanent fix, runbook, test or automated…
Added 4 hours ago
M
Senior Hardware Engineer
Microsoft
CA · WA · TX · NC · OR · Mountain View, CA · Redmond, WA · Austin, TX · Raleigh, NC · Hillsboro, OR · Mountain View · Redmond · Austin · Raleigh · Hillsboro
- $119,800 to $261,000 USD / year
Develop world-class at-scale debug methodologies, diagnostic strategies, validation techniques, and automated test routines for datacenter systems.
Added 4 hours ago
Early-Career R&D Electrical Engineer - System Test and Integration, Onsite
Sandia National Laboratories
Albuquerque, NM
Design and Innovate: Create and test electrical circuit boards, sub-components, cables, and other automated tester hardware to meet rigorous performance standards.

Gradientflo Labs

QA Automation Engineer (Founding Team)

San Francisco, CA, United States

$24,000 to $32,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $24,000 to $32,000 USD / year
Posting language: English

Original job description

Role Overview You’ll be the safety net and enabler of speed at Vibecoderz. In a system where AI agents generate artifacts, mini-apps, and real-time interactions, quality isn’t just about correctness - it’s about trust. As the founding QA Automation Engineer, you’ll design frameworks that validate not just code but also AI outputs. From traditional E2E flows to specialized agent evaluation pipelines, you’ll ensure that everything shipping to production meets our gold standard. This role is about building autonomous, self-healing QA systems that scale with our multi-agent platform. You’ll work hand-in-hand with PM, FE, BE, AI, and DevOps to embed quality into every phase of the pipeline using Linear (execution), Notion (test cases/playbooks), and GitHub (CI/CD integration) as your tools of record.

Key Responsibilities

Automation Framework Ownership
Design and implement end-to-end automated test frameworks for FE, BE, and AI agents.
Cover unit, integration, regression, and smoke tests.
Frontend Testing
Build E2E suites for Next.js UI using Playwright or Cypress.
Simulate real-user flows: onboarding, AI tutor interaction, artifact generation.
Backend Testing
Automate FastAPI endpoint validation.
Ensure <100ms response SLA is consistently met.
AI Agent Evals
Develop golden sets for agent responses (TutorAgent, Mini-App Generator, Quiz Generator).
Automate detection of hallucinations, irrelevant answers, or unsafe outputs.
Continuous Testing Integration
Integrate all test suites into GitHub Actions CI/CD pipelines.
Block merges unless all critical tests pass.
Realtime Testing
Validate WebSocket-based real-time interactions (voice transcription, live artifacts).
Create stress tests for multi-user editing and presence.
Security & Reliability Testing
Build automated checks for prompt injection and unsafe code generation.
Implement sandboxed test environments for mini-app execution.
Performance & Load Testing
Benchmark system under 100K concurrent learners.
Identify latency bottlenecks across Pub/Sub and Cloud Run services.
Defect Tracking & Reporting
Use Linear for bug triage and prioritization.
Create dashboards of defect density, test coverage, and agent accuracy.
Culture of Quality
Define QA best practices, playbooks, and documentation in Notion.
Mentor engineers to write better testable code.

Success Metrics

90 Days (Probation):

Automated framework live for FE (Next.js) and BE (FastAPI).
AI TutorAgent “Text → Course” flow covered by golden set evals.
CI/CD pipeline enforces automated test runs before deploy.

12 Months:

85% test automation coverage across platform.
<1% escaped defects in production.
AI agent eval accuracy >90% on golden set outputs.
System proven under 100K concurrent users with <1% error rate.

Must-Haves

10+ years QA automation in SaaS/product engineering.
Strong expertise with Playwright/Cypress, Pytest, Postman/Newman.
Proven experience integrating test suites into CI/CD pipelines.
Familiarity with real-time systems (WebSockets, CRDT/OT).
Deep understanding of QA methodologies: smoke, regression, load, E2E.

Nice-to-Haves

Experience with LLM/AI output evaluation (evals).
Prior startup or founding engineer experience.
Knowledge of security testing (XSS, prompt injection defense).
Contributions to open-source QA frameworks.

Tech Stack Visibility

Frontend: Playwright, Cypress
Backend: Pytest, Postman/Newman
AI Evals: Custom golden set framework, LangSmith/Langfuse for prompt monitoring
CI/CD: GitHub Actions integration
Observability: OpenTelemetry hooks into test reports
Infra: Cloud Run staging environments for test runs

Assessment Objective: Validate ability to design automated test frameworks that cover FE, BE, and AI outputs.

Challenge (Candidate PoC):

Frontend (FE) E2E Test
Write Playwright/Cypress tests for:
User onboarding (sign-up with Google).
Chatting with TutorAgent mock API.
Rendering a Quiz block with answer validation.
Backend (BE) API Test
Write Pytest suite for FastAPI service:
/generate-course endpoint (input: “Teach React Hooks”).
Assert valid JSON structure for output.
AI Agent Eval
Create a golden set with 5 input prompts and expected TutorAgent responses.
Automate comparison with tolerance for natural language variation.
CI/CD Integration
Configure GitHub Actions to run FE + BE + AI eval tests on each PR.
Block merge if any critical tests fail.

Deliverables:

GitHub repo with FE, BE, and AI test suites.
GitHub Actions workflow file with integrated tests.
Golden set JSON for TutorAgent eval.
README documenting design choices and edge cases.

Evaluation Criteria:

Automation Framework Quality (30%)
AI Evals & Innovation (25%)
CI/CD Integration (20%)
Test Coverage & Reliability (15%)
Documentation & Clarity (10%)

Apply About employer

Report this job