Jobs in the United States

25+ jobs

Save this search

Added 1 min ago
Senior Founding Engineer, Platform
Alpaca Health
New York City, NY
…- the data models, the money logic, the state machines, the API contracts, the AI infrastructure. Not "owns" in the sense of reviewing PRs. Owns in the sense of: you designed it, you built…
Added 2 mins ago
R
Site Reliability Engineer (In-Person)
Refine
New York City, NY
- $95,000 to $125,000 USD / year
We are seeking a DevOps/Site Reliability Engineer to join our team. This role will focus on building and maintaining our continuous integration/deployment pipelines, ensuring high…
Added 2 mins ago
R
Backend Engineer (In-Person)
Refine
New York City, NY
- $75,000 to $100,000 USD / year
Strong understanding of API security, OAuth 2.0, and credential management
Added 4 mins ago
Senior Backend Engineer
Gaya
Remote, United States
As a Senior Backend Engineer at Gaya, you will be critical in enhancing and scaling our backend systems. You will be responsible for improving our API infrastructures, optimizing core…
Added 6 mins ago
Site Reliability Platform Engineer (SRE)
Luupli
Remote, United States
We’re looking for a Site Reliability Platform Engineer (SRE) to help design, optimise, and scale Luupli’s infrastructure, with a specific focus on our mobile-first platform and Google Cloud…
Added 7 mins ago
MT
Lead Engineer
Medina Talent Group
Irvine, CA
- $135,000 to $150,000 USD / year
The Lead Engineer for Enterprise Applications is responsible for driving the design, development, modernization, and lifecycle management of enterprise application portfolios. This role…
Added 8 mins ago
SD
Senior Security Operations Engineer
SunCore Digital
Remote, United States
- $165,000 to $205,000 USD / year
SunCore Digital is seeking a hands-on Senior Security Operations Engineer to assess and improve the security of our applications, cloud environments, development practices, identities,…
Added 8 mins ago
SD
Senior DevOps Engineer (Security)
SunCore Digital
Scottsdale, AZ · Bellevue, WA
- $130,000 to $145,000 USD / year
…throughout the southwest. We seek a motivated Senior DevOps Engineer (Security) with an unstoppable drive for excellence and the well-deserved rewards that come with it. Alongside a…
Added 8 mins ago
SD
Mobile Test Engineer
SunCore Digital
Scottsdale, AZ · Bellevue, WA
- $135,000 to $155,000 USD / year
SunCore Digital is seeking a Mobile QA Engineer, Flutter to own hands-on testing of our Flutter-based mobile applications across supported Android and iOS devices.
Added 8 mins ago
Dev/Sec/Ops and Threat Engineering Specialist
SunCore Digital
Scottsdale, AZ · Bellevue, WA
Implement access control policies, secrets management, and API security standards
Added 13 mins ago
Backend Software Engineer ($10,000) - Washington, DC
Jobs Xchange
Washington, DC
Impactful Work: Your contributions will directly influence critical national security programs and initiatives.
Added 14 mins ago
GL
QA Automation Engineer (Founding Team)
Gradientflo Labs
San Francisco, CA
- $24,000 to $32,000 USD / year
As the founding QA Automation Engineer, you’ll design frameworks that validate not just code but also AI outputs. From traditional E2E flows to specialized agent evaluation pipelines,…
Added 14 mins ago
GL
Full Stack Developer (Founding Team)
Gradientflo Labs
San Francisco, CA
- $40,000 to $60,000 USD / year
Authentication & Security - Implement Firebase Authentication (Google/GitHub OAuth), JWT validation in API Gateway, and FE session handling.
Added 14 mins ago
GL
DevOps Engineer (Founding Team)
Gradientflo Labs
San Francisco, CA
- $24,000 to $32,000 USD / year
As the founding DevOps engineer, you will design our CI/CD pipelines, observability stack, infra-as-code, and security posture from day zero. You’ll ensure that when a developer ships code,…
Added 14 mins ago
GL
Chief Technology Officer (Tech - Founding Pillar)
Gradientflo Labs
San Francisco, CA
- $40,000 to $60,000 USD / year
Ensure enterprise-grade security (IAM, guardrails, sandboxing, prompt injection defense).
Added 14 mins ago
GL
Backend Developer (Founding Team)
Gradientflo Labs
San Francisco, CA
- $32,000 to $40,000 USD / year
API Gateway & Security - Expose services through GCP API Gateway with JWT verification, rate limiting, and role-based access.
Added 14 mins ago
GL
AI Engineer (Founding Engineer – AI)
Gradientflo Labs
San Francisco, CA
- $24,000 to $32,000 USD / year
You will be the intelligence architect of Vibecoderz. As the AI Engineer, you’ll design, implement, and optimize the multi-agent orchestration system that powers the TutorAgent,…
Added 16 mins ago
Security Engineer Internship
Metasys
Remote, United States
The Security Engineer is a critical role responsible for implementing and enforcing robust security practices across our entire integrated supply chain e-commerce platform. You will…
Added 16 mins ago
Backend Engineer Internship
Metasys
Remote, United States
The Backend Engineer is a hands-on role responsible for the development, scaling, and maintenance of the core API services that power our entire integrated platform. You will build highly…
Added 16 mins ago
Automation QA Engineer Internship
Metasys
Remote, United States
Integration Validation: Verify the quality of data retrieval and the security of the API layers connecting AI agents (using LangChain/LlamaIndex) to the core WMS/OMS data.
Added 16 mins ago
CG
Principal Cybersecurity Engineer/ Jr Cybersecurity Architect
Comspark group
Bellevue, WA · Frisco, TX · Overland Park, KS
- $90 USD / hour
Principal Cybersecurity Engineer will help ensure that client’s software, systems and infrastructure are designed and implemented to the highest security standards. Performs technical…
Added 16 mins ago
DevSecOps AWS (EKS) Lead Engineer
Comspark group
Phoenix, AZ
We are seeking an experienced DevSecOps AWS ( EKS) Lead Engineer to join our team at Capgemini. This role focuses on CI/CD of applications including Kubernetes-based container orchestration…
Added 17 mins ago
Senior Software Engineer, Custody
Prospex CRM
United States
…transactions across chains, and expose all of that as a developer API and a regulated-grade institutional…
Added 17 mins ago
Head of Engineering - Custody
Prospex CRM
United States
…transactions across chains, and expose all of that as a developer API and a regulated-grade institutional…
Added 17 mins ago
Head of Engineering
Prospex CRM
United States
…transactions across chains, and expose all of that as a developer API and a regulated-grade institutional…

Gradientflo Labs

QA Automation Engineer (Founding Team)

San Francisco, CA, United States

$24,000 to $32,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $24,000 to $32,000 USD / year
Posting language: English

Original job description

Role Overview You’ll be the safety net and enabler of speed at Vibecoderz. In a system where AI agents generate artifacts, mini-apps, and real-time interactions, quality isn’t just about correctness - it’s about trust. As the founding QA Automation Engineer, you’ll design frameworks that validate not just code but also AI outputs. From traditional E2E flows to specialized agent evaluation pipelines, you’ll ensure that everything shipping to production meets our gold standard. This role is about building autonomous, self-healing QA systems that scale with our multi-agent platform. You’ll work hand-in-hand with PM, FE, BE, AI, and DevOps to embed quality into every phase of the pipeline using Linear (execution), Notion (test cases/playbooks), and GitHub (CI/CD integration) as your tools of record.

Key Responsibilities

Automation Framework Ownership
Design and implement end-to-end automated test frameworks for FE, BE, and AI agents.
Cover unit, integration, regression, and smoke tests.
Frontend Testing
Build E2E suites for Next.js UI using Playwright or Cypress.
Simulate real-user flows: onboarding, AI tutor interaction, artifact generation.
Backend Testing
Automate FastAPI endpoint validation.
Ensure <100ms response SLA is consistently met.
AI Agent Evals
Develop golden sets for agent responses (TutorAgent, Mini-App Generator, Quiz Generator).
Automate detection of hallucinations, irrelevant answers, or unsafe outputs.
Continuous Testing Integration
Integrate all test suites into GitHub Actions CI/CD pipelines.
Block merges unless all critical tests pass.
Realtime Testing
Validate WebSocket-based real-time interactions (voice transcription, live artifacts).
Create stress tests for multi-user editing and presence.
Security & Reliability Testing
Build automated checks for prompt injection and unsafe code generation.
Implement sandboxed test environments for mini-app execution.
Performance & Load Testing
Benchmark system under 100K concurrent learners.
Identify latency bottlenecks across Pub/Sub and Cloud Run services.
Defect Tracking & Reporting
Use Linear for bug triage and prioritization.
Create dashboards of defect density, test coverage, and agent accuracy.
Culture of Quality
Define QA best practices, playbooks, and documentation in Notion.
Mentor engineers to write better testable code.

Success Metrics

90 Days (Probation):

Automated framework live for FE (Next.js) and BE (FastAPI).
AI TutorAgent “Text → Course” flow covered by golden set evals.
CI/CD pipeline enforces automated test runs before deploy.

12 Months:

85% test automation coverage across platform.
<1% escaped defects in production.
AI agent eval accuracy >90% on golden set outputs.
System proven under 100K concurrent users with <1% error rate.

Must-Haves

10+ years QA automation in SaaS/product engineering.
Strong expertise with Playwright/Cypress, Pytest, Postman/Newman.
Proven experience integrating test suites into CI/CD pipelines.
Familiarity with real-time systems (WebSockets, CRDT/OT).
Deep understanding of QA methodologies: smoke, regression, load, E2E.

Nice-to-Haves

Experience with LLM/AI output evaluation (evals).
Prior startup or founding engineer experience.
Knowledge of security testing (XSS, prompt injection defense).
Contributions to open-source QA frameworks.

Tech Stack Visibility

Frontend: Playwright, Cypress
Backend: Pytest, Postman/Newman
AI Evals: Custom golden set framework, LangSmith/Langfuse for prompt monitoring
CI/CD: GitHub Actions integration
Observability: OpenTelemetry hooks into test reports
Infra: Cloud Run staging environments for test runs

Assessment Objective: Validate ability to design automated test frameworks that cover FE, BE, and AI outputs.

Challenge (Candidate PoC):

Frontend (FE) E2E Test
Write Playwright/Cypress tests for:
User onboarding (sign-up with Google).
Chatting with TutorAgent mock API.
Rendering a Quiz block with answer validation.
Backend (BE) API Test
Write Pytest suite for FastAPI service:
/generate-course endpoint (input: “Teach React Hooks”).
Assert valid JSON structure for output.
AI Agent Eval
Create a golden set with 5 input prompts and expected TutorAgent responses.
Automate comparison with tolerance for natural language variation.
CI/CD Integration
Configure GitHub Actions to run FE + BE + AI eval tests on each PR.
Block merge if any critical tests fail.

Deliverables:

GitHub repo with FE, BE, and AI test suites.
GitHub Actions workflow file with integrated tests.
Golden set JSON for TutorAgent eval.
README documenting design choices and edge cases.

Evaluation Criteria:

Automation Framework Quality (30%)
AI Evals & Innovation (25%)
CI/CD Integration (20%)
Test Coverage & Reliability (15%)
Documentation & Clarity (10%)

Apply About employer

Report this job