Jobs in the United States

25+ jobs

Group jobs by employerEmail me new jobs Save this search

Added 1 min ago
M
Senior Systems Engineer
Microsoft
WA · CA · OR · Redmond, WA · Mountain View, CA · Hillsboro, OR · Redmond · Mountain View · Hillsboro
- $119,800 to $261,000 USD / year
Microsoft is a highly innovative company that collaborates across disciplines to produce cutting edge technology that changes our world. The Azure Cloud Hardware and Infrastructure…
Added 2 mins ago
M
Senior Package Design Engineer
Microsoft
TX · OR · CA · NC · WA · Austin, TX · Hillsboro, OR · Mountain View, CA · Raleigh, NC · Redmond, WA · Austin · Hillsboro · Mountain View · Raleigh · Redmond
- $119,800 to $261,000 USD / year
The Senior Package Design Engineer will be responsible for providing expertise, insight, and support to the Silicon development team through the entire life cycle of the product. In this…
Added 2 mins ago
M
Senior Software Engineer
Microsoft
WA · DC · Redmond, WA · Washington, DC · Redmond · Washington
- $119,800 to $261,000 USD / year
…services and data. We are looking for a motivated cybersecurity engineer with experience in AI penetration testing and proactive risk…
Added 8 mins ago
Senior Director Software Engineering
Capital One
Jefferson City, MO
As a Senior Director, Full-stack Engineer at Capital One, you will lead high-performing engineering teams working to define the future of banking in the cloud. You will partner closely with…
Added 8 mins ago
EG
Forward Deployed AI/ML Engineer IV
Eliassen Group
Jefferson City, MO
- $170,000 to $190,000 USD / year
Experience using AI coding assistants and agentic development tools to rapidly prototype, test, and iterate.
Added 10 mins ago
Senior AI Quality & Evaluation Engineer
Solvd
Remote, United States
Solvd Inc. is a rapidly growing AI-native consulting and technology services firm delivering enterprise transformation across cloud, data, software engineering, and artificial intelligence.…
Added 10 mins ago
Senior Data Engineer - Retrieval & RAG Platform
Solvd
Remote, United States
Solvd Inc. is a rapidly growing AI-native consulting and technology services firm delivering enterprise transformation across cloud, data, software engineering, and artificial intelligence.…
Added 11 mins ago
Staff Mobile Engineer - Session Replay
Jobgether
Remote, United States
…applications and next steps. Our partner is looking for a Staff Mobile Engineer – Session Replay based in the United…
Added 20 mins ago
Senior Director Software Engineering
Capital One
Honolulu, HI
As a Senior Director, Full-stack Engineer at Capital One, you will lead high-performing engineering teams working to define the future of banking in the cloud. You will partner closely with…
Added 20 mins ago
EG
Forward Deployed AI/ML Engineer IV
Eliassen Group
Honolulu, HI
- $170,000 to $190,000 USD / year
Experience using AI coding assistants and agentic development tools to rapidly prototype, test, and iterate.
Added 25 mins ago
Senior Director Software Engineering
Capital One
Bismarck, ND
As a Senior Director, Full-stack Engineer at Capital One, you will lead high-performing engineering teams working to define the future of banking in the cloud. You will partner closely with…
Added 25 mins ago
EG
Forward Deployed AI/ML Engineer IV
Eliassen Group
Bismarck, ND
- $170,000 to $190,000 USD / year
Experience using AI coding assistants and agentic development tools to rapidly prototype, test, and iterate.
Added 33 mins ago
Principal Machine Learning Engineer
Vail Resorts
CO
We are looking for a curious, driven, innovative machine learning engineer who takes initiative to solve problems and create environments that accelerate the development, deployment, and…
Added 33 mins ago
Principal Systems Integrator
Altamira Technology Corporation
Englewood, CO
…contract effort by NRO/GED/SGSPO whose scope includes: CONOPS development and mission thread development, systems engineering and integration analysis, and verification support to the…
Added 33 mins ago
MBSE Systems Engineer Stf
Lockheed Martin
Littleton, CO
…cultivate innovation, reduce costs, and push the boundaries of what technology can achieve. We’re creating future-ready solutions, focusing on resiliency and urgency through our 21st…
Added 34 mins ago
EG
Forward Deployed AI/ML Engineer IV
Eliassen Group
Denver, CO
- $170,000 to $190,000 USD / year
Experience using AI coding assistants and agentic development tools to rapidly prototype, test, and iterate.
Added 35 mins ago
M
Senior Platform Systems Engineer
Microsoft
WA · CA · OR · TX · Redmond, WA · Mountain View, CA · Hillsboro, OR · Austin, TX · Redmond · Mountain View · Hillsboro · Austin
- $119,800 to $261,000 USD / year
…cloud hardware. We are looking for a Senior Platform Systems Engineer with a dedicated passion for customer focused solutions, insight and industry knowledge to envision and implement…
Added 43 mins ago
J
Senior Engineer, Cyber Engineering
Jobgether
Remote, United States
- $160,000 to $175,000 USD / year
…applications and next steps. Our partner is looking for a Senior Engineer, Cyber Engineering based in the United…
Added 46 mins ago
RTL Design Engineer, Google
Google
Sunnyvale, CA
Experience in ASIC RTL design, with a focus on clocking, reset, or timing-critical RTL development.
Added 51 mins ago
EG
Forward Deployed AI/ML Engineer IV
Eliassen Group
Lincoln, NE
- $170,000 to $190,000 USD / year
Experience using AI coding assistants and agentic development tools to rapidly prototype, test, and iterate.
Added 59 mins ago
Electro-Optical/Infrared Signature Analyst Engineer (Mid-Level or Senior)
The Boeing Company
Berkeley, MO
…organization is seeking an Electro-Optical/Infrared Signature Analyst Engineer (Mid-Level or Senior) to join the Phantom Works team located in St. Louis,…
Added 1 hour ago
JH
Simulation & Physics Software Engineer for ISRT Scientific Applications
Johns Hopkins Applied Physics Laboratory
Laurel, MD
- $100,000 USD / year
Are you a software engineer, engine programmer, or low-level C++ engineer who enjoys building core technology for complex real-time software?
Added 1 hour ago
Senior Full-Stack Software Engineer - Procurement Systems
Blue Origin LLC
Seattle, WA
We are seeking a Senior Software Engineer to design and build Blue Origin's next-generation procurement platform that will reinvent the entire procure-to-pay lifecycle. This role is on the…
Added 1 hour ago
J
Senior Software Development Engineer, Backend
Jobgether
Remote, United States
- $156,200 to $262,600 USD / year
…and next steps. Our partner is looking for a Senior Software Development Engineer, Backend based in the United…
Added 1 hour ago
Sr. Specialist, Integration & Test Engineer (Reliability & Maintainability)
L3Harris
Greenville, TX
Job Title: Sr. Specialist, Integration & Test Engineer (Reliability & Maintainability)

Solvd

Senior AI Quality & Evaluation Engineer

Remote, United States

Apply on jobs.ashbyhq.com About employer

Applications open in a new tab.

Job details

Posting language: English
Posted: Nov 13, 2026

Original job description

Solvd Inc. is a rapidly growing AI-native consulting and technology services firm delivering enterprise transformation across cloud, data, software engineering, and artificial intelligence. We work with industry-leading organizations to design, build, and operationalize technology solutions that drive measurable business outcomes.

Following the acquisition of Tooploox, a premier AI and product development company, Solvd now offers true end-to-end delivery - from strategic advisory and solution design to custom AI development and enterprise-scale implementation. Our capability centers combine deep technical expertise, proven delivery methodologies, and sector-specific knowledge to address complex business challenges quickly and effectively.

We are looking for a Senior AI Quality & Evaluation Engineer to own the independent proof that retrieval works and that permissions hold. You’ll build and run the golden question suite, negative entitlement tests, and drift alarms that serve as acceptance gates - so the engineers who built the system are not grading their own work.

This is a 50% allocation role across weeks 3–10, drawn from a Quality Engineering or GRC background, reporting to the POD Lead and working day-to-day with the retrieval engineer.

WHAT YOU’LL DO

Build the golden question suite from top contact drivers and needs workshops - question, expected source, expected passage, and acceptance thresholds for relevance, grounding, and citation accuracy, versioned alongside the content it tests.
Build the negative entitlement test set - queries issued under unentitled identities that must return no restricted passage, with any failure raised as a severity-one alarm.
Build freshness and retirement tests - source-modified-to-index-available lag per item against the agreed threshold, and verification that retired items disappear on the next sync.
Automate the harness to run on schedule and after every content, configuration, or model change - publishing scores to the quality dashboard with trend lines and regression alarms.
Define the QA gate content analysts apply to remediated content.
Run the suite before every pilot change in weeks 7–9 and report misses with root cause - content, chunking, ranking, or permissions - to the right owner.
Write the evaluation runbook and train client stewards and platform owners to add questions, re-run the suite, and interpret regressions.

REQUIRED SKILLS & EXPERIENCE

3+ years in software quality engineering or test automation with Python, including building test harnesses that run in CI on a schedule.
Hands-on evaluation of a search or RAG system in production - writing ground truth sets, scoring relevance and grounding, and tracking drift over time.
Experience designing security-focused negative tests - authorization, entitlement, data exposure - and reporting them at the severity a client security team expects.
Ability to work in AWS - invoking Bedrock and OpenSearch APIs, reading CloudWatch, deploying a scheduled job with infrastructure code.
Clear written reporting; the suite’s output is read by client stewards, not only engineers.

PREFERRED QUALIFICATIONS

AWS Certified AI Practitioner or Machine Learning Engineer Associate.
Experience with RAG evaluation libraries and LLM-as-judge scoring with human calibration.
Contact center or knowledge management domain exposure.
Familiarity with SOC 2 or ISO 27001 evidence practices.

When you join Solvd, you'll…

Shape real-world AI-driven projects across key industries, working with clients from startup innovation to enterprise transformation.
Be part of a global team with equal opportunities for collaboration across continents and cultures.
Thrive in an inclusive environment that prioritizes continuous learning, innovation, and ethical AI standards.

Ready to make an impact?

If you're excited to build things that matter, champion responsible AI, and grow with some of the industry’s sharpest minds. Apply today and let’s innovate together.

Solvd is an equal opportunity employer.

Apply on jobs.ashbyhq.com About employer

Report this job

Work requirements

Jobs in the United States