Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 4 mins ago
Sr Validation Engineer (RF, mmWave & Mixed-Signal) - TeraWave
Blue Origin Personnel, LLC
San Diego, CA · Cupertino, CA
…interfaces. As a Senior RF/mmWave & Mixed-Signal Validation Engineer, you prove that silicon works on the bench. You bring up first samples, characterize RF and mixed signal performance…
Added 4 mins ago
Software Engineer II - TeraWave
Blue Origin Personnel, LLC
Cupertino, CA
We are seeking a Software Engineer to join our PoP development team which is the high-performance, fast-path software that powers the TeraWave Point of Presence (PoP).
Added 17 mins ago
Cloud Efficiency Engineer (AWS/FinOps)
Buildkite
United States
Buildkite's CI platform is trusted by the world's leading engineering teams, helping ship software used by more than 1 billion people every day. Companies including Canva, Meta, Mistral AI,…
Added 18 mins ago
Senior Vehicle Power Electronics Engineer
Anduril Industries
Santa Ana, CA · Costa Mesa, CA
We are scaling at a historic pace, anchored by Arsenal-2, our new software-defined shipyard being built in Baltimore County, Maryland. It's part of a new $6.6 billion investment in…
Added 18 mins ago
Flight Test Engineer - Space Systems
Anduril Industries
Costa Mesa, CA
We are seeking a highly motivated and experienced Flight Test Engineer with a focus on developmental testing of a full range of Sub-sonic to Hypersonic systems. You will collaborate closely…
Added 22 mins ago
Software Development Engineer II, KMS
Amazon Development Center U.S., Inc.
Seattle, WA
Are you excited about building software that operates at massive scale and directly impacts millions of customers? As a Software Development Engineer II at Amazon, you will design,…
Added 24 mins ago
W
Senior Software Engineer (Quality, Health Records)
Whoop
Boston, MA
- $150,000 to $215,000 USD / year
As a Senior Software Engineer you will own quality engineering for Health Records while building shared data-mocking capabilities that support teams across WHOOP.
Added 25 mins ago
O
Lead Principal Software Engineer, Core Infrastructure
Oracle
Nashville, TN
- $146,300 to $306,400 USD / year
As a Consulting Member of Technical Staff, you will own the software design and development for major components of Oracle's Cloud Infrastructure. You should be both a rock-solid lead…
Added 27 mins ago
EG
Quality Assurance Engineer - PHP
Eliassen Group
Montpelier, VT
- $40 to $50 USD / hour
Our client seeks a results-driven Senior Quality Assurance Engineer to support Policyholder Portal applications. The role requires manual, automation, mobile, API, and database testing…
Added 29 mins ago
Mechatronics Engineer III
JPL/NASA
Pasadena, CA
You will be working as a Mechatronics Engineer within the Mechanical Engineering, Integration, and Test Section, which leads the design, analysis, development, integration, and test of…
Added 30 mins ago
Mechanical Engineering Specialist
Lynntech, Inc.
College Station, TX
Lynntech is seeking a mechanical engineer to support the development of technologies, prototypes, and products in the defense, aerospace, and medical domains. A successful candidate will…
Added 31 mins ago
Aerodynamicist
Ford
Long Beach, CA · Long Beach
…end-to-end organization, we turn ideas into high-quality vehicles, software and experiences across gas, hybrid and electric, designed around what customers…
Added 38 mins ago
ZT
Quality Control Laboratory Technician, Support I
Zebra Technologies
Morris Plains, NJ
- $23 USD / hour
Monitor and report test status of various production runs to QC Manager, Manufacturing Manager, QC Supervisor, Technology, or Process Engineer.
Added 38 mins ago
Senior Test Engineer
ASRC Federal Holding Company
Egg Harbor Township, NJ
ASRC Federal Advanced Research LLC (AFAR) is seeking a Senior Test Engineer to join our Common Support Services - Weather (CSS-Wx) team that supports the Federal Aviation…
Added 40 mins ago
Sr. Antenna Production Engineer
Echodyne Corp
Kirkland, WA
We are seeking an Sr. Antenna Production Engineer to help transition our proprietary antenna technology into manufacturable MESA radar hardware. This role works closely with RF,…
Added 47 mins ago
IG
Software Quality Engineer
Insight Global
Sunset Hills, MO
- $35 to $40 USD / hour
We are seeking a Junior Software Quality Engineer with strong test automation experience and prior hands-on experience applying AI-assisted testing practices. In this role, you will help…
Added 56 mins ago
H
R&D Writing System Engineer
HP
Corvallis
- $93,400 to $143,800 USD / year
…HP’s industry leading print businesses. This Specialist-level engineer primarily supports the PageWide Industrial (PWI) business by leading R&D development and integration of ink and…
Added 57 mins ago
Principal Systems Engineer - FMS
Aerospace
United States
…project. We are on the lookout for a highly experienced Systems Engineer who is passionate about developing and defining software requirements that meet the evolving needs of our…
Added 1 hour ago
Maintenance Engineer (Engineer 1) - MSS Talent Development Program
Los Alamos National Laboratory
Los Alamos, NM
…a career path towards facilities management. Starting as an Engineer I within the Maintenance and Site Services (MSS) Division, you will be provided the opportunity to experience all…
Added 1 hour ago
Materials Science Post-Bachelor's Engineer
Los Alamos National Laboratory
Los Alamos, NM
The MST-8 Materials Scattering Team is seeking a post-bachelor's engineer to support in situ microstructural characterization using neutron and synchrotron X-ray sources. The successful…
Added 1 hour ago
Sr Mechanical Engineer
Leidos
Albuquerque, NM
Leidos Defense Systems is seeking a highly motivated Sr Mechanical Engineer who loves collaboration and being a part of something big, to join our team. This role drives the implementation…
Added 1 hour ago
Senior/Principal Mechanical Engineer - Experimental Solid Mechanics, Onsite
Sandia National Laboratories
Albuquerque, NM
…security! We are seeking a highly motivated R&D Mechanical Engineer to join our dynamic team in the Experimental Solid Mechanics…
Added 1 hour ago
Computing Infrastructure Engineer, System Administrator Network Management & Virtualization Focus, Onsite
Sandia National Laboratories
Albuquerque, NM
We are seeking a motivated and skilled Computing Infrastructure Engineer (CIE) to join our High Security Operations IT Services team. This role supports the full lifecycle management of…
Added 1 hour ago
Senior/Principal Software Solutions Architect - Data Integration Framework, Hybrid
Sandia National Laboratories
Albuquerque, NM
If so, Sandia National Laboratories' Science and Information Engineering System's group is seeking a motivated Software Solution Architect to join the Design and Development Solutions team…
Added 1 hour ago
EG
Quality Assurance Engineer - PHP
Eliassen Group
Santa Fe, NM
- $40 to $50 USD / hour
Our client seeks a results-driven Senior Quality Assurance Engineer to support Policyholder Portal applications. The role requires manual, automation, mobile, API, and database testing…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job