Find work in the United States.

6,202,818 jobs across our US network. How we count

Jobs in the United States

25+ jobs

Save this search

Added just now
Maintenance Engineer (Engineer 1) - MSS Talent Development Program
Los Alamos National Laboratory
Los Alamos, NM
…a career path towards facilities management. Starting as an Engineer I within the Maintenance and Site Services (MSS) Division, you will be provided the opportunity to experience all…
Added just now
Sr Mechanical Engineer
Leidos
Albuquerque, NM
Leidos Defense Systems is seeking a highly motivated Sr Mechanical Engineer who loves collaboration and being a part of something big, to join our team. This role drives the implementation…
Added just now
Sr Electrical Engineer
Leidos
Albuquerque, NM
Leidos Defense Systems is seeking a highly motivated Sr Electrical Engineer who loves collaboration and being a part of something big, to join our team. This role drives the implementation…
Added just now
Sr Surety Engineer
Leidos
Albuquerque, NM
Leidos Defense Systems is seeking a highly motivated Sr Surety Engineer who loves collaboration and being a part of something big, to join our team. This role drives the implementation of…
Added just now
Computing Infrastructure Engineer, System Administrator Network Management & Virtualization Focus, Onsite
Sandia National Laboratories
Albuquerque, NM
We are seeking a motivated and skilled Computing Infrastructure Engineer (CIE) to join our High Security Operations IT Services team. This role supports the full lifecycle management of…
Added just now
Senior/Principal Software Solutions Architect - Data Integration Framework, Hybrid
Sandia National Laboratories
Albuquerque, NM
…to translate complex needs into actionable architecture and implementation…
Added just now
O
Associate Support Engineer
Oracle
Santa Fe, NM
- $21.3 to $45.05 USD / hour
Added just now
O
Senior Core Infrastructure Engineer - Nashville, TN
Oracle
Santa Fe, NM
- $79,200 to $209,500 USD / year
–Implements scalability requirements for assigned components and reviews implementation of team members.
Added just now
AWS Connect Developer
ASM Research, An Accenture Federal Services Company
Santa Fe, NM
…The developer also maintains technical documentation, guides implementation decisions, and mentors junior…
Added just now
O
Senior Database Site Reliability Engineer – Oracle Database and GoldenGate (IC3)
Oracle
Santa Fe, NM
- $81,100 to $187,000 USD / year
Oracle Health Database Services is seeking a highly experienced, Site Reliability Engineer to lead database reliability engineering across mission-critical healthcare platforms.
Added 1 min ago
Technical Account Manager
Red Hat
Santa Fe, NM
The Red Hat services team is looking for an experienced System administrator engineer to join us as a Technical Account Manager. In this role, you will provide adoption guidance,…
Added 1 min ago
Senior AWS Cloud Engineer
Serco
Santa Fe, NM
We are seeking a highly skilled and experienced AWS Cloud Engineer to join our IT Operations organization and contribute to the ongoing development of our cloud-native platform.This role…
Added 1 min ago
Mainframe Technical Engineer
Highmark Health
Santa Fe, NM
Collaborate on the design and implementation of with architecture to ensure solution alignment to enterprise goals and standards
Added 1 min ago
Lead Site Reliability Engineer - Remote
SitusAMC
Santa Fe, NM
Drive the development and implementation of automation solutions to remove “toil”, streamline processes, reduce manual interventions, and enhance the overall efficiency of the product…
Added 1 min ago
Software Engineer
Teradata
Santa Fe, NM
This position is for a Software Engineer in the POD Services team within the TD Factory team. The position involves development, integration, functional and performance testing, and fault…
Added 1 min ago
Senior Network Engineer
Cognizant
Santa Fe, NM
Cognizant is seeking a Senior Network Engineer to design, implement, secure, operate, and optimize enterprise datacenter, campus, branch, wireless, and hybrid-cloud networks. This role…
Added 4 mins ago
G
Manufacturing Equipment Engineer
GPAC
Ogden, KS
- $80,000 to $98,000 USD / year
Confidential Manufacturing/Equipment Engineer – Agricultural Equipment
Added 4 mins ago
G
Manufacturing Equipment Engineer
GPAC
Milford, KS
- $80,000 to $98,000 USD / year
Confidential Manufacturing/Equipment Engineer – Agricultural Equipment
Added 6 mins ago
G
Manufacturing Equipment Engineer
GPAC
Wamego, KS
- $80,000 to $98,000 USD / year
Confidential Manufacturing/Equipment Engineer – Agricultural Equipment
Added 7 mins ago
Applications Architect (Enterprise Modernization & Cloud Migration) - (Active Top Secret/SCI Eligibility)
Akima, LLC
Alexandria, VA
…into practical reference architectures, solution designs, and implementation roadmaps aligned with TOGAF, DODAF, or similar industry…
Added 8 mins ago
G
Manufacturing Equipment Engineer
GPAC
Abilene, KS
- $80,000 to $98,000 USD / year
Confidential Manufacturing/Equipment Engineer – Agricultural Equipment
Added 8 mins ago
G
Manufacturing Equipment Engineer
GPAC
Salina, KS
- $80,000 to $98,000 USD / year
Confidential Manufacturing/Equipment Engineer – Agricultural Equipment
Added 8 mins ago
Sr. Principal DevOps Engineer (AHT) - Position qualifies for sign-on bonus starting at $7k
Northrop Grumman
Aurora · Morrisville · Sacramento
Northrop Grumman’s Space Systems Sector is looking for an experienced Sr Principal Cloud Engineer. This position can be based in Aurora, CO, Morrisville, NC or Sacramento, CA.
Added 8 mins ago
DevOps Engineer or Principal DevOps Engineer - Position qualifies for sign-on bonus starting at $5k
Northrop Grumman
Aurora · Morrisville · Sacramento
Northrop Grumman’s Space Systems Sector is looking for an experienced DevOps Engineer or Principal DevOps Engineer. This position can be based in Aurora, CO, Morrisville, NC or Sacramento,…
Added 8 mins ago
Staff DevOps Engineer - Position qualifies for sign-on bonus starting at $10k
Northrop Grumman
Aurora · Morrisville · Sacramento
Northrop Grumman’s Space Systems Sector is looking for an experienced Staff DevOps Engineer to augment our team. This position can be based in Aurora, CO, Morrisville, NC or Sacramento, CA.

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job