Find work in the United States.

6,202,818 jobs across our US network. How we count

Jobs in the United States

25+ jobs

Save this search

Added just now
AI Solutions Engineer
Stratolaunch
Mojave, CA
The AI Solutions Engineer contributes to the development, integration, and operation of AI capabilities across Stratolaunch. The Enterprise Software Team integrates a mix of COTS and custom…
Added 1 min ago
Software Engineer, Lead
The MITRE Corporation
Aberdeen, MD
…integrity at our core, we lead federally funded research and development centers for U.S. government agencies – collaborating with industry and academia to deliver integrated, high-impact…
Added 1 min ago
G
Staff Software Engineer, Applied AI
GridUnity
Remote, United States
- $250,000 to $265,000 USD / year
GridUnity is seeking a Staff Software Engineer to join our Platform team, reporting to the Director of Engineering, Applied AI. You will turn our customer workflow vision into reliable…
Added 4 mins ago
UO
Stationary Engineer (8253C), Facilities Services #89791
University of California, Berkeley
CA
- $65.47 USD / hour
…continuous learning and growth. We actively support professional development by providing all full-time staff employees with at least 80 hours of paid time per year and provide space for…
Added 5 mins ago
Principal ASIC Engineer
Echodyne Corp
Kirkland, WA
Echodyne is seeking a Principal ASIC Engineer to join our Analog/Mixed-Signal (AMS) Engineering team and lead the development and lifecycle support of custom integrated circuits for our…
Added 7 mins ago
UG
Principal Data Engineering - Remote
UnitedHealth Group
Minnetonka, MN
- $112,700 to $193,200 USD / year
The Principal Data Engineer is a senior individual contributor responsible for designing, building, and scaling enterprise data platforms and data pipelines. This role combines cloud-based…
Added 7 mins ago
Spring Migration Engineer
NTT DATA Americas, Inc.
Austin
We are currently seeking a Spring Migration Engineer to join our team in Austin, Texas (US-TX), United States (US).
Added 9 mins ago
SO
IT System Administration - Senior/Specialist
State of Washington
Olympia, WA
- $8,522 to $11,460 USD / month
As a DevOps Engineer, you will help DCYF modernize and improve how technology is delivered by creating automation that makes systems more reliable, secure, and efficient. You will support…
Added 11 mins ago
Software Engineer II - Argo Applications
Frost Bank
San Antonio, TX
Strong working knowledge of JavaScript and modern web application development concepts
Added 11 mins ago
Product Support Engineer
Samsung Electronics America, Inc.
Plano, TX
Position Objective: The DA Product Support Engineer is responsible for managing New Product Introduction (NPI) product quality evaluations, post-launch field quality support, and service…
Added 16 mins ago
CB
Senior Data Engineer
Commercial Bank of California
United States
- $135,000 to $170,000 USD / year
The Senior Data Engineer is responsible for designing, building, and operating scalable, secure data platforms that power enterprise analytics, reporting, and advanced data use cases,…
Added 19 mins ago
T
Manufacturing Engineer
Toyon
Goleta, CA
- $100,000 to $160,000 USD / year
…national security applications. We’re seeking a Manufacturing Engineer to help build next-generation RF sensors for some of the world’s most advanced military…
Added 20 mins ago
Mechanical Engineer Intern - Test Solutions
United States
Support of development hardware procurement and testing
Added 21 mins ago
Applications Development Sr Programmer Analyst
Citibank, N.A.
United States
5-8 years of overall experience in large-scale application development with recent experience in high-volume ELT/ETL processes using PySpark.
Added 21 mins ago
Applications Support Sr Analyst
Citibank, N.A.
United States
The Observability Engineer will support the organization's observability transformation initiatives by helping onboard applications to modern observability platforms and OpenTelemetry…
Added 21 mins ago
DO
Senior Systems Administrator - Hybrid Telework
Department of State Hospitals
CA
- $8,625 to $11,557 USD / month
A hands-on cloud or systems engineer who thrives in complex environments
Added 24 mins ago
Platform Engineer (TS/SCI CI Poly)
Seamless Migration
Reston · Alexandria · Washington · Colorado Springs
Collaborate with development teams to support application integrations, REST APIs, and automated workflows.
Added 24 mins ago
AI Engineer
NCR VOYIX
ATLANTA, GA
…driving data-informed decision-making and contributing to the development of cutting-edge solutions for our…
Added 26 mins ago
AC-DC Power Design Engineer
Apple
San Jose, CA
Apple is looking for an experienced AC-DC Adapter Design Engineer to develop the next generation of high-density offline power supplies for our portable products.
Added 27 mins ago
Test Engineer
Koniag Government Services
Vienna, VA
Must be able to demonstrate expertise in information system design, including application programming on large-scale database management services and the development of complex software to…
Added 27 mins ago
Cloud Platform Engineer IL6/cArmy
Koniag Government Services
Washington, DC
Tuknik Government Services, LLC, a Koniag Government Services company, is seeking a Cloud Platform Engineer IL6/cArmy with a Top Secret security clearance to support TGS in supporting our…
Added 27 mins ago
Generative AI Software Engineer
Koniag Government Services
Washington, DC
Tuknik Government Services, LLC a Koniag Government Services company, is seeking a Generative AI Software Engineer with a Secret security clearance to support TGS in supporting our…
Added 27 mins ago
Lead DevSecOps Appian Engineer
Koniag Government Services
Washington, DC
Tuknik Government Services, LLC a Koniag Government Services company, is seeking a Lead DevSecOps Appian Engineer with a Top Secret security clearance to support TGS in supporting our…
Added 27 mins ago
Jr./Mid-Level Software Engineer
Koniag Government Services
Annapolis, MD
Koniag IT Systems, LLC, a Koniag Government Services company, is seeking a Jr./Mid-Level Software Engineer to support KITS and our government customer. The position is hybrid, with partial…
Added 27 mins ago
Software Developer
Koniag Government Services
Huntsville, AL
KGS is currently seeking an accomplished Senior Full Stack Software Engineer/Developer to spearhead the engineering, enhancement, and architectural modernization of core Line of Business…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job