Find work in the United States.

6,202,818 jobs across our US network. How we count

Jobs in the United States

25+ jobs

Save this search

Added just now
Civil Engineer - Entry
VLMK Engineering + Design
Portland, OR
…building design services for a variety of clients in the A/E/C and development industry. Now with offices in Vancouver, WA and Phoenix, AZ and a staff of almost 50 companywide, we have a…
Added just now
G
Project Manager
GPAC
Waterville, ME
- $90,000 to $130,000 USD / year
We are seeking an experienced Professional Engineer (PE) to join our Land Development team in a Project Manager capacity. This individual will play a key role in leading projects, managing…
Added just now
G
Manufacturing Equipment Engineer
GPAC
Junction City, KS
- $80,000 to $98,000 USD / year
Manufacturing Engineer • Equipment Engineer • Mechanical Engineer • Agricultural Equipment • Agricultural Machinery • Fabrication Equipment • Heavy Equipment • Product Design •…
Added 1 min ago
G
HVAC Designer
GPAC
San Ramon, CA
- $75,000 to $110,000 USD / year
A well-established local Mechanical Contractor is looking to fill a HVAC Designer Engineer position working on large projects.
Added 2 mins ago
Software Engineer (Apps) 4
WEX
Remote, United States
We are seeking a seasoned Sr. Software Engineer in the WEX Mobility Engineering organization. This role will sit in the Mobility team that caters to fleet management and mobility payments…
Added 2 mins ago
G
Sales Engineer
GPAC
Cleveland, OH
- $70,000 to $110,000 USD / year
Position Summary: The Sales Engineer will support the company’s strategic growth by identifying and developing new opportunities in both existing and potential markets. This role requires a…
Added 3 mins ago
Senior Software Engineer - AL/ML Platforms
Nike
Beaverton, OR
This Software Engineer will develop and/or implement services and paved-path solutions to enable data scientists and ML engineers at Nike. These services will enable and accelerate model…
Added 4 mins ago
G
Engineering Manager
GPAC
Carlinville, IL
- $100,000 to $135,000 USD / year
An established precision manufacturer is seeking an experienced Engineering Manager to lead engineering operations, CNC programming, process improvements, and technical team development.
Added 4 mins ago
G
Engineering Manager
GPAC
Auburn, IL
- $100,000 to $135,000 USD / year
An established precision manufacturer is seeking an experienced Engineering Manager to lead engineering operations, CNC programming, process improvements, and technical team development.
Added 4 mins ago
G
Engineering Manager
GPAC
Divernon, IL
- $100,000 to $135,000 USD / year
An established precision manufacturer is seeking an experienced Engineering Manager to lead engineering operations, CNC programming, process improvements, and technical team development.
Added 4 mins ago
M
Teleop Platform Engineer, Real-Time Systems & UI
Mecka
New York, NY
- $200,000 to $240,000 USD / year
- Game development, VR/XR, or other latency-sensitive interaction design.
Added 4 mins ago
G
Engineering Manager
GPAC
Decatur, IL
- $100,000 to $135,000 USD / year
An established precision manufacturer is seeking an experienced Engineering Manager to lead engineering operations, CNC programming, process improvements, and technical team development.
Added 4 mins ago
G
Engineering Manager
GPAC
Lincoln, IL
- $100,000 to $135,000 USD / year
An established precision manufacturer is seeking an experienced Engineering Manager to lead engineering operations, CNC programming, process improvements, and technical team development.
Added 4 mins ago
G
Engineering Manager
GPAC
Jacksonville, IL
- $100,000 to $135,000 USD / year
An established precision manufacturer is seeking an experienced Engineering Manager to lead engineering operations, CNC programming, process improvements, and technical team development.
Added 4 mins ago
G
Engineering Manager
GPAC
Rochester, IL
- $100,000 to $135,000 USD / year
An established precision manufacturer is seeking an experienced Engineering Manager to lead engineering operations, CNC programming, process improvements, and technical team development.
Added 5 mins ago
Senior SSD Debug Engineer - 2270
Kioxia
Folsom, CA
As an SSD Debug Engineer, your primary role is to triage and debug SSD drive failures for our data center SSD product line. In this role, you will work closely with the Product Support…
Added 5 mins ago
IT Manufacturing Data Solutions Engineer
Corning Incorporated
Newton, NC
Trusted products that accelerate drug discovery, development, and delivery to save lives
Added 5 mins ago
Seibi Technician - Production Engineering
Toyota
Liberty, NC
Industrial Controls Engineer/Engineering Tech experience in a manufacturing environment, including equipment installation, modification, and optimization
Added 5 mins ago
Test Infrastructure Technician I (Sun.-Tues.-Sun. - Wed. 6am-6pm)
Jabil Inc.
Florence, KY
· Supports the Test Engineer in the development and commission of test solutions executing the test qualification of the fixture and program as per the specific system qualification process.
Added 5 mins ago
Test Infrastructure Technician I (Wed.- Sat./Thurs.-Sat. 6am-6pm)
Jabil Inc.
Florence, KY
· Supports the Test Engineer in the development and commission of test solutions executing the test qualification of the fixture and program as per the specific system qualification process.
Added 5 mins ago
Automation Engineer I/II (MES)
Novo Nordisk
Clayton, NC
Own self-development & initiative to understand the site process control systems
Added 5 mins ago
Packaging Engineer Co-Op
Coty
Sanford, NC
Kick off a challenging and rewarding engineering career with Coty! Our Supply Chain Packaging Engineer Co-Op assists the Research and Development Packaging team in the definition of…
Added 5 mins ago
Sr. Associate Design Engineer
Caterpillar, Inc.
Sanford, NC
Caterpillar’s Building Construction Products division in Sanford, NC is hiring a Senior Associate Design Engineer in the Compact Track Loaders (CTL) and Skid Steer Loader (SSL)…
Added 5 mins ago
Electrical Design Engineer
Lafayette, IN · Mossville, IL
As a Design Engineer with the Large Power Systems Division (LPSD), you will have the opportunity to direct and contribute to the concept, development, and design of complex components,…
Added 5 mins ago
Senior Performance/Simulation/Application Engineer
Mossville, IL
The Aftertreatment & Exhaust Systems (AT&ES) organization is seeking a Senior Performance, Simulation, and Application Engineer to provide technical leadership across simulation, controls…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job