Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 1 min ago
MA
Principal Corporate Technology Engineer
Microsoft AI
NY · Mountain View, CA · Redmond, WA
- $142,800 to $304,200 USD / year
We are looking for a Principal Corporate Technology Engineer to engineer the identity, SaaS, access-governance, internal application, integration, and endpoint capabilities that Microsoft…
Added 2 mins ago
Distinguished Engineer, AI Security Systems (Remote)
CrowdStrike
Tallahassee, FL
We are seeking a Distinguished Engineer who has built AI and security systems that operate in the real world and stop real adversaries.
Added 21 mins ago
Sr Validation Engineer (RF, mmWave & Mixed-Signal) - TeraWave
Blue Origin Personnel, LLC
San Diego, CA · Cupertino, CA
…interfaces. As a Senior RF/mmWave & Mixed-Signal Validation Engineer, you prove that silicon works on the bench. You bring up first samples, characterize RF and mixed signal performance…
Added 25 mins ago
G
Staff Software Engineer, Full Stack
Gridware
San Francisco, CA
- $245,000 to $260,000 USD / year
We’re seeking a Staff Software Engineer who will serve as a technical force multiplier and elevate the entire software engineering organization through architectural excellence, mentorship,…
Added 27 mins ago
H
Advanced Chemical Engineer- Fired Heaters
Honeywell
Rosemont, IL · Houston, TX
- $104,000 to $149,000 USD / year
Execute fired heater thermal design and performance evaluations, including advanced use of FRNC Software.
Added 27 mins ago
H
Advanced Mechanical Engineer - Heat Exchangers
Honeywell
Rosemont, IL · Houston, TX
- $114,000 to $163,000 USD / year
Significant experience in the design and selection of heat exchangers including experience with HTRI or similar exchanger design software
Added 34 mins ago
Principal Software Engineer
DriveTime
Tempe, AZ
In short, as a Principal Software Engineer, you will serve as a technical leader responsible for driving complex, business-critical initiatives across the organization. You will partner…
Added 41 mins ago
W
Senior Software Engineer (Quality, Health Records)
Whoop
Boston, MA
- $150,000 to $215,000 USD / year
As a Senior Software Engineer you will own quality engineering for Health Records while building shared data-mocking capabilities that support teams across WHOOP.
Added 41 mins ago
O
Lead Principal Software Engineer, Core Infrastructure
Oracle
Nashville, TN
- $146,300 to $306,400 USD / year
As a Consulting Member of Technical Staff, you will own the software design and development for major components of Oracle's Cloud Infrastructure. You should be both a rock-solid lead…
Added 41 mins ago
O
Lead Principal Platform Software Engineer
Oracle
Seattle, WA · Santa Clara, CA
- $113,600 to $306,400 USD / year
We are seeking a Lead Principal Platform Software Engineer to lead the architecture and delivery of critical platform systems and initiatives across teams. This is a hands-on technical…
Added 43 mins ago
EG
Quality Assurance Engineer - PHP
Eliassen Group
Montpelier, VT
- $40 to $50 USD / hour
Our client seeks a results-driven Senior Quality Assurance Engineer to support Policyholder Portal applications. The role requires manual, automation, mobile, API, and database testing…
Added 54 mins ago
ZT
Quality Control Laboratory Technician, Support I
Zebra Technologies
Morris Plains, NJ
- $23 USD / hour
Monitor and report test status of various production runs to QC Manager, Manufacturing Manager, QC Supervisor, Technology, or Process Engineer.
Added 54 mins ago
Lead AI Engineer
Guardian Life
Holmdel, NJ
The employee serves as a Lead AI Engineer responsible for architecting, developing, deploying, and supporting enterprise-scale Agentic AI solutions.
Added 56 mins ago
Sr. Antenna Production Engineer
Echodyne Corp
Kirkland, WA
…combines the patented technology of metamaterials with powerful software to create a radar sensor with unprecedented performance at commercial price points. Echodyne offers its radars to…
Added 59 mins ago
WI
(USA) Systems and Infrastructure Engineer III, Information Security
Walmart Inc.
HERNDON, VA
- $90,000 to $216,000 USD / year
The Systems and Infrastructure Engineer III, Information Security, plays a critical role in designing, implementing, and maintaining secure and scalable infrastructure solutions. This…
Added 59 mins ago
WI
(USA) Staff, Systems And Infrastructure Engineer, Information Security
Walmart Inc.
HERNDON, VA
- $110,000 to $264,000 USD / year
The (USA) Staff, Systems and Infrastructure Engineer, Information Security, plays a critical role in designing, implementing, and maintaining secure and scalable IT infrastructure. This…
Added 1 hour ago
TT
Forward Deployed Engineer
True Talent Solutions Partners
San Francisco, CA
- $160,000 to $275,000 USD / year
Join a team building AI-powered automation for complex enterprise workflows. You will partner closely with customers to understand critical processes, deliver production software and…
Added 1 hour ago
Analytics Engineer (X)
SpaceXSI
Palo Alto, CA
We are seeking a skilled Analytics Engineer to build and maintain robust data systems that enable high-impact quantitative analysis and business decision-making. This role combines strong…
Added 1 hour ago
IG
Software Quality Engineer
Insight Global
Sunset Hills, MO
- $35 to $40 USD / hour
We are seeking a Junior Software Quality Engineer with strong test automation experience and prior hands-on experience applying AI-assisted testing practices. In this role, you will help…
Added 1 hour ago
H
R&D Writing System Engineer
HP
Corvallis
- $93,400 to $143,800 USD / year
…HP’s industry leading print businesses. This Specialist-level engineer primarily supports the PageWide Industrial (PWI) business by leading R&D development and integration of ink and…
Added 1 hour ago
EG
Quality Assurance Engineer - PHP
Eliassen Group
Santa Fe, NM
- $40 to $50 USD / hour
Our client seeks a results-driven Senior Quality Assurance Engineer to support Policyholder Portal applications. The role requires manual, automation, mobile, API, and database testing…
Added 1 hour ago
Automation & Controls Software Engineer
Enchanted Rock Management LLC
Houston, TX
The Automation & Controls Engineer develops, tests, deploys, and maintains control logic and software for ERock’s power generation and microgrid systems. The role focuses on PLC and RTAC…
Added 1 hour ago
G
Manufacturing Equipment Engineer
GPAC
Ogden, KS
- $80,000 to $98,000 USD / year
Manufacturing Engineer • Equipment Engineer • Mechanical Engineer • Agricultural Equipment • Agricultural Machinery • Fabrication Equipment • Heavy Equipment • Product Design •…
Added 1 hour ago
G
Manufacturing Equipment Engineer
GPAC
Milford, KS
- $80,000 to $98,000 USD / year
Manufacturing Engineer • Equipment Engineer • Mechanical Engineer • Agricultural Equipment • Agricultural Machinery • Fabrication Equipment • Heavy Equipment • Product Design •…
Added 1 hour ago
G
Manufacturing Equipment Engineer
GPAC
Wamego, KS
- $80,000 to $98,000 USD / year
Manufacturing Engineer • Equipment Engineer • Mechanical Engineer • Agricultural Equipment • Agricultural Machinery • Fabrication Equipment • Heavy Equipment • Product Design •…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job