Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 1 min ago
Senior R&D Test Engineer - Electrical
Stryker
Flower Mound, TX
As a Senior R&D Test Engineer, Electrical, you'll work closely with electrical, software, firmware, and systems engineers to evaluate new product designs, troubleshoot technical challenges,…
Added 1 min ago
Operational Excellence Manager
Stryker
Fort Lauderdale, FL
…These activities may include digital transformation of the software used by GRCC, process optimization, and customer experience improvement projects. Partners with business leaders to…
Added 1 min ago
Senior Systems Engineer
Philips
San Diego, California, United States, CA
Bachelor's or Master's degree in Systems, Biomedical, Electrical, Mechanical, Software Engineering, or a related discipline, with 5+ years of experience in systems engineering, product…
Added 4 mins ago
Preconstruction Manager
GPAC
Madison, WI
Lead value engineering and constructability reviews.
Added 4 mins ago
Assistant Project Manager - Commercial Construction
GPAC
McKinney, TX
Proficiency with construction management software (Procore, Bluebeam, MS Project, or similar)
Added 4 mins ago
Program Manager III, Companion Diagnostics
Invitrogen Holdings LLC
Carlsbad, CA
Coordinate contributors across assay and reagent development, bioinformatics and software, clinical and analytical validation, regulatory affairs, quality, manufacturing, supply chain,…
Added 5 mins ago
BO
Engineering Technician
Bureau of Safety and Environmental Enforcement
Jefferson, LA
- $50,460 to $72,644 USD / year
Added 5 mins ago
Engineer I-Sr (Pwr Gen)
Entergy
Westwego, LA
Provide engineering and technical oversight in the various Power Generation environments such as: Plants, Plant support, and Fleet Maintenance.
Added 5 mins ago
A
Substation Construction Manager
Actalent
Covington, LA
- $60 to $70 USD / hour
Proficiency with computer tools and software used for project documentation and reporting.
Added 5 mins ago
Appraisal Quality Assurance Analyst (58384)
Applied Technical Services, Inc.
METAIRIE, LA
…accident reconstruction; structural, mechanical, and electrical engineering; fire origin and cause; and environmental services. U.S. Forensic tailors its consulting engineering services to…
Added 5 mins ago
Junior Java Developer
CGI Technologies and Solutions, Inc.
Lafayette, LA
…exciting, positive team that moves quickly and delivers quality software alongside a passionate…
Added 5 mins ago
Java Developer
CGI Technologies and Solutions, Inc.
Lafayette, LA
…exciting, positive team that moves quickly and delivers quality software alongside a passionate…
Added 5 mins ago
General Superintendent
WSP USA
New Orleans, LA
…data and reports into the web-based construction management software application. Assist in preparation of 2-week look ahead…
Added 5 mins ago
Mechanical Engineer - Emerging Capabilities (New Product Development)
Intralox
New Orleans, LA
Join our Emerging Capabilities team and grow your engineering career by helping design, develop, test, and industrialize mechanical and electro-mechanical systems that open new product and…
Added 5 mins ago
Maintenance Manager
Veolia North America
New Orleans, LA
Added 5 mins ago
Senior Engineer
PwC
New Orleans, LA
…applications within our Advisory practice, bringing together software engineering, cloud-based solution delivery, and product innovation work that supports client needs. As a Senior…
Added 5 mins ago
Lead Product Manager AI productivity and adoption, Azure
PwC
New Orleans, LA
…support the development of AI-enabled solutions across the software product lifecycle within our Advisory practice. As a Manager you will lead teams, coach junior professionals, manage…
Added 5 mins ago
Lead Engineer
PwC
New Orleans, LA
…across cloud-native and on-premise environments, using modern engineering approaches, low-code platforms, and a range of programming languages. As a Manager you will lead teams, coach…
Added 5 mins ago
O
Lead Consulting Developer- Python, AI/ML, Distributed System, Microservices
Oracle
Baton Rouge, LA
- $97,500 to $209,500 USD / year
Extensive hands-on software design and programming experience, including deep expertise with Python
Added 5 mins ago
Associate Application Developer Oracle - 2027
IBM
BATON ROUGE, LA
…associates join IBM with a broad foundation in computer science or software engineering, not prior experience in a specific platform or…
Added 5 mins ago
O
Principal Program Manager - OCI Product Marketing
Oracle
Baton Rouge, LA
- $90,100 to $209,500 USD / year
Bachelor’s degree in Business, Communications, Marketing, Computer Science, Engineering, or a related field or equivalent relevant experience.
Added 5 mins ago
CH
Behavioral Scientist, Senior
Cayuse Holdings
Baton Rouge, LA
- $160,000 to $175,000 USD / year
Engage with senior government stakeholders to understand complex business needs, translate them into advanced data engineering and analytics objectives, and develop comprehensive strategies…
Added 5 mins ago
CH
Data Analyst
Cayuse Holdings
Baton Rouge, LA
- $100,000 to $110,000 USD / year
The Data Engineer/Analyst collaborates with clients, data scientists, and software engineers to develop and optimize data pipelines, ETL processes, and database structures that support IRS…
Added 5 mins ago
Senior Software Engineer (Institutional, Financing)
Coinbase
Baton Rouge, LA
As a Senior Software Engineer on the Financing Engineering team within the Institutional group, you'll design, build, and scale the systems that safely manage billions of dollars in loans…
Added 5 mins ago
Project Leader Middletown RI
Serco
Belle Chasse, LA · Barksdale AFB, LA · Alexandria, LA · Breaux Bridge, LA · Bossier City, LA · Fort Polk, LA · Baton Rouge, LA
Have a Bachelor’s degree in computer science, engineering, applied science, or management

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job