Jobs in the United States

25+ jobs

Group jobs by employerEmail me new jobs Save this search

Added 1 min ago
Sr Principal Engineer, Rendezvous & Proximity Operations (RPO), Guidance, Navigation & Control, Lead
Sierra Space
Centennial, CO · Louisville, CO
Sierra Space is seeking Sr Principal Guidance, Navigation & Control Engineer to lead the development, integration, and innovation of advanced GNC systems for spacecraft applications. This…
Added 1 min ago
Software Engineering Manager
Terabase Energy
Remote, United States
…owners, and engineering & construction firms to support the design, optimization, and construction of huge solar projects around the world. Our team is a blend of solar-industry veterans…
Added 2 mins ago
Educational Concierge
Innovative Network of Knowledge
Remote, United States
Innovative Network of Knowledge seeks to change education forever through innovation and technology. We offer a well of knowledge, a spring of possibilities and a place where your family…
Added 3 mins ago
Housekeeping Room Inspector - The Sinclair
Coury Hospitality
Fort Worth, TX
At The Sinclair, Autograph Collection, we redefine luxury and technology in downtown Fort Worth. As a cutting-edge smart hotel, The Sinclair offers high-tech guest experiences, sleek…
Added 3 mins ago
Volunteer Expert Reviewer
LOVE JUSTICE INTERNATIONAL
Remote, United States
Organizational development and leadership - organizational culture and values, management and leadership development, performance management, team collaboration, operations management,…
Added 4 mins ago
Senior Principal, AI/ML
Hewlett Packard Enterprise
San Jose, CA
This role has been designed as 'Hybrid' with a requirement that you will work on average 2 days per week from an HPE office
Added 4 mins ago
Principal Solutions Architecture
Hewlett Packard Enterprise
All, MI
Provides mentoring to others to set an example of software systems design and development innovation and excellence.
Added 4 mins ago
Principal Cloud Developer
Hewlett Packard Enterprise
Ft. Collins, CO
This role has been designed as 'Hybrid' with a requirement that you will work on average 2 days per week from an HPE office
Added 5 mins ago
Manager, Learning and Development Automotive
AAA Northern California, Nevada & Utah
Walnut Creek, CA
Deep expertise in adult learning theory, instructional design frameworks (ADDIE/SAM), multi-modal curriculum development, e-learning design, and technical content creation.
Added 5 mins ago
PD&M Specialist III
Faith Technologies, Inc.
Remote, United States
Must have some experience in tool/add-in creation
Added 5 mins ago
Business System Administrator I
Faith Technologies, Inc.
Remote, United States
…customers, vendor support teams, as well as with the business technology team. The Business Systems Administrator I role will leverage virtual collaboration tools to engage with their…
Added 5 mins ago
Business System Administrator II
Faith Technologies, Inc.
Remote, United States
…customers, vendor support teams, as well as with the business technology team. Additionally, will leverage virtual collaboration tools to engage with their customers, troubleshoot and…
Added 5 mins ago
Intern I (Convention & Event Services [CCT] Dept., Unclassified Position) *Multiple Vacancies, Summer 2027 Term*
City of Dallas
United States
…functions through hands-on learning, job shadowing, professional development workshops, and individualized…
Added 6 mins ago
Engineering Tech II/Sr/Lead
Wolf Creek Nuclear Operating Corporation
Burlington, KS
Must have a satisfactory work record including excellent attendance
Added 6 mins ago
Sr. IAM Engineer | Remote, USA
Optiv
Leawood, KS
You will collaborate with cross-functional teams, lead IAM initiatives, and contribute to the development and execution of the client's overall security strategy.
Added 6 mins ago
Principal Sales Representative, CAS (Kansas City)
Medtronic
Kansas City, KS
The individual must reside within the territory and regularly travel to customer accounts, including hospitals, clinics, and procedural settings
Added 6 mins ago
ServiceNow Platform Architect and Engineering Lead Job Details | Black & Veatch Family of Companies
Black & Veatch
Overland Park, KS
Act as the design authority for the ServiceNow platform across Information Technology Service Management (ITSM), IT Operations Management (ITOM), Security Operations (SecOps), Integrated…
Added 6 mins ago
D
Advertising Sales Intern
DIRECTV
Wichita, KS
- $20 to $28 USD / hour
Duration: 10-week hybrid internship (in New York City office 2 days per week) · Eligibility: Must have full U.S
Added 6 mins ago
A
Project Controls Analyst
Actalent
Wichita, KS
- $60 to $70 USD / hour
…intersection of project controls, cost analysis, forecasting, and technology? If so, this could be the opportunity to take your project controls career to the next…
Added 6 mins ago
Product Manager
Meta
Topeka, KS
Plan, initiate, and manage information technology projects for web-based products, applications, and platforms
Added 6 mins ago
Software Engineer - AI
SHI
Topeka, KS
Since 1989, SHI International Corp. has helped organizations change the world through technology. We’ve grown every year since, and today we’re proud to be a $16 billion global provider of…
Added 6 mins ago
F
Staff Product Designer, Financial Products
Faire
New York City, NY · San Francisco, CA · NY
- $216,000 to $297,000 USD / year
Specific Workplace and Information Technology positions may require onsite attendance 5 days per week as will be indicated in the job posting
Added 6 mins ago
O
Enterprise Architect - Network and Connectivity
Oracle
Topeka, KS
- $97,800 to $209,500 USD / year
…on non-routine and very complex aspects of applications and technology installations. Provides direction and mentoring to project teams, and effectively influences customer leadership on…
Added 6 mins ago
Senior CXO Customer Advocacy Manager
Rubrik
Topeka, KS
Rubrik's Senior CXO Customer Advocacy Manager sits at the intersection of technology, storytelling, and enterprise customer engagement. It lives within the CXO Transformation team, a…
Added 6 mins ago
O
Federal Oracle Health Automation Engineer - Eggplant
Oracle
Topeka, KS
- $23.8 to $56.83 USD / hour
Understanding of Oracle Health/Cerner Millennium solutions , software development and coding best practices, quality assurance methodologies, and technology design principles.

Bask Health

AI Evaluation Engineer

Remote, United States

Apply on apply.workable.com About employer

Applications open in a new tab.

Job details

Experience: 3+ years of professional experience in software engineering, AI engineering, test automation, quality engineering, or a closely related technical field.
Posting language: English
Posted: Nov 12, 2026

Original job description

About Bask Health Bask Health is a software platform that enables entrepreneurs, doctors, and healthcare businesses to launch, manage, and grow direct-to-consumer telehealth businesses. Our platform brings together patient experiences, clinical workflows, pharmacy operations, and commerce tools in one place. We're an entrepreneurial, product-focused engineering team that values ownership, technical excellence, and building software that solves real problems. Engineers work closely with product teams and company leadership, take features from idea to production, and have a direct role in shaping the platform. Engineering at Bask is AI-first and builder-led. We use tools like Cursor and Claude Code throughout development while maintaining high standards for reliability, testing, and software quality. About the Role We're looking for an AI Evaluation Engineer to help ensure that AI-powered features across Bask Health behave accurately, reliably, and consistently before and after they reach production. Bask Health supports the technology behind direct-to-consumer telehealth businesses, connecting patient experiences, merchant operations, clinical workflows, pharmacy coordination, and commerce capabilities. As we introduce more AI-powered functionality, we need dependable ways to verify that these systems work as intended under realistic conditions. You'll build and maintain evaluation workflows that test large language models, AI assistants, retrieval systems, and automated features against clearly defined product requirements. This is a hands-on engineering role that combines AI evaluation, software testing, automation, and production quality assurance. You'll develop test scenarios, evaluation datasets, automated checks, and tools that help engineering teams identify problems before they affect users. Your work may include testing AI-generated responses, validating structured outputs, evaluating tool-calling behavior, checking retrieval accuracy, and identifying regressions caused by changes to prompts, models, or application code. You'll collaborate with applied AI engineers, machine learning engineers, backend engineers, product managers, and quality-focused engineering teams to define expected behavior and translate those expectations into repeatable evaluations. You'll also help establish processes for reviewing AI failures, documenting recurring issues, and determining whether a feature meets its acceptance criteria. You'll work with engineers to reproduce problems and verify that fixes address the underlying issue. Because Bask operates in healthcare-related environments, you'll pay particular attention to privacy, permissions, unsupported medical claims, and AI behavior that could interfere with approved clinical or pharmacy workflows. Evaluation systems should help verify that AI features remain within their intended operational boundaries. This position emphasizes practical testing, thoughtful engineering, and continuous improvement. You'll contribute to evaluation standards while independently implementing and maintaining the tools and tests that support AI quality. We're looking for an engineer who enjoys finding difficult edge cases, understands how AI behavior differs from traditional deterministic software, and wants to help build products that users can depend on. What You'll Do

Design and implement evaluation workflows for AI-powered applications across Bask Health.
Build automated tests for large language model outputs and AI application behavior.
Develop evaluation datasets that represent realistic user requests and product workflows.
Create test scenarios for patient-facing, merchant-facing, and internal AI features.
Implement checks for response accuracy, relevance, completeness, and consistency.
Evaluate AI-generated content for unsupported claims and misleading information.
Test structured model outputs for schema compliance and application compatibility.
Build evaluations for AI tool calling and authorized action execution.
Validate that AI assistants respect application permissions and tenant boundaries.
Test retrieval-augmented generation systems for grounding and information relevance.
Develop regression tests that identify unexpected changes in AI behavior.
Integrate evaluation checks into engineering development and release workflows.
Create automated comparison tests for prompts, model versions, and configuration changes.
Develop scripts and utilities that simplify AI evaluation execution.
Build reporting tools that summarize evaluation results and failure patterns.
Investigate failed evaluations and help engineers reproduce underlying issues.
Work with product managers to define expected AI behavior and acceptance criteria.
Collaborate with applied AI engineers to improve prompts and application logic.
Test AI applications against unusual inputs, ambiguous requests, and difficult edge cases.
Develop evaluation cases for prompt injection and unauthorized tool-use attempts.
Support human review workflows for outputs that require subjective assessment.
Track recurring AI failures and help prioritize reliability improvements.
Monitor production evaluation signals and investigate quality regressions.
Document test coverage, evaluation procedures, and known limitations.
Improve evaluation efficiency, repeatability, and maintainability.
Participate in code reviews, engineering planning, and technical discussions.
Support controlled testing and rollout of new AI-powered features.
Own assigned AI evaluation projects from test design through implementation, reporting, and ongoing improvement.

Requirements

3+ years of professional experience in software engineering, AI engineering, test automation, quality engineering, or a closely related technical field.
Experience developing automated tests for production software applications.
Strong programming skills in Python, TypeScript, JavaScript, or comparable languages.
Familiarity with large language models and modern AI-powered applications.
Understanding of software testing methodologies and test case design.
Experience building automated testing scripts, frameworks, or validation tools.
Familiarity with API testing and backend application integrations.
Understanding of prompt engineering and common AI model failure modes.
Ability to evaluate AI-generated responses against expected requirements.
Experience working with structured data and validating application outputs.
Familiarity with regression testing and software release processes.
Experience debugging application failures and reproducing technical issues.
Understanding of authentication, authorization, and secure application behavior.
Familiarity with databases and application data models.
Experience working with Git and collaborative software development workflows.
Understanding of continuous integration and automated testing pipelines.
Familiarity with logging, monitoring, and production troubleshooting.
Ability to create clear, reproducible test cases.
Strong analytical thinking and attention to detail.
Ability to distinguish application defects from model behavior limitations.
Experience documenting technical issues and communicating findings.
Familiarity with cloud-based application environments.
Ability to collaborate effectively with engineers, product managers, and other stakeholders.
Strong problem-solving skills and a quality-focused engineering mindset.
Ability to independently own evaluation and test automation projects.

Nice to Have

Experience evaluating large language models or generative AI applications.
Familiarity with OpenAI, Anthropic, or comparable model APIs.
Experience with LangSmith, DeepEval, Ragas, promptfoo, or comparable evaluation tools.
Familiarity with LLM-as-a-judge evaluation techniques.
Experience developing AI evaluation datasets and test benchmarks.
Understanding of human evaluation and response labeling workflows.
Experience testing retrieval-augmented generation systems.
Familiarity with embeddings, semantic search, and vector databases.
Experience evaluating AI agents and tool-calling behavior.
Familiarity with prompt injection testing and AI security evaluations.
Experience implementing AI regression tests in CI/CD pipelines.
Experience building automated evaluation reports or dashboards.
Familiarity with Playwright, Jest, Vitest, Pytest, or comparable testing frameworks.
Experience testing TypeScript or Node.js applications.
Familiarity with React or Next.js application testing.
Experience working with AWS or serverless infrastructure.
Experience evaluating AI features in healthcare or other sensitive environments.
Understanding of HIPAA and healthcare data privacy considerations.
Experience developing human-in-the-loop quality review processes.
Experience using AI-assisted development tools such as Cursor or Claude Code.

Apply on apply.workable.com About employer

Report this job

Jobs in the United States

Sr Principal Engineer, Rendezvous & Proximity Operations (RPO), Guidance, Navigation & Control, Lead

Software Engineering Manager

Educational Concierge

Housekeeping Room Inspector - The Sinclair

Volunteer Expert Reviewer

Senior Principal, AI/ML

Principal Solutions Architecture

Principal Cloud Developer

Manager, Learning and Development Automotive

PD&M Specialist III

Business System Administrator I

Business System Administrator II

Intern I (Convention & Event Services [CCT] Dept., Unclassified Position) Multiple Vacancies, Summer 2027 Term

Engineering Tech II/Sr/Lead

Sr. IAM Engineer | Remote, USA

Principal Sales Representative, CAS (Kansas City)

ServiceNow Platform Architect and Engineering Lead Job Details | Black & Veatch Family of Companies

Advertising Sales Intern

Project Controls Analyst

Product Manager

Software Engineer - AI

Staff Product Designer, Financial Products

Enterprise Architect - Network and Connectivity

Senior CXO Customer Advocacy Manager

Federal Oracle Health Automation Engineer - Eggplant

Contact us

Work requirements

Jobs in the United States