Overview
TITLE (TRACKING CODE-REQUISITION NUMBER): AI/ML Engineer – LLM Applications
Job Summary
Bowhead is seeking a talented and driven mid-level AI/ML Engineer to join our team in Norco CA supporting a large Navy program. In this role, you will design, develop, and evaluate AI-enabled applications built on large language models (LLMs), including retrieval-augmented generation (RAG) and multi-step agentic workflows. You will build data ingestion and retrieval pipelines, integrate LLMs with enterprise data sources and tools, and implement controls to improve system safety, reliability, and output quality. This position supports deployment in secure, on-premises environments and requires clear documentation to support accreditation and traceability.
As with any position, additional expectations include: meeting deadlines, communicating regularly with assigned supervisors, attending required meetings, and completing other duties as assigned.
Responsibilities
Responsibilities• Design, develop, and maintain LLM-based applications, including RAG solutions and multi-step agentic workflows.• Develop data ingestion, document processing, embedding, and retrieval pipelines.• Integrate LLMs with enterprise data sources, APIs, and downstream tools using tool or function calling and structured outputs.• Implement safety controls, input and output validation, and quality checks to support reliable system behavior.• Build evaluation datasets and testing processes to measure retrieval accuracy, response quality, and overall system performance over time.• Deploy and support open-source or self-hosted models and application pipelines in on-premises environments.• Collaborate with cross-functional teams to translate customer requirements into practical technical solutions.• Document system architecture, data flows, model configurations, evaluation results, and system behavior to support accreditation and traceability.• Follow production software development practices, including version control, automated testing, code review, and maintainable documentation.• Ensure compliance with applicable security protocols and confidentiality requirements.• Stay current with developments in LLMs, retrieval methods, agentic workflows, and model deployment technologies.
Qualifications
Basic Qualifications
- Bachelor’s degree in Computer Science, Data Science, Mathematics, Statistics, Engineering, or a related field.• At least five (5) years of professional software development experience, including three (3) years developing LLM-based applications or retrieval-augmented systems deployed to end users.• Proficiency in Python for production software development, including version control, testing, and code review.• Experience with vector databases, embeddings, and embedding-based retrieval.• Experience integrating LLMs into applications, including prompt design, tool or function calling, and structured outputs.• Familiarity with evaluating LLM system quality, including retrieval relevance, response accuracy, and consistency.• Experience working with open-source or self-hosted language models.• Strong analytical, troubleshooting, and problem-solving skills.• Experience with model serving frameworks and inference platforms.• Experience implementing hybrid keyword and semantic search.• Experience integrating guardrails, content-safety controls, and defenses against prompt injection.• Experience with document parsing, optical character recognition (OCR), and ingestion of complex document formats.• Excellent written and verbal communication skills, with the ability to explain technical concepts to technical and non-technical stakeholders.• Experience working with Atlassian product and project management tools, including Jira and Confluence.• Ability to work onsite five days per week in Norco, California.• If applicable: If you are or have been recently employed by the U.S. government, a post-employment ethics letter will be required if employment is offered.
Preferred Skills and Qualifications• Master’s degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, Engineering, or a related field is a plus.• Knowledge of SQL and experience with PostgreSQL.• Experience with containerized deployment using Docker or Kubernetes.• Familiarity with CI/CD pipelines and DevOps practices for AI-enabled applications.• Experience supporting secure, disconnected, or resource-constrained on-premises environments.• Experience working with classified government data or in a defense-related environment.• Experience preparing technical documentation and evidence to support system accreditation and audit requirements.
Physical Demands:
- Must be able to lift up to 50 pounds
- Must be able to stand and walk for prolonged amounts of time
- Must be able to twist, bend and squat periodically
Target salary is Salary Range: $100,000–$135,000 annually, depending on qualifications and experience
SECURITY CLEARANCE REQUIREMENTS: Must currently hold a security clearance at the Secret level. US Citizenship is a requirement for Secret clearance at this location.