Required Skills & Experience
-3+ years of software engineering experience, with at least 1 year building LLM-powered systems (RAG pipelines, agent workflows, or fine-tuning) in a production or near-production environment
-Hands-on experience with at least one of the following:
-Retrieval-augmented generation (RAG) and reranking -Agent orchestration with LangGraph or similar frameworks LLM fine-tuning -Strong proficiency in Python, including experience with async code, data pipelines, and REST APIs -Experience with LLM evaluation methodologies (e.g., contributing to or building evaluation datasets or test suites) -Familiarity with observability tools for LLM/agent systems (tracing, logging, monitoring model/tool calls) -Solid understanding of modern LLM and information retrieval concepts, with the ability to discuss trade-offs
Nice to Have Skills & Experience
-Experience with knowledge graphs or property graph query languages -Exposure to document AI (PDF parsing, table extraction, OCR pipelines) -Experience building evaluation datasets and labeling workflows -Understanding of prompt injection risks and mitigation strategies in agent/retrieval systems -Open-source contributions to LangGraph, sentence-transformers, or similar projects
Job Description
You will work across the AI layer of the platform, contributing to retrieval pipelines, agent workflows, and model evaluation. This role is highly hands-on and empirical - you’ll run experiments, measure outcomes, and iterate quickly. You’ll partner closely with the Senior AI Engineer and broader engineering team, taking on increasing ownership as you build depth across the platform’s AI systems.
Key Responsibilities
Contribute to the hybrid retrieval pipeline by implementing and tuning retrieval components, running experiments, and validating results with real evaluations Build and maintain components of the agent orchestration layer, including tool integrations, prompt management, and human-in-the-loop workflows Instrument AI and agent systems for observability (e.g., tracing model/tool calls, tracking token usage and latency, and supporting failure analysis) Develop and maintain evaluation datasets and test suites across retrieval and agent workflows, contributing to CI-level quality gates Support document AI capabilities, including parsing, extraction, and OCR pipelines for new client engagements Implement per-tenant isolation checks across retrieval and agent layers to ensure no cross-tenant data leakage Contribute to model behavior evaluation, including running prompt-injection and adversarial tests as part of ongoing evaluation efforts