Job#: 3058816
Job Description:
About the Role We are seeking a Principal AI Platform Engineer to help build, operate, and scale an enterprise AI platform supporting a growing portfolio of AI applications, agents, and business use cases. This is a highly hands-on Principal-level position with significant technical ownership and organizational influence. You will own critical components of the AI platform, including AI gateways, GPU/model serving, RAG, observability, cost management, security, governance, and production operations. This is an opportunity to join at an important stage of the organization's AI journey. You will help establish the technical standards, infrastructure, and operating model that future AI solutions are built against. What You'll Be Doing
- Own key components of the enterprise AI platform from infrastructure through production operations.
- Build and operate an AI gateway supporting multiple models/providers, including routing, fallback, security, quotas, and cost allocation.
- Deploy and operate open-weight models on GPU infrastructure using technologies such as vLLM, NVIDIA Triton/TensorRT-LLM, or similar.
- Build and operate production RAG systems, including vector stores, embedding pipelines, document processing, retrieval, and governed knowledge sources.
- Establish AI observability and cost management, including visibility into token usage, GPU utilization, application/team consumption, performance, and overall platform cost.
- Implement tracing, monitoring, evaluation, and regression testing using tools such as Langfuse, LangSmith, Arize, or similar platforms.
- Deploy, monitor, troubleshoot, and support production AI agents and workflows.
- Evaluate model performance and economics to determine the appropriate balance between managed cloud models and self-hosted/open-weight solutions.
- Partner with engineering, infrastructure, security, operations, and business teams to operationalize AI safely and efficiently.
- Help establish standards around AI security, model access, RBAC, guardrails, governance, and production readiness.
- Provide technical guidance around model selection, routing, caching, batching, token optimization, and cost-versus-capability decisions.
- Communicate technical recommendations effectively with engineering teams, architects, and senior leadership.
- Remain hands-on with engineering and coding while providing Principal-level technical leadership.
Required Qualifications
- 6+ years of experience in Platform Engineering, Site Reliability Engineering, Infrastructure Engineering, or related technical operations.
- 2+ years running LLM/AI systems in production for real users, or significant ML platform operations experience.
- Hands-on experience implementing and operating an AI/LLM gateway in front of one or more model providers.
- Examples include LiteLLM, Azure APIM, Kong, Portkey, or similar.
- Hands-on experience deploying open-weight models on GPUs.
- Experience with vLLM, NVIDIA Triton/TensorRT-LLM, or comparable technologies strongly preferred.
- Experience building and operating RAG solutions in production.
- Experience with:
- Vector stores
- Embedding pipelines
- Document processing
- Azure AI Search, pgvector, or similar technologies
- Experience measuring and managing AI/LLM costs by application, workload, or team.
- Experience with AI observability, tracing, monitoring, and evaluation.
- Strong hands-on engineering skills with:
- Python
- Terraform and/or Bicep
- CI/CD
- Kubernetes
- AKS or comparable Kubernetes environments
This employer is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or protected veteran status and will not be discriminated against on the basis of disability.