Archived listing. The details below describe a past opening.
turbopuffer is looking for a software engineer who wants to build inference for searching +1T documents.
what you'll do:
- work closely with our inference providers to make sure that the p99 inference latency stays as low as possible
- write code to improve how the database interacts with our inference providers
- integrate new embedding & reranking models into turbopuffer
- automate yourself out of a job
must have:
- engineering
- you've worked in a systems language (Rust, C++, Java, Go, etc.)
- experience in inference engineering, e.g. managing a production deployment of an LLM/embedding/reranking model
- worked on handling/processing large datasets
- you love shipping things to prod
- domain expertise
- you understand the tradeoff between latency and throughput for model deployments and you know how to think about improving model performance
- keeping up with state-of-the-art embedding & reranker models and their evaluation
- you’re familiar with the most common inference engines like vLLM and SGLang
- you know how to manage the infra of model deployments and how to reason about GPU autoscaling.
- typist - you can write well and can explain complex ideas simply
- human - you build trust and admit what you don’t know
nice to have:
- an interest in embedding & reranker model training
- prior experience in vector and full-text search
- knowledge about in-depth areas of inference engines & GPU engineering
tpuf is hardcore & whimsical. we value:
- overstep > understep
- show > tell
- complexity is earned
- showing up as part of our customer’s team
- making stuff easy to react to