Open jobs in the United States

Save this search

Added 16 mins ago
TT
Staff Product Manager
True Talent Solutions Partners
New York, NY
- $180,000 to $250,000 USD / year
Added 16 mins ago
TT
Senior Software Engineer
True Talent Solutions Partners
San Francisco, CA
- $200,000 to $240,000 USD / year
Added 16 mins ago
TT
Member of Technical Staff, Applied Researcher
True Talent Solutions Partners
New York, NY
- $300,000 to $500,000 USD / year
Added 16 mins ago
TT
Member of Technical Staff, AI Agents
True Talent Solutions Partners
New York, NY
- $150,000 to $300,000 USD / year
Added 16 mins ago
TT
Principal Software Engineer, Perception
True Talent Solutions Partners
Alameda, CA
- $300,000 to $380,000 USD / year
Added 47 mins ago
TT
Senior/Staff Product Engineer, Applied AI
True Talent Solutions Partners
San Francisco, CA
- $200,000 to $300,000 USD / year
Added 47 mins ago
TT
Lead Platform Engineer
True Talent Solutions Partners
San Francisco, CA
- $220,000 to $300,000 USD / year
Added 47 mins ago
TT
Backend Infrastructure Engineer, Anti-Bot
True Talent Solutions Partners
San Francisco, CA
- $235,000 to $260,000 USD / year
Added 47 mins ago
TT
Software Engineer, C++ Systems
True Talent Solutions Partners
San Francisco, CA
- $250,000 to $325,000 USD / year
Added 47 mins ago
TT
Litigator / Transactional Attorney, HOA Law
True Talent Solutions Partners
Los Angeles, CA
- $155,000 to $195,000 USD / year
Added 47 mins ago
TT
Lead Software Engineer, Platform
True Talent Solutions Partners
San Francisco, CA
- $220,000 to $320,000 USD / year
Added 1 hour ago
TT
Trusts and Estates Associate
True Talent Solutions Partners
Atlanta, GA
- $235,000 to $300,000 USD / year
Added 1 hour ago
TT
Litigation Associate
True Talent Solutions Partners
New York City, NY
- $125,000 to $165,000 USD / year
Added 1 hour ago
TT
Case Investigation Attorney, Securities Class Action
True Talent Solutions Partners
Boston, MA
- $160,000 to $220,000 USD / year
Added 1 hour ago
TT
Bankruptcy Attorney, Debtor Litigation, Partner or Senior Associate
True Talent Solutions Partners
New York, NY
- $275,000 to $400,000 USD / year
Added 1 hour ago
TT
Senior Litigation Attorney
True Talent Solutions Partners
Eagle, ID
- $130,000 to $150,000 USD / year
Added 1 hour ago
TT
Technical Product Manager
True Talent Solutions Partners
Sunnyvale, CA
- $150,000 to $250,000 USD / year
Added 1 hour ago
TT
Software Engineer
True Talent Solutions Partners
New York, NY
- $180,000 to $230,000 USD / year
Added 1 hour ago
TT
Strategic Projects Lead (Legal)
True Talent Solutions Partners
San Francisco, CA
- $150,000 to $200,000 USD / year
Added 1 hour ago
TT
Senior Control Electronics Engineer
True Talent Solutions Partners
Rio Rancho, NM
- $150,000 to $200,000 USD / year
Added 1 hour ago
TT
Principal Analog/Mixed Signal Engineer
True Talent Solutions Partners
Boston, MA
- $200,000 to $325,000 USD / year
Added 1 hour ago
TT
Founding Product Engineer
True Talent Solutions Partners
San Francisco, CA
- $165,000 to $210,000 USD / year
Added 1 hour ago
TT
Fullstack Software Engineer
True Talent Solutions Partners
San Francisco, CA
- $160,000 to $200,000 USD / year
Added 1 hour ago
TT
Frontend Engineer
True Talent Solutions Partners
New York, NY
- $185,000 to $300,000 USD / year
Added 1 hour ago
TT
Enterprise Account Executive
True Talent Solutions Partners
San Francisco, CA
- $155,000 to $165,000 USD / year

True Talent Solutions Partners

Research Crawling Engineer

Remote, United States

$160,000 to $250,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $160,000 to $250,000 USD / year
Experience: At least 2 years of experience building web crawlers or large-scale data pipelines; typically 4 or more years, or 2 or more years with a PhD.
Posting language: English
Posted: Oct 27, 2026

Original job description

ABOUT THE ROLE

Join a data infrastructure team building large-scale web crawling systems and datasets for AI research and model development. This role focuses on reliable data acquisition across diverse, changing web environments.

WHAT YOU'LL DO

Build and maintain web crawlers across social, travel, and multilingual websites.
Design high-throughput, fault-tolerant systems that collect millions to billions of URLs per day.
Adapt crawling approaches to anti-bot systems, rate limits, and dynamic or JavaScript-heavy sites.
Develop pipelines to clean, deduplicate, filter, and normalize web data at TB-to-PB scale.
Build and maintain datasets for research and model training in collaboration with research teams.
Monitor crawl performance, coverage, and data quality, and adjust systems as websites change.
Optimize crawling infrastructure for cost, latency, and reliability across cloud or bare-metal environments.

WHAT WE'RE LOOKING FOR

At least 2 years of experience building web crawlers or large-scale data pipelines; typically 4 or more years, or 2 or more years with a PhD.
Hands-on experience building and operating web crawlers at scale, including data pipelines handling TB-scale workloads.
Strong proficiency in Go, Rust, Python, Java, or C++, with experience in distributed systems or parallel processing.
Deep knowledge of HTTP, networking, and browser behavior, including experience working with anti-bot systems, rate limits, and JavaScript-heavy sites.
Experience with headless browsers, proxy systems, IP rotation, and request orchestration is valuable.
Familiarity with data quality evaluation, large-scale data processing, and cloud or bare-metal infrastructure.
Experience with NLP pipelines, machine learning dataset curation, or AI research data is a plus.
A degree in computer science, engineering, or a related technical field is helpful; a PhD is a plus.

COMPENSATION & BENEFITS

Salary range: $160,000 to $250,000 annually. Visa sponsorship is not available.

LOCATION

Remote.

Apply About employer

More about this job

Skills

Research
Team building
JavaScript
Collaboration
Go
Rust
Python
Oracle Java
C++
Process data
Natural language processing
Machine learning
Computer science

Report this job