Archived listing. The details below describe a past opening.
Description
Amazon's Customer Experience and Business Trends (CXBT) organization is hiring a Senior Data Scientist for its Benchmarking, Economics, Analytics and Measurement (BEAM) team. BEAM's mission is to improve customer experience across every Amazon business by turning fragmented internal and external data into decision-grade intelligence for senior leaders.
This role spans two of BEAM's core products. The Topline View of Retail is an always-on competitive intelligence capability that fuses third-party transaction panels with Amazon's internal signals to answer how Amazon is winning or losing share of wallet, across segments, categories and time. The Intelligence Engine is an LLM-powered analytics assistant that lets teams across Amazon ask business questions in natural language and get trustworthy, data-backed answers. You will not just analyze metrics. You will decide which metrics should exist and build the models that generate, forecast and explain them. And as the Intelligence Engine scales, you will own how we measure its quality and drive its accuracy forward: defining evaluation standards, diagnosing where and why it fails, building the labeled data that improves it, and running the experiments, including fine-tuning, that prove which changes actually work.
We are looking for someone with exceptional business judgment and metric intuition, paired with deep hands-on modeling skills: a strong point of view on what to measure, and the technical ability to measure it at scale, whether the subject is customer spending or the quality of an LLM's answers. You should be equally comfortable pressure-testing a model's assumptions and explaining its "so what" to a VP in two sentences.
Key job responsibilities
- Own the definition, methodology and evolution of competitiveness metrics such as share of wallet and segment penetration, including validation and calibration of third-party panel data against internal sources.
- Design and validate causal inference, forecasting, segmentation and anomaly detection models that explain what drives shifts in customer spending.
- Define the evaluation framework for the Intelligence Engine: representative benchmark question sets, accuracy and groundedness metrics, LLM-as-judge methods validated against human labels, and clear launch bars.
- Run systematic error analysis on LLM outputs to identify failure patterns, and turn those insights into targeted improvements across knowledge base content, prompts and model behavior.
- Curate and maintain high-quality labeled datasets from expert review and user feedback, and use them to design and measure the impact of model improvement experiments, such as supervised fine-tuning and preference-based tuning, in partnership with applied scientists and engineers.
- Build reproducible pipelines that combine panel and internal data, and productionize recurring analyses and evaluations with data engineers.
- Translate ambiguous business questions into defensible metrics and answers, and communicate findings to senior leadership through written narratives and monthly reporting.
Set modeling, evaluation and scientific rigor standards for the team, and mentor scientists and analysts.
About the team
CXBT is a group of diverse functions dedicated to understanding, improving, and influencing customer experience globally, across all of Amazon. We are builders who develop products, services, and data-driven approaches that shape offerings for nearly every Amazon business and customer type - consumers, developers, sellers, brands, employees, investors, streamers, and gamers. We work backwards from customer needs, combine technical and non-technical methods, and track industry and business trends. Our roles span Product Managers, Data Scientists, Economists, Business Intelligence Engineers, Data Engine, Applied Scentist, and Product Manager.
Basic Qualifications
- Master's degree or above in a quantitative field, or PhD and 6+ years of data scientist experience
- Experience applying statistical modeling and machine learning methods (e.g., regression, forecasting/time series, classification, clustering, causal inference) to business problems.
- Proficiency with data scripting and modeling languages (SQL and Python or R) and building analyses on large, messy datasets.
- Demonstrated experience defining metrics and KPIs from scratch and connecting them to business decisions.
- Experience communicating analytical and modeling results to senior, non-technical business audiences in writing.
Preferred Qualifications
- Familiarity with experimentation/A-B testing and with modern ML tooling and cloud data environments (e.g., AWS, Spark).
- Experience mentoring scientists/analysts and providing technical leadership on modeling and metric-design decisions.
- Strong background in retail, e-commerce, or CPG competitive and market-share analytics (share-of-wallet, category penetration, price/selection benchmarking).
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits .
USA, VA, Arlington - 159,200.00 - 215,300.00 USD annually
USA, WA, Seattle - 159,200.00 - 215,300.00 USD annually