Job Title: Data Annotation SpecialistJob DescriptionThe Data Annotation Specialist improves the quality of chatbot responses by carefully reviewing AI-generated conversations and applying structured feedback. In this role, you evaluate chatbot interactions against detailed guidelines, label key attributes, and help create a robust feedback loop that drives continuous improvement in conversational AI performance. You work with both voice and text data, ensuring accuracy, safety, and policy compliance across a wide range of use cases.
Responsibilities
- Review chatbot conversations and related artifacts, including individual turns, model responses, retrieved content, citations, and metadata using approved tools.
- Apply approved labeling guidelines, taxonomies, and decision rules, completing all required labels, rationales, confidence assessments, and ambiguity indicators for each conversation.
- Evaluate chatbot performance across multiple dimensions such as intent, task completion, correctness, completeness, relevance, groundedness, source attribution, response quality, safety, policy compliance, escalation need, and failure category.
- Identify content that is ambiguous, sensitive, safety-relevant, outside current guidelines, or requires subject-matter-expert review, and escalate these cases appropriately.
- Follow existing standards and guidance precisely, without inferring or creating new standards where approved guidance does not exist.
- Participate in onboarding, calibration, refresher training, quality reviews, and adjudication sessions to maintain consistent labeling decisions across the team.
- Conduct first-level quality checks on assigned work, identify errors, and correct issues highlighted through internal or external quality reviews.
- Maintain complete and accurate records of completed annotations, quality results, exceptions, escalations, and open issues in designated systems.
- Prepare clear weekly reports summarizing completed volume, productivity, quality results, backlog, open questions, risks, guideline gaps, and required actions.
- Recommend and document potential improvements to annotation guidelines and taxonomy based on recurring ambiguities, labeling patterns, and adjudication outcomes, ensuring changes are implemented only after formal approval.
Essential Skills
- Hands-on experience labeling conversational AI data, including both text and audio.
- Professional fluency in English, with strong reading comprehension and written communication skills.
- Demonstrated ability to interpret and consistently apply detailed guidelines, taxonomies, and decision rules.
- Strong analytical judgment and attention to detail, including the ability to identify factual errors, incomplete responses, unsupported claims, policy concerns, and ambiguous cases.
- Ability to document labeling decisions and provide clear, concise rationales for each annotation.
- Ability to work accurately on repetitive tasks while meeting defined quality, productivity, and turnaround expectations.
- Ability to participate in calibration sessions, incorporate feedback, and adjust decisions as guidelines evolve.
- Ability to prepare clear weekly reports summarizing labeling volume, productivity, quality results, progress, and open issues.
- Research and analysis skills with a focus on data integrity and data annotation.
- Strong analytical and communication skills relevant to data annotation for both voice and text.
Additional Skills & Qualifications
- Prior experience in the automotive industry is preferred.
- Background as a Data Annotator or Data Analyst is beneficial.
- Experience working with data annotation processes for voice and text interactions.
- Comfort working with structured taxonomies and decision frameworks.
- Ability to collaborate effectively with cross-functional teams during training, calibration, and adjudication sessions.
Work EnvironmentThis role follows a standard 9:00 AM to 5:00 PM schedule, Monday through Friday. The position operates in a hybrid environment, with four days onsite and one day working from home each week. You work within approved tools and designated systems to perform annotations, quality checks, and reporting, in a structured setting focused on accuracy, consistency, and continuous improvement. Job Type & LocationThis is a Contract position based out of Dearborn, MI. Pay and BenefitsThe pay range for this position is $38.00 - $38.00/hr. Individual compensation offered for this position within this range will depend on many factors, including qualifications, skills, relevant experience, job knowledge, geographic location, internal equity, and other pertinent job-related factors.Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits
available for this temporary role may include the following:
Medical, dental & vision Critical Illness, Accident, and Hospital 401(k) Retirement Plan Pre-tax and Roth post-tax contributions available Life Insurance (Voluntary Life & AD&D for the employee and dependents) Short and long-term disability Health Spending Account (HSA) Transportation benefits Employee Assistance Program Time Off/Leave (PTO, Vacation or Sick Leave) Job Type & LocationThis is a Contract position based out of Dearborn, MI. Application DeadlineThis position is anticipated to close on Nov 25, 2026. Workplace TypeThis is a hybrid position in Dearborn,MI.
About Aston Carter
Aston Carter provides world-class corporate talent solutions to thousands of clients across the globe. Specialized in accounting, finance, human resources, talent acquisition, procurement, supply chain and select administrative professions, we extend the capabilities of industry-leading companies. We draw on our deep recruiting expertise and expansive network to meet the evolving needs of our clients and talent community with agility and excellence. With offices across the U.S., Canada, Asia Pacific and Europe, Aston Carter serves many of the Fortune 500. We are proud to be a ClearlyRated Best of Staffing® Platinum Award winner for both client and talent service.
The company is an equal opportunity employer and will consider all applications without regard to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.
If you would like to request a reasonable accommodation, such as the modification or adjustment of the job application process or interviewing process due to a disability, please email astoncarteraccommodation@astoncarter.com for other accommodation options.
San Francisco Fair Chance Ordinance: Pursuant to the San Francisco Fair Chance Ordinance, for all positions located in the city and county of San Francisco, we will consider for employment qualified applicants with arrest and conviction records.
Massachusetts Lie Detector: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.
Use of Artificial Intelligence (AI): We may use Artificial Intelligence (AI) to support parts of our hiring process, including sourcing, screening, and evaluating candidates. AI helps assess applications and qualifications, but final decisions are made by our hiring team. By applying, you acknowledge and agree that your application may be reviewed using AI tools.
