About the Team
Customer Education helps customers build the practical skills and confidence to use AI and OpenAI products safely and effectively. We bring together customer insight, learning and enablement expertise, product signals, and measurement to help organizations move from access to durable adoption.
Within Customer Education, the Certification team helps individuals demonstrate competence for in-demand AI skills and helps customers and partners validate that their workforce has the skills they need to meet their AI-enabled business goals. Our work centers on the belief that learners should understand where to go to build skills and capabilities, how to demonstrate them, and what to pursue next. Similarly, Enterprise leaders should understand what each credential signifies and how it supports workforce readiness. We do this with transparency for the overall OpenAI credential framework and providing trusted, role-based certifications that verify practical capability.
About the Role
As a Certification Assessment Specialist, you will design, build, test, launch, and maintain the hands-on assessments behind an OpenAI technical role-based certification. Your work will help candidates demonstrate what they can do in realistic scenarios and give reviewers reliable evidence to score their work.
This is a hands-on individual contributor role reporting to the Technical Certification Lead. You will turn competency models, assessment blueprints and scoring rubrics into working tasks, assessment environments and workflows that capture candidate evidence and produce results reviewers can verify. You will use scripting and automation and partner with engineering on software development and delivery integrations.
You will work with the certification, measurement, engineering and operations teams, as well as technical subject matter experts and external delivery providers. The Technical Certification Lead will coordinate alignment with learning content while keeping confidential assessment details separate from learning materials.
You will take assessments from prototype through pilot, launch, and ongoing maintenance across live and virtual delivery. As products and tools change, you will update tasks, environments, and scoring together so candidates are assessed against the intended skills.
Success means candidates have the opportunity to demonstrate the competencies required for the certification, and reviewers can consistently score that work using evidence they can trace back to the submission.
In this role, you will:
- Translate assessment blueprints into realistic scenarios, candidate instructions, and hands-on tasks. Work with subject matter experts to check that the tasks reflect the job and elicit the appropriate evidence for each rubric.
- Define assessment environment requirements and build or configure the environments candidates will use. Establish repeatable provisioning and reset workflows for accounts, tools, data, and permissions so candidates start from consistent conditions.
- Build and maintain pipelines that capture candidate submissions and other scoring evidence, apply scoring rules, and route results for review. Make scores traceable to the underlying evidence and protect sensitive data under approved controls.
- Build and test model-assisted scoring with qualified human reviewers using established validation criteria. Compare model scores with calibrated expert judgments, investigate disagreements, implement agreed rules for routing cases to human review, and rerun validation tests after material changes to the model, task, or rubric.
- Test assessments end to end, from environment setup to evidence capture and scoring. Use dry runs and pilots to check instructions, timing, accessibility, and failure recovery; fix defects and verify the fixes before release. Test scoring against multiple valid approaches, partial and incorrect submissions, and failures in the assessment environment, so a platform issue isn’t mistaken for a candidate error.
- Translate assessment needs into requirements for delivery integrations. Partner with engineering and delivery providers to test candidate access, submission and results handoffs and troubleshoot issues.
- Prepare assessments for launch with platform and operations partners. Document setup, version dependencies, and troubleshooting so delivery teams can run assessments consistently and respond when something breaks.
- Maintain versioned tasks, environments, scoring configurations, and test fixtures. Use candidate feedback and delivery and scoring data to identify issues, update or retire tasks, and check that changes preserve the intended skills and difficulty.
- Investigate technical issues raised during delivery or candidate review. Reproduce defects where possible, document what was affected, and test fixes with the relevant delivery partners.
You might thrive in this role if you:
- Have personally designed and built performance-based, hands-on assessments, labs or simulations for technical roles, such as software developers or solution architects, and taken them through testing, launch and maintenance. Experience creating question-and-answer tests alone does not meet this requirement.
- Have used scripting and automation to build and troubleshoot technical assessment environments. You can work with technical subject matter experts to translate realistic solutions into assessments; deep AI-engineering expertise or prior experience with a particular language or delivery platform is not required.
- Have experience working at a technology company in a fast-paced environment, setting priorities and delivering work amid changing requirements and evolving business needs.
- Can use a blueprint and rubric to design realistic tasks and specify what evidence distinguishes competent performance from task completion.
- Have built or maintained workflows that capture submissions, process evidence, and produce traceable scores or reviewer outputs.
- Have implemented or validated model-assisted scoring or evaluation against expert human judgments, and can explain when a score should go to human review.
- Have worked directly with external technical experts or delivery providers and can translate requirements into integrations you can test end to end.
- Have used pilot results, candidate feedback, and scoring or delivery data to improve assessment quality.
- Understand how to build for accessibility and protect candidate evidence and confidential assessment content.
- Communicate technical tradeoffs clearly to subject matter experts, measurement specialists, platform engineers, and operations teams.
It’s a plus if you have experience:
- Building hands-on assessments for AI application development, agentic workflows, or customer deployments.
- Working with psychometricians or measurement specialists on rubric calibration, scoring consistency, or validation of performance assessments.
- Launching performance-based certification assessments across both live and remote delivery.
About OpenAI
OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.
We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic.
For additional information, please see OpenAI’s Affirmative Action and Equal Employment Opportunity Policy Statement https://cdn.openai.com/policies/eeo-policy-statement.pdf.
Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.
To notify OpenAI that you believe this job posting is non-compliant, please submit a report through this form https://form.asana.com/?d=57018692298241&k=5MqR40fZd7jlxVUh5J-UeA. No response will be provided to inquiries unrelated to job posting compliance.
We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link https://form.asana.com/?k=bQ7w9h3iexRlicUdWRiwvg&d=57018692298241.
OpenAI Global Applicant Privacy Policy https://cdn.openai.com/policies/global-employee-and-contractor-privacy-policy.pdf
At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.