Technical Certification Lead
OpenAI
Summary
As Technical Certification Lead, you will lead the design, development, and ongoing quality of OpenAI's technical role-based certifications. This is a hands-on people manager role on the Certification team, managing Certification Assessment Specialists and turning shared certification frameworks into competency models, job-task analyses, assessment blueprints, scoring rubrics, and passing criteria.
Job description
About the Role
As Technical Certification Lead, you will lead the design, development, and ongoing quality of OpenAI's technical role-based certifications. This is a hands-on people manager role on the Certification team, managing Certification Assessment Specialists and turning shared certification frameworks into competency models, job-task analyses, assessment blueprints, scoring rubrics, and passing criteria.
In this role, you will:
- Build and maintain competency models and facilitate job-task analyses for technical roles.
- Work with practitioners and subject matter experts (SMEs) to identify critical tasks, define required skills and proficiency, and keep models current.
- Define and maintain assessment blueprints and write scoring rubrics.
- Lead standard-setting with subject matter experts and a psychometrician or measurement specialist.
- Validate AI-assisted scoring against judgments from calibrated human reviewers.
- Manage and develop Certification Assessment Specialists across technical certifications.
- Secure and manage SME commitments for role analysis, task design, technical review, pilot feedback, and maintenance.
- Share competency models and permitted blueprint information with learning content developers and review content for alignment.
- Own ongoing assessment quality for each technical certification within shared program standards.
- Provide technical input when candidates challenge an assessment task and partner with operations on fixes and validation.
- Define what credential holders must demonstrate to stay certified and align renewal requirements with shared standards.
- Make evidence-based readiness recommendations for pilots, launches, and updates.
- Coordinate technical handoff and delivery readiness with operations and platform teams.
You might thrive in this role if you:
- Have led the development and maintenance of role-based technical certifications or other high-stakes technical assessments.
- Have directly managed and developed engineers or technical assessment specialists.
- Have experience working at a technology company in a fast-paced environment.
- Have personally built competency models, facilitated job-task analyses, and used them to define assessment blueprints and write scoring rubrics.
- Have designed hands-on assessments for software engineering, AI, cloud, or technical implementation roles.
- Have built or validated AI-assisted scoring for assessments with appropriate human review.
- Have recruited and coordinated subject matter experts across teams.
- Have used evidence from pilots and live assessments to improve tasks and scoring.
- Have worked with learning content developers to map curriculum and practice to competencies.
- Communicate technical and measurement tradeoffs clearly to engineers, learning teams, operations partners, and leaders.
- Can review technical assessment work directly and coach others through design decisions.
- Have maintained multiple technical assessments as products, tools, or job expectations changed.
- Care deeply about accessible assessments, consistent scoring, and making only the claims the evidence supports.
It's a plus if you have experience:
- Designing practical assessments for AI application development, agentic workflows, or customer-facing AI deployments.
- Working with psychometricians on standard setting, rubric calibration, or validation of performance assessments.
- Launching hands-on assessments across live and virtual delivery environments.
Requirements
- Led development and maintenance of role-based technical certifications or other high-stakes technical assessments
- Directly managed and developed engineers or technical assessment specialists
- Experience working at a technology company in a fast-paced environment
- Built competency models and facilitated job-task analyses
- Defined assessment blueprints and wrote scoring rubrics
- Designed hands-on assessments for software engineering, AI, cloud, or technical implementation roles
- Built or validated AI-assisted scoring for assessments with human review
- Recruited and coordinated subject matter experts across teams
- Used evidence from pilots and live assessments to improve tasks and scoring
- Worked with learning content developers to map curriculum to competencies
- Communicate technical and measurement tradeoffs clearly
- Review technical assessment work directly and coach others
- Maintained multiple technical assessments as products or job expectations changed
- Preferred: Designing practical assessments for AI application development, agentic workflows, or customer-facing AI deployments
- Preferred: Working with psychometricians on standard setting, rubric calibration, or validation
- Preferred: Launching hands-on assessments across live and virtual delivery environments
Benefits
- Medical, Dental & Vision
- Health Savings Accounts
- Health FSA
- Dependent Care FSA
- Commuter Expenses
- 401(k) Matching
- Paid Parental Leave
- Paid Medical Leave
- Paid Caregiver Leave
- Flexible PTO
- Paid Company Holidays
- Paid Sick Time
- Mental Health Support
- Life Insurance
- Disability Coverage
- Learning & Development Stipend
- Daily Meals
- Relocation Support
- Charitable Donation Matching
- Wellness Stipends
Company overview
OpenAI is an artificial intelligence research laboratory and company based in San Francisco, California. Founded in 2015 by Elon Musk, Sam Altman, Greg Brockman, Ilya Sutskever, Wojciech Zaremba, and John Schulman, OpenAI initially operated as a non-profit research organization before transitioning to a capped-profit model. The company's mission is to ensure that artificial general intelligence (AGI) benefits all of humanity. OpenAI has developed groundbreaking AI technologies including GPT (Generative Pre-trained Transformer) models, DALL-E for image generation, and Codex for programming assistance. Their most notable product, ChatGPT, has revolutionized human-AI interaction and brought large language models to mainstream audiences. OpenAI conducts cutting-edge research in machine learning, natural language processing, computer vision, and reinforcement learning, while also developing practical AI applications that are transforming industries worldwide.