Senior Efficacy Research and Psychometrics Analyst (24 months fixed-term) - Khan Academy Kids

Khan Academy Kids is seeking a Senior Efficacy Research and Psychometrics Analyst (fixed-term for 24 months) to enhance the evidence base for assessments and learning content for children in PreK through 2nd grade.

The role spans two key areas:

Core Responsibilities:

• Psychometrics & Efficacy Research:
• Lead technical aspects of psychometric research, including managing external partnerships, data extraction, and analysis of findings like item difficulty, score distributions, growth analyses, concurrent validity, and IRT-based scoring.
• Collaborate with product and engineering teams to translate research into product improvements, such as updates to scoring models and assessment discontinuation rules.
• Recruit and manage research partner sites, execute MOUs, and coordinate data collection.
• Partner with legal teams to draft data-sharing agreements for pseudonymized and identified data.
• Coordinate participation in ongoing efficacy studies and design future studies.
• Analytics Reporting & Data Infrastructure:
• Build and maintain the KPI dashboard for Khan Academy Kids, integrating with App Store Connect and Google Play Console.
• Prepare recurring app performance reports, including monthly/quarterly reviews and ad-hoc data requests.

Qualifications:

• Advanced degree (PhD or Master’s) in psychometrics, educational measurement, or related fields.
• Strong expertise in psychometric methods, including item analysis, reliability, validity, growth modeling, and Item Response Theory (IRT).
• Proficiency in SQL and statistical programming (R or Python).
• Experience with business intelligence tools like Looker and dashboard/data pipeline development.
• Familiarity with student data privacy (e.g., FERPA, COPPA) and work with pseudonymized/deidentified data.
• Excellent communication skills to translate complex findings for product teams, funders, and partners.
• Strong project management and cross-cultural competency.

Nice to have: Experience in early childhood education or assessment of young learners.