Data Scientist Evaluator for AI Research Projects

We are building a network of experienced data scientists for potential future projects with leading AI research organizations. These projects may focus on evaluating how effectively AI systems perform real-world data science work.

There is no immediate project opening, but qualified applicants may be contacted as relevant opportunities become available.

Requirements

Potential Responsibilities

Future projects may involve:

• Designing precise, task-specific grading criteria for data science deliverables, including exploratory data analyses, statistical modeling work, machine learning pipelines, experimentation and A/B test write-ups, feature engineering, and technical reports or notebooks
• Evaluating AI-generated or human-created work against established criteria
• Providing detailed written justifications for evaluations and scores
• Applying consistent, evidence-based judgment so that assessments are reproducible and defensible
• Incorporating structured feedback from senior reviewers and iterating on submitted work

Specific responsibilities will vary depending on the project.

Ideal Qualifications

• 1+ years of professional data science experience
• Experience at a leading technology, research, or quantitative firm (such as top FAANG, AI labs, top-tier quant funds, or equivalent)
• Strong command of Python, SQL, statistical modeling, machine learning, experimentation and causal inference, and translating messy real-world data into rigorous analyses
• Exceptional written communication skills, including the ability to convey technical findings clearly
• A detail-oriented and consistent approach to evaluating complex work
• Comfort receiving feedback and calibrating judgment against established standards