Trust & Safety Lead (AI Responsible AI) at MagicSchool

About MagicSchool

MagicSchool is the premier generative AI platform for teachers, serving over 7 million teachers worldwide. Join a fast-growing, mission-driven team working towards real social impact in education.

Try the platform at our website or connect with the community on Wall of Love.

Responsibilities

• Own the safety of AI output for teachers and students, including policy, model-level guardrails, evaluations, red-teaming, and incident response.

• Lead trust posture with school districts, handling safety documentation, RFP responses, and alignment with state AI-in-education guidelines.

• Develop safety evaluations and monitoring as core product infrastructure, ensuring safety acceptance criteria are established during product scoping—not post-launch.

• Specialize in student-facing safety, including age-appropriate content, self-harm/crisis handling, and educator escalation pathways.

• Manage the end-to-end safety incident process, including postmortems and feedback loops for product/engineering improvements.

• Collaborate with Legal on compliance with COPPA, FERPA, state privacy laws, and emerging AI regulations. Balance operational execution with legal positioning.

• Build and scale a small team (1–3 members) to grow the safety function through 2027.

Qualifications & Skills

Must Haves:

• Proven ownership of trust & safety or responsible AI for a consumer or youth-facing product at scale, including real incidents, policy decisions, and regulatory interactions.

• Hands-on expertise in LLM safety, including evals, classifiers, prompt/system-level guardrails, and red-teaming. Ability to interpret eval results and implement fixes.

• Experience making safety a velocity enabler—building guardrails as infrastructure to accelerate shipping with confidence.

• Strong judgment under ambiguity; publicly owns decisions to districts and executive teams.

• Excellent writer for safety documentation, district-facing explainers, and incident communications.

• Operates with hypothesis-driven decision-making and escalates critical risks quickly.

Nice to Haves:

• Experience in K-12 or youth-platforms (e.g., GoGuardian, Securly, Roblox, Discord, YouTube Kids, Epic Games).

• Familiarity with state AI-in-education frameworks and district procurement questionnaires.

• Background in child-safety NGOs or standards bodies (e.g., Common Sense Media, NCMEC, Thorn, ISTE).

• Data/ML fluency to collaborate directly with engineering on classifiers and eval pipelines.

• Comfortable representing safety externally (conferences, press, district boards).

Required Experience

• Proven ownership of trust & safety or responsible AI for a consumer or youth-facing product at scale.

• Hands-on experience with LLM safety tooling, including evals, classifiers, and red-teaming.

• Experience handling live incidents involving minors or vulnerable users.

• Track record of embedding safety as product infrastructure, not a gate.

• Ability to communicate safety tradeoffs clearly to executives and district-level stakeholders.

Tech Stack

• LLM Layer: Multi-provider (OpenAI, Anthropic, Google Vertex AI) with moderation APIs and custom classifiers.

• Eval & Red-Team Tooling: Custom-built (to be developed) with Datadog for observability and alerting.

• Product Stack: TypeScript/Next.js (Vercel), Supabase (PostgreSQL backend).

• Incident & Knowledge Tools: Linear, Slack workflows, Notion.

Why Join MagicSchool?

• Work on cutting-edge AI technology directly impacting educators and students.

• Join a mission-driven team passionate about equitable, efficient education.

• Flexible remote work with a culture built on trust, communication, and collaboration.

Benefits

• Unlimited time off to prioritize work-life balance.

• Choice of employer-paid health insurance, including dental/vision at low premiums.

• Generous stock options (vested over 4 years).

• 401k match and monthly wellness stipend.

Company Values

• Educators are Magic: Trust, empower, and center educators in driving change.

• Joy and Magic: Bring joy and innovation to learning experiences.

• Community: Foster supportive relationships during rapid technological change.

• Innovation: Use AI to modernize education, ensuring equity and access.

• Responsibility: Prioritize safety and ethical AI in education.

• Diversity: Embrace diverse perspectives to serve global educators and students.

• Excellence: Strive for the highest quality in everything we do.