Content & LLM Evaluation Annotators

Innodata (Nasdaq: INOD), a global data engineering company, specializes in data and Artificial Intelligence (AI). Their mission is to enable responsible AI advancement by providing data, evaluation frameworks, and expertise to build trustworthy AI systems at scale. They offer solutions, platforms, and services for Generative AI and AI builders.

Scope of the Role:

We are seeking detail-oriented Content & LLM Evaluation Annotators to review public social media content and apply complex classification taxonomies consistently across text, images, carousels, and video.

What You’ll Own:

• Review individual social media posts in their entirety, including captions, images, carousels, video, overlaid text, emojis, hashtags, logos, packaging, and spoken content.
• Watch/Review videos according to the prescribed sampling protocol.
• Apply detailed classification taxonomies covering potentially problematic content, including mature themes, body-focused content, suffering and distress, risky behavior, violence, social aggression, and other defined categories.
• Identify signals associated with suspected branded content.
• Complete reviewability checks and use the appropriate rejection or escalation path when content cannot be accurately assessed.
• Apply multi-select labeling rules precisely, including rules governing mutually exclusive or non-combinable labels.
• Distinguish between similar concepts based on the written specification rather than personal interpretation or assumptions.

You’ll Thrive in This Role If You Have:

• A Bachelor’s degree.
• Previous experience in content annotation, content moderation, data labeling, measurement, research labeling, integrity operations, ads review, or ML training-data programs.
• Experience evaluating or analyzing LLM outputs, including model reasoning or failure modes.
• Excellent critical-thinking and judgment skills, particularly when content falls into ambiguous or borderline categories.
• Strong attention to detail across both text and multimedia content.
• Ability to distinguish meaningful contextual signals from superficial indicators—for example, understanding that a visual element may need to be both sufficiently visible and a focus of the content to meet a particular classification.
• Ability to recognize commercial intent without assuming that a tag, logo, mention, or positive statement alone constitutes branded content.
• Comfort working with ambiguous, sensitive, graphic, and distressing material.