Validation against human-coded samples

[satra]

Summary

Validate the automated pedagogical, engagement, and mastery metrics against human-coded samples so the platform has defensible evaluation quality.

Scope

• define human-coding rubric for question type, scaffolding, affect support, disengagement, and mastery events
• build or curate a gold-sample validation dataset
• implement comparison scripts and quality reports
• define acceptable agreement thresholds and review loops
• document known weak areas and confidence limitations

Deliverables

• validation rubric
• gold-sample dataset scaffolding
• evaluation scripts and reports
• quality criteria for automated measures

Testing

• evaluation pipeline tests
• fixture-based comparisons against hand-coded examples
• report generation tests

Dependencies

• depends on event pipeline and metric definitions