A lightweight Streamlit application for collecting human annotations on Dutch football social media comments.
The project was developed as part of a Data Engineering proof of concept to improve the validation of automated sentiment analysis models.
Instead of relying solely on machine learning predictions, this tool collects human interpretations of football-related comments, creating a reference dataset that can later be used for model evaluation, benchmarking, or retrieval-based validation (RAG).
- Random assignment of comments to participants
- Anonymous respondents
- Emotion annotation
- Sentiment annotation
- Communication style annotation
- Confidence scoring
- Google Sheets integration
- Ready for Streamlit Cloud deployment
PSV Comments ↓ Random Sampling (Cochran) ↓ Survey Dataset ↓ Streamlit Annotation Platform ↓ Google Sheets ↓ Human-labelled Dataset ↓ Ground Truth ↓ Future RAG Validation
- Positive
- Neutral
- Negative
- Admiration
- Amusement
- Approval
- Support
- Excitement
- Gratitude
- Optimism
- Pride
- Relief
- Anger
- Frustration
- Disapproval
- Disappointment
- Fear
- Remorse
- Sadness
- Confusion
- Curiosity
- Realization
- Surprise
- Unclear
- Literal
- Humour / Meme
- Sarcasm / Irony
- Banter
- Other
- Python
- Streamlit
- Pandas
- Google Sheets API
- Streamlit Community Cloud
- Majority voting
- Inter-annotator agreement
- Quality control
- Active learning
- RAG-based sentiment validation
- Fine-tuning multilingual sentiment models