Skip to content

Y3454R/vsp-mvp

BranchesTags

Repository files navigation

VSP MVP - Virtual Simulated Patient Chatbot

VSP Logo License Python Next.js

AI-Powered Virtual Simulated Patients for Mental Health Interview Training

Practice psychiatric interviewing skills with realistic AI-powered virtual patients and receive instant feedback on your performance.

Features • Quick Start • Documentation • API Docs

📋 Table of Contents

🎯 Overview

VSP MVP is a full-stack web application that provides AI-powered virtual simulated patients for training mental health professionals. Students can conduct realistic psychiatric interviews and receive automated evaluations based on 8 core competencies.

Why VSP?

  • 🎓 For Students: Practice psychiatric interviews anytime, anywhere without hiring actors
  • 👨‍⚕️ For Educators: Scale clinical training with consistent, evidence-based evaluations
  • 🏥 For Institutions: Supplement clinical training programs cost-effectively with virtual simulations

Key Differentiators

  • Adapted from peer-reviewed research (CureFun paper, arXiv:2404.13066v2)
  • Specialized for mental health education (not general medicine)
  • Comprehensive evaluation on psychiatric interviewing competencies
  • Real-time conversation with contextual AI responses
  • Complete audit trail with conversation history

✨ Features

🤖 AI-Powered Virtual Patients

  • Natural conversational interactions using GPT-4o-mini
  • Emotionally authentic responses matching mental health conditions
  • Context-aware dialogue with conversation memory
  • Role consistency (patient never acts as clinician)

📊 Automated Evaluation System

8 Core Competencies (0-10 scale each):

  1. Rapport Building & Therapeutic Alliance
  2. Active Listening & Empathy
  3. Psychiatric History Taking
  4. Risk Assessment (suicide/homicide screening)
  5. Biopsychosocial Assessment
  6. Communication Skills
  7. Cultural Sensitivity & Respect
  8. Interview Structure & Completeness

Non-Scoring Metrics (CureFun methodology):

  • Information Density (medical terminology usage)
  • Emotional Tendency (communication warmth)
  • Response Length (average message length)
  • Turn Number (conversation exchanges)

🗂️ Case Management

Built-in Cases (4 mental health conditions):

  • Major Depressive Disorder
  • Generalized Anxiety Disorder
  • Post-Traumatic Stress Disorder (PTSD)
  • Bipolar Disorder

Admin Features:

  • Create custom cases via web UI
  • Edit existing cases
  • Set difficulty levels
  • Define expected questions
  • Custom prompt templates

👥 User Management

3 User Roles:

  • Student: Conduct interviews, receive evaluations
  • Evaluator: View all evaluations and conversations
  • Admin: Full access including case and user management

Features:

  • Secure authentication (NextAuth + JWT)
  • Role-based access control (RBAC)
  • User approval workflow
  • Conversation history per student

📜 Conversation History

  • Permanent storage of all conversations
  • View full transcripts with timestamps
  • Export transcripts as text files
  • Link conversations to evaluations
  • Evaluator access to all student conversations

🛠️ Technology Stack

Backend

  • Framework: FastAPI (Python 3.10+)
  • LLM: OpenAI GPT-4o-mini via LangChain
  • Database: PostgreSQL with SQLAlchemy ORM
  • Migrations: Alembic
  • Authentication: JWT tokens, bcrypt password hashing

Frontend

  • Framework: Next.js 14 (React 18, TypeScript)
  • Styling: Tailwind CSS with dark mode
  • Authentication: NextAuth.js
  • HTTP Client: Axios
  • UI Icons: Lucide React

DevOps

  • Containerization: Docker & Docker Compose
  • Development: Hot reload for both frontend and backend
  • API Docs: Auto-generated Swagger/OpenAPI

🚀 Quick Start

Prerequisites

  • Node.js 18+ and npm
  • Python 3.10+
  • PostgreSQL 14+
  • OpenAI API Key (required)

1. Clone Repository

git clone <repository-url>
cd vsp-mvp

2. Set Up Environment Variables

# Copy example files
cp env.example .env

# Backend environment
cd backend
cp .env.example .env
# Edit backend/.env and add your OpenAI API key

# Frontend environment
cd ../frontend
cp .env.example .env.local
# Edit frontend/.env.local
cd ..

Minimum Required in backend/.env:

LLM_PROVIDER=openai
OPENAI_API_KEY=sk-your-actual-openai-key-here
OPENAI_MODEL=gpt-4o-mini
DATABASE_URL=postgresql://vsp_dummy_user:password@localhost:5432/vsp_dummy
NEXTAUTH_SECRET=your-random-secret-here
NEXTAUTH_URL=http://localhost:3000

Minimum Required in frontend/.env.local:

NEXT_PUBLIC_API_URL=http://localhost:8000
NEXTAUTH_SECRET=your-random-secret-here
NEXTAUTH_URL=http://localhost:3000

3. Set Up Database

# Create PostgreSQL database
createdb vsp_dummy

# Run migrations
cd backend
source venv/bin/activate  # or venv\Scripts\activate on Windows
pip install -r requirements.txt
alembic upgrade head

4. Create Admin User

cd backend
python -m scripts.create_admin

5. Start Application

Option A: Using Docker (Recommended)

docker-compose up --build

Option B: Local Development

# Terminal 1 - Backend
cd backend
source venv/bin/activate
cd ..
python -m uvicorn backend.main:app --reload --port 8000

# Terminal 2 - Frontend
cd frontend
npm install
npm run dev

6. Access Application

📚 Detailed Setup

See detailed guides:

📁 Project Structure

vsp-mvp/
├── backend/                    # FastAPI backend
│   ├── alembic/               # Database migrations
│   ├── ai/                    # LangChain AI layer
│   │   ├── chains/            # LangChain chains
│   │   │   ├── patient_chain.py  # Patient simulation
│   │   │   └── evaluation_chain.py # Evaluation chain
│   │   ├── memory/            # Conversation memory
│   │   └── prompts/           # Prompt templates
│   ├── core/                  # Core configuration
│   │   ├── config.py         # Settings (LLM, DB, etc.)
│   │   ├── auth.py           # Authentication logic
│   │   └── llm_client.py     # LLM client initialization
│   ├── models/                # SQLAlchemy models
│   │   ├── user.py           # User model
│   │   ├── case_db.py        # Case model
│   │   ├── evaluation.py     # Evaluation model
│   │   └── chat_history_db.py # Chat history model
│   ├── routers/               # API endpoints
│   │   ├── auth.py           # Authentication endpoints
│   │   ├── cases.py          # Case management
│   │   ├── chat.py           # Chat & conversation endpoints
│   │   └── evaluate.py       # Evaluation endpoints
│   ├── services/              # Business logic
│   │   ├── chat_service.py   # Chat orchestration
│   │   ├── evaluation_service.py # Evaluation logic
│   │   └── metrics_service.py    # Non-scoring metrics
│   ├── data/cases/            # Case definitions (JSON)
│   └── main.py                # FastAPI app entry point
├── frontend/                  # Next.js frontend
│   ├── app/                   # App router pages
│   │   ├── page.tsx          # Homepage
│   │   ├── chat/             # Interview page
│   │   ├── results/          # Evaluation results
│   │   ├── history/          # Student history
│   │   ├── conversation/     # Transcript viewer
│   │   ├── auth/             # Sign in/up pages
│   │   └── admin/            # Admin pages
│   ├── lib/                   # Utilities
│   │   ├── api.ts            # API client
│   │   └── errorMessages.ts  # Error handling
│   └── providers/             # React providers
├── docker-compose.yml         # Docker configuration
└── README.md                  # This file

🔌 API Documentation

Public Endpoints (No Auth Required)

Get Case Summary

GET /api/cases/summary

Returns aggregated case information without sensitive details.

Response:

{
  "total_cases": 4,
  "difficulty_counts": {
    "easy": 1,
    "medium": 2,
    "hard": 1
  },
  "categories": ["Depression", "Anxiety", "PTSD", "Bipolar Disorder"]
}

Protected Endpoints (Authentication Required)

Authentication

  • POST /api/auth/register - User registration
  • POST /api/auth/login - User login
  • GET /api/auth/me - Current user info

Cases

  • GET /api/cases/ - List all cases (full details)
  • GET /api/cases/{case_id} - Get specific case
  • POST /api/cases/ - Create case (admin only)
  • PUT /api/cases/{case_id} - Update case (admin only)
  • DELETE /api/cases/{case_id} - Delete case (admin only)

Chat

  • POST /api/chat/ - Send message to patient
  • POST /api/chat/end-session - End conversation
  • GET /api/chat/conversations - Get student's conversations
  • GET /api/chat/conversations/{chat_id} - Get full transcript
  • GET /api/chat/admin/conversations - Get all conversations (evaluator)

Evaluation

  • POST /api/evaluate/ - Evaluate conversation
  • GET /api/evaluate/ - List evaluations (evaluator)
  • GET /api/evaluate/{evaluation_id} - Get specific evaluation

Users (Admin Only)

  • GET /api/users/ - List all users
  • PUT /api/users/{user_id}/approve - Approve user
  • PUT /api/users/{user_id}/role - Update user role

Full API Documentation: http://localhost:8000/docs (when running)

📖 Usage Guide

For Students

  1. Sign Up: Create an account at /auth/signup
  2. Browse Cases: View available cases on homepage
  3. Start Interview: Click "Start Interview" on any case
  4. Conduct Interview: Ask questions naturally, as you would in a real clinical setting
  5. End Interview: Click "End Interview" when finished
  6. Review Results: View your scores, strengths, and areas for improvement
  7. View History: Access past conversations at /history
  8. Review Transcripts: Click on any conversation to see full transcript

For Evaluators

  1. View All Conversations: Access /admin/conversations
  2. Filter by Case: Use filters to find specific conversations
  3. Review Transcripts: View full conversation details
  4. Access Evaluations: Link directly to evaluation results
  5. Monitor Progress: Track student performance over time

For Admins

  1. Manage Users: Approve new users, assign roles at /admin/users
  2. Create Cases: Add new patient cases at /cases/create
  3. Edit Cases: Update existing cases
  4. Monitor System: Access all evaluator features plus case management

🔬 Research Background

This project implements the methodology from the CureFun paper:

"Leveraging Large Language Model as Simulated Patients for Clinical Education"
Li, Y., Zeng, C., Zhong, J., Zhang, R., Zhang, M., & Zou, L. (2024)
arXiv preprint arXiv:2404.13066v2

Key Adaptations

  • Domain: Physical medicine → Mental health
  • Focus: Medical history → Psychiatric interviewing
  • Risk Assessment: Added suicide/homicide screening
  • Evaluation: Adapted criteria for mental health context

See PAPER_ALIGNMENT.md for detailed methodology alignment.

💰 Cost Optimization

Recommended Configuration

The system defaults to GPT-4o-mini for optimal cost-effectiveness:

Model Cost per Interview Monthly Cost (5000 interviews)
GPT-4o-mini ✅ $0.018 $90
GPT-3.5-turbo $0.024 $120
GPT-4 $0.36 $1,800

Savings: 95% cost reduction vs GPT-4 with comparable quality

Configuration

Set in backend/.env:

OPENAI_MODEL=gpt-4o-mini  # Default in code

For hybrid approach (GPT-4o-mini for chat, GPT-4 for evaluation), modify service classes to use different models.

🏗️ Architecture

graph LR
    Student[Student Browser] --> Frontend[Next.js Frontend]
    Frontend --> Backend[FastAPI Backend]
    Backend --> LLM[OpenAI GPT-4o-mini]
    Backend --> DB[(PostgreSQL)]
    Backend --> Memory[LangChain Memory]

    LLM --> PatientSim[Patient Simulation]
    LLM --> Evaluation[Evaluation]
Loading

Data Flow

  1. Student selects a case and starts interview
  2. Frontend sends message to backend API
  3. Backend retrieves conversation context from memory
  4. LangChain constructs prompt with case data + history
  5. OpenAI generates patient response
  6. Backend saves message to database and memory
  7. Frontend displays response to student
  8. On interview end: Evaluation runs and scores performance

📦 Database Schema

Tables

users

  • User accounts with roles (admin, evaluator, student)
  • Authentication and authorization data

cases

  • Patient case definitions
  • Can supplement JSON case files

evaluations

  • Student evaluation results
  • Linked to students and sessions

chat_history

  • Full conversation transcripts
  • Permanent storage with timestamps
  • Linked to students and cases

Migrations

# Check current version
cd backend
alembic current

# Apply all migrations
alembic upgrade head

# Rollback one migration
alembic downgrade -1

🎨 User Interface

Student Experience

  1. Homepage (Unauthenticated):

    • View case count and categories (no sensitive details)
    • Sign in to access full cases

  2. Homepage (Authenticated):

    • Browse all available cases with full details
    • Filter by difficulty level
    • Start interviews immediately

  3. Chat Interface:

    • Real-time conversation with virtual patient
    • Message history with auto-scroll
    • Visual loading indicators
    • End interview button

  4. Results Page:

    • Detailed score breakdown with visualizations
    • Strengths and areas for improvement
    • Narrative feedback from AI evaluator
    • Conversation analytics

  5. History Page:

    • List of all past conversations
    • Message counts and duration
    • Links to transcripts and evaluations

  6. Transcript Viewer:

    • Full conversation replay
    • Color-coded messages
    • Export functionality
    • Timestamp display

Admin/Evaluator Experience

  • User Management: Approve users, assign roles
  • Case Management: Create, edit, delete cases
  • Conversation Dashboard: View all student conversations
  • Evaluation Access: Review all assessments
  • Filtering & Search: Find specific conversations or students

🔐 Security

Implemented

  • ✅ Password hashing with bcrypt
  • ✅ JWT token authentication
  • ✅ Role-based access control (RBAC)
  • ✅ CORS configuration
  • ✅ SQL injection prevention (SQLAlchemy ORM)
  • ✅ Input validation (Pydantic models)
  • ✅ Authorization checks on all protected endpoints

Production Checklist

  • Enable HTTPS
  • Add rate limiting
  • Implement session expiration
  • Set up audit logging
  • Configure CSP headers
  • Add input sanitization for case creation
  • Implement GDPR compliance (data retention)

🧪 Testing

Manual Testing

  1. Unauthenticated User:

    • Visit homepage → should see case summary (not full details)
    • Try to access /chat → should redirect to sign-in

  2. Authenticated Student:

    • Browse full case list
    • Start interview and send messages
    • End interview and receive evaluation
    • View history and transcripts

  3. Admin/Evaluator:

    • Access user management
    • View all conversations
    • Create new cases

Test Accounts

Create test accounts using the admin script:

cd backend
python -m scripts.create_admin --email student@test.com --username student --password test123

🐛 Troubleshooting

"Failed to connect" Error

Cause: Backend not running or wrong API URL

Solution:

# Check backend is running on port 8000
curl http://localhost:8000/health

# Verify NEXT_PUBLIC_API_URL in frontend/.env.local

"Your session has expired"

Cause: NextAuth token expired or invalid

Solution:

  1. Sign out and sign in again
  2. Verify NEXTAUTH_SECRET matches in both backend/.env and frontend/.env.local
  3. Clear browser cookies for localhost

"Unable to connect. Please check your internet connection"

Cause: Network error or API unreachable

Solution:

  1. Check backend is running: curl http://localhost:8000/health
  2. Check CORS settings in backend/core/config.py
  3. Verify firewall not blocking localhost

Migration Errors

# Reset database (⚠️ WARNING: Deletes all data)
cd backend
alembic downgrade base
alembic upgrade head

LLM API Errors

Gemini quota exceeded:

  • Set LLM_PROVIDER=openai in backend/.env
  • Restart backend server

OpenAI authentication failed:

  • Verify API key is valid
  • Check key has not expired
  • Ensure API key has sufficient credits

📊 Performance

Typical Response Times

  • Homepage load: < 1 second
  • Case list (authenticated): < 500ms
  • Chat message: 2-4 seconds (LLM generation)
  • Evaluation: 5-8 seconds (LLM analysis)
  • Transcript view: < 1 second

Scalability Considerations

  • In-memory session storage (consider Redis for production)
  • Database indexes on session_id, student_id, case_id
  • Pagination recommended for >100 conversations
  • Connection pooling configured (SQLAlchemy)

🤝 Contributing

Development Workflow

  1. Create a feature branch
  2. Make changes with clear commit messages
  3. Test thoroughly (authentication, roles, error cases)
  4. Submit pull request with description

Code Style

  • Python: Follow PEP 8, use type hints
  • TypeScript: Use ESLint configuration
  • React: Functional components with hooks
  • CSS: Use Tailwind utility classes

Adding New Features

New Case:

  1. Create JSON file in backend/data/cases/
  2. Or use admin UI to create via database

New Evaluation Criterion:

  1. Update evaluation_checklist.json
  2. Update evaluation prompt
  3. Update frontend evaluation display

New API Endpoint:

  1. Add to appropriate router in backend/routers/
  2. Add service method if needed
  3. Update frontend lib/api.ts
  4. Add TypeScript interfaces

🎓 Educational Use Cases

Medical Schools

  • Pre-clinical psychiatric training
  • OSCE (Objective Structured Clinical Examination) preparation
  • Therapeutic communication skills
  • Risk assessment practice

Psychiatric Residency

  • Interview technique refinement
  • Cultural competency training
  • Crisis assessment skills
  • Diagnostic interviewing

Nursing Programs

  • Mental health assessment
  • Patient communication
  • Empathy development
  • Documentation skills

Continuing Education

  • Skills refresh for practitioners
  • New assessment technique training
  • Exposure to rare conditions
  • Telehealth interview practice

🚧 Roadmap

Phase 1: Core Improvements ✅

  • Switch to GPT-4o-mini (cost optimization)
  • Add conversation history database
  • Build transcript viewer UI
  • Authentication guards on all pages
  • User-friendly error messages

Phase 2: Enhanced Features

  • Student progress dashboard
  • Evaluator analytics dashboard
  • Pagination for conversation lists
  • Advanced filtering and search
  • Email notifications

Phase 3: Advanced Capabilities

  • Voice interface (speech-to-text, text-to-speech)
  • Multi-language support
  • Mobile app
  • LMS integration (Canvas, Moodle)
  • PDF export for evaluations

Phase 4: Research Features

  • Graph database for case knowledge (Neo4j)
  • Full ERRG pipeline implementation
  • Ensemble evaluation (multiple AI judges)
  • Longitudinal progress tracking
  • Predictive analytics

📄 License

MIT License - See LICENSE file for details

🙏 Acknowledgments

  • CureFun Paper: Li et al., 2024 - Research methodology foundation
  • LangChain: Framework for LLM chains and memory
  • OpenAI: GPT-4o-mini for cost-effective AI conversations
  • FastAPI: Modern Python web framework
  • Next.js: React framework for production-ready apps

📞 Support

For issues, questions, or contributions:

⚡ Quick Commands Reference

# Backend
cd backend
source venv/bin/activate
uvicorn backend.main:app --reload --port 8000
alembic upgrade head
python -m scripts.create_admin

# Frontend
cd frontend
npm install
npm run dev
npm run build

# Docker
docker-compose up --build
docker-compose down

# Database
psql vsp_dummy
alembic current
alembic upgrade head

Built with ❤️ for mental health education

⬆ Back to Top

About

Virtual stimulated mental patient for education

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages