Misconception Dialogue Simulator

A three-party dialogue simulator for teacher training where teachers practice pedagogical questioning with AI-powered student chatbots exhibiting misconceptions, while tutor chatbots provide real-time feedback and intervention.

🎯 Project Status

Implementation Progress: 99/112 tasks (88.4%) | Phase 8: 8/13 tasks (62%)

Completed Features

✅ Phase 1-2: Project setup and infrastructure (Complete)
✅ Phase 3: MVP Dialogue System (Complete)
- Teacher authentication with session management
- Scenario selection and dialogue interface
- AI student bot with misconception role-play (GPT-5/GPT-4-turbo)
- AI tutor bot with intervention logic (GPT-5/GPT-4-turbo)
- Three-party real-time conversation flow
✅ Phase 4: Session Analysis (Complete)
- Post-dialogue question classification
- Frequency distribution analysis
- CSV export with anonymization
✅ Phase 5: Admin Scenario Management (Complete)
- CRUD operations for dialogue scenarios
- Role-based access control (admin)
✅ Phase 6: Framework Configuration (Complete)
- Custom analysis framework creation
- Dynamic label configuration (2-20 labels)
✅ Phase 7: Admin Session Logs (Complete)
- Session filtering and search
- Bulk CSV export
- Aggregated statistics dashboard
✅ Phase 3 (Advanced): Dynamic Configuration & Analytics (Complete)
- API usage tracking with cost analysis
- Prompt template management with versioning
- Dynamic prompt loading with caching
🚧 Phase 8: Production Polish (62% complete)
- ✅ Error handling with exponential backoff
- ✅ Rate limiting (slowapi)
- ✅ SQLite WAL mode
- ✅ Structured JSON logging
- ✅ CORS and security headers
- ✅ Health and metrics endpoints
- ✅ Code review and refactoring
- ⏳ Documentation updates (in progress)
- ⏳ Pre-commit hooks
- ⏳ Performance optimization
- ⏳ Security hardening

See specs/001-misconception-dialogue-sim/STATUS.md for detailed status.

🚀 Quick Start

Prerequisites

Python 3.11 or higher
OpenAI API key with GPT-5/GPT-4 access (Responses API support)
2GB RAM minimum, 4GB recommended

Installation

# 1. Clone repository
git clone https://github.com/your-org/misconcept_platform.git
cd misconcept_platform

# 2. Create virtual environment with uv
uv venv
source .venv/bin/activate  # Linux/Mac
# .venv\Scripts\activate   # Windows

# 3. Install dependencies
uv pip install -e ".[dev]"

# 4. Configure environment
cp .env.example .env

# Edit .env and configure:
# - OPENAI_API_KEY=sk-your-key-here
# - SESSION_SECRET=your-secure-secret-key
# - Other settings as needed

# 5. Initialize database
python -m src.db.seed

# This creates:
# - Default analysis framework ("High/Low Leverage")
# - Admin user (student_uid: admin_001, nickname: 관리자)
# - Sample scenario

# 6. Run development server
uvicorn src.main:app --reload --host 0.0.0.0 --port 8000

Visit http://localhost:8000

First Login

Login URL: http://localhost:8000/login
Teacher Login: Use any student_uid + nickname (creates new user)
Admin Login: student_uid=admin_001, nickname=관리자

📚 Core Features

🎓 Teacher Experience

Login: Simple authentication with student ID and nickname
Scenario Selection: Browse and select active dialogue scenarios
Three-Party Dialogue:
- Teacher asks questions to student bot
- Student bot responds maintaining misconception
- Tutor bot provides pedagogical feedback
- Real-time interventions when needed
Session Analysis:
- Question classification by analysis framework
- Frequency distribution visualization
- CSV export for research

👨‍💼 Admin Features

Dashboard: Aggregate statistics and session overview
Scenario Management:
- Create, edit, and activate/deactivate scenarios
- Configure misconception profiles
- Associate with analysis frameworks
Framework Configuration:
- Define custom question classification systems
- Configure 2-20 labels per framework
- Apply to scenarios
Session Logs:
- Filter sessions by date range and teacher
- View detailed session transcripts
- Bulk CSV export for research
- Statistics dashboard
API Usage Analytics (Phase 3):
- Real-time API usage tracking with token counts
- Cost analysis by model, scenario, and bot type
- Date range filtering and CSV export
- Accurate cost calculation for gpt-4o, gpt-4o-mini, etc.
Prompt Template Management (Phase 3):
- Create and manage system prompts via web UI
- Version control with automatic versioning
- Dynamic prompt loading with 5-minute TTL caching
- Fallback mechanism: Cache → DB → File System → Hardcoded

⚙️ Chatbot Configuration

All chatbot parameters are managed via environment variables (.env file). Server restart required after configuration changes.

All services use OpenAI Responses API (Phase 1 & 1.5 complete)

Model Configuration

CHAT_MODEL - StudentBot/TutorBot LLM model (default: gpt-5-mini)
- Primary Models (권장): gpt-5, gpt-5.1, gpt-5.1-chat-latest
- Fallback Models (지원): gpt-4-turbo
- NOT supported: gpt-3.5 (Responses API limitation)
ANALYSIS_MODEL - Analyzer/Synthesizer LLM model (default: gpt-5.2)
- Same model support as CHAT_MODEL
DIALOGUE_ANALYSIS_MODEL - Dialogue similarity analysis model (default: gpt-5.2)

GPT-5 Reasoning Configuration

ANALYSIS_REASONING - Legacy misconception-analysis reasoning (default: high)
ANALYSIS_CLASSIFICATION_REASONING - Teacher question classification reasoning (default: low)
ANALYSIS_GREETING_REASONING - Greeting detection reasoning (default: low)
ANALYSIS_SYNTHESIS_REASONING - Session feedback synthesis reasoning (default: high)
STUDENT_REASONING - StudentBot reasoning (default: medium)
TUTOR_REASONING - TutorBot reasoning (default: low)
- Valid values: none, minimal, low, medium, high

Output Length Configuration

STUDENT_MAX_TOKENS - StudentBot response budget (default: 1500)
TUTOR_MAX_TOKENS - TutorBot feedback budget (default: 1500)
ANALYSIS_CLASSIFICATION_MAX_TOKENS - Classification first-attempt budget (default: 2500)
ANALYSIS_CLASSIFICATION_RETRY_MAX_TOKENS - Classification retry budget after max_output_tokens exhaustion (default: 4000)
ANALYSIS_GREETING_MAX_TOKENS - Greeting detection first-attempt budget (default: 1000)
ANALYSIS_GREETING_RETRY_MAX_TOKENS - Greeting detection retry budget after max_output_tokens exhaustion (default: 1500)

TutorBot Intervention

TUTOR_INTERVENTION_THRESHOLD - Intervention frequency per 10 questions, 1-10 (default: 3)

Configuration Update Process

Edit .env file with desired parameter values
Restart application: systemctl restart misconcept_platform
Verify changes through health endpoint

Note:

Temperature is NOT configurable (fixed at 1.0 for Responses API)
Scenario-specific model overrides available in admin panel

🏗️ Technology Stack

Backend

Framework: FastAPI (async web framework)
Database: SQLite3 with SQLAlchemy 2.x (async ORM)
LLM: OpenAI Responses API (GPT-5, GPT-5.1, GPT-4-turbo)
Authentication: Session-based with secure cookies
Rate Limiting: slowapi (IP-based)
Logging: python-json-logger (structured logs)

Frontend

Templates: Jinja2 (server-side rendering)
Interactivity: HTMX (partial updates, polling)
Styling: CSS (lightweight, responsive)

Production

Server: uvicorn with multiple workers
Reverse Proxy: nginx
SSL: Let's Encrypt
Monitoring: Health and metrics endpoints

📁 Project Structure

misconcept_platform/
├── src/
│   ├── models/              # SQLAlchemy ORM models
│   │   ├── user.py
│   │   ├── scenario.py
│   │   ├── session.py
│   │   ├── message.py
│   │   ├── analysis_framework.py
│   │   ├── question_analysis.py
│   │   ├── session_summary.py
│   │   └── prompt_template.py     # Phase 3: Prompt versioning
│   ├── services/            # Business logic
│   │   ├── student_bot.py   # AI student with misconception
│   │   ├── tutor_bot.py     # AI tutor with interventions
│   │   ├── analyzer.py      # Question classification
│   │   ├── session_mgr.py   # Dialogue orchestration
│   │   ├── export.py        # CSV export with anonymization
│   │   └── prompt_manager.py # Phase 3: Dynamic prompt loading
│   ├── api/
│   │   ├── routes/          # FastAPI endpoints
│   │   │   ├── auth.py      # Login/logout
│   │   │   ├── scenarios.py # Scenario selection
│   │   │   ├── sessions.py  # Dialogue and analysis
│   │   │   ├── admin.py     # Admin dashboard
│   │   │   ├── admin_scenarios.py    # Scenario CRUD
│   │   │   ├── admin_frameworks.py   # Framework CRUD
│   │   │   ├── admin_sessions.py     # Session logs
│   │   │   └── health.py    # Health/metrics
│   │   ├── dependencies.py  # Dependency injection
│   │   └── schemas.py       # Pydantic models
│   ├── templates/           # Jinja2 HTML templates
│   │   ├── login.html
│   │   ├── scenarios.html
│   │   ├── chat.html
│   │   ├── analysis.html
│   │   └── admin/           # Admin UI
│   ├── prompts/             # LLM system prompts
│   │   ├── student_system.txt
│   │   ├── tutor_system.txt
│   │   └── analysis_prompt.txt
│   ├── db/                  # Database utilities
│   │   ├── connection.py    # Async engine & session
│   │   └── seed.py          # Initial data
│   ├── config.py            # Environment configuration
│   └── main.py              # FastAPI application
├── tests/
│   ├── contract/            # API contract tests
│   ├── integration/         # End-to-end tests
│   └── unit/                # Service/model tests
├── static/                  # Static assets
│   ├── css/
│   └── js/
├── docs/                    # Documentation
│   ├── deployment.md        # Production deployment guide
│   ├── security.md          # Security hardening guide
│   └── spec.md              # Feature specification
├── specs/                   # Implementation specs
│   └── 001-misconception-dialogue-sim/
│       ├── STATUS.md        # Current status
│       ├── spec.md          # Feature spec
│       ├── plan.md          # Implementation plan
│       └── tasks.md         # Task breakdown
└── pyproject.toml           # Project metadata

🧪 Testing

Run All Tests

pytest

Run Specific Test Suites

# Contract tests (API endpoints)
pytest tests/contract/

# Integration tests (end-to-end flows)
pytest tests/integration/

# Unit tests (services and models)
pytest tests/unit/

Run with Coverage

pytest --cov=src --cov-report=html
# View report: open htmlcov/index.html

Known Test Issues

Admin endpoint tests have isolation issues when run together
Individual test classes pass successfully

Run test classes individually for CI/CD:

pytest tests/contract/test_admin_endpoints.py::TestAdminDashboard
pytest tests/contract/test_admin_endpoints.py::TestScenarioCreation
# etc.

🌐 API Endpoints

Public Endpoints

GET / - Home page (redirects to /login)
GET /login - Login page
POST /login - Authenticate user
POST /logout - End session
GET /health - Health check
GET /metrics - Application metrics

Teacher Endpoints (requires authentication)

GET /scenarios - List active scenarios
GET /scenarios/{id} - Start dialogue session
POST /sessions - Create new session
POST /sessions/{id}/messages - Send message
GET /sessions/{id}/messages/updates - Poll for new messages (HTMX)
- Query param: since (optional, last message ID)
- Returns: 200 OK with HTML partial (new messages) or 204 No Content
POST /sessions/{id}/end - End session
GET /sessions/{id}/analysis - View analysis
GET /sessions/{id}/export.csv - Download CSV

Admin Endpoints (requires admin role)

GET /admin - Admin dashboard
GET /admin/scenarios - Manage scenarios
POST /admin/scenarios - Create scenario
PUT /admin/scenarios/{id} - Update scenario
GET /admin/frameworks - List frameworks
POST /admin/frameworks - Create framework
GET /admin/sessions - Session logs
GET /admin/sessions/export - Bulk CSV export
GET /admin/stats - Statistics
GET /admin/api-usage-page - API usage analytics (Phase 3)
GET /admin/prompts-page - Prompt template management (Phase 3)
POST /admin/prompts - Create prompt template (Phase 3)
PUT /admin/prompts/{id}/activate - Activate prompt (Phase 3)

🚀 Production Deployment

See docs/deployment.md for detailed production deployment guide.

Quick Production Setup

# 1. Configure systemd service
sudo cp deployment/misconcept.service /etc/systemd/system/
sudo systemctl enable misconcept
sudo systemctl start misconcept

# 2. Configure nginx
sudo cp deployment/nginx.conf /etc/nginx/sites-available/misconcept
sudo ln -s /etc/nginx/sites-available/misconcept \
    /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl restart nginx

# 3. Set up SSL
sudo certbot --nginx -d your-domain.com

# 4. Configure backups
sudo cp deployment/backup.sh /opt/misconcept_platform/scripts/
sudo chmod +x /opt/misconcept_platform/scripts/backup.sh
sudo crontab -e  # Add: 0 2 * * * /opt/.../backup.sh

🔧 Development

Code Style

Formatter: Black (line length: 80)
Linter: Ruff (E, F, I, N, W rules)
Type Hints: Required for public APIs
File Length: Maximum 300 lines (per project constitution)
Line Length: Maximum 80 characters (per project constitution)

Format and Lint

# Format code
black .

# Lint code
ruff check .

# Auto-fix issues
ruff check --fix .

Database Migrations

# Current approach: Manual schema updates
# TODO: Add Alembic for automated migrations

Adding New Features

Create feature spec in specs/
Write tests first (TDD workflow)
Implement feature
Run tests and ensure they pass
Update documentation

📝 Documentation

Feature Spec: specs/001-misconception-dialogue-sim/spec.md
Implementation Plan: specs/001-misconception-dialogue-sim/plan.md
Task Breakdown: specs/001-misconception-dialogue-sim/tasks.md
Status: specs/001-misconception-dialogue-sim/STATUS.md
Deployment: docs/deployment.md
API Reference: See API Endpoints section above

🤝 Contributing

Guidelines

Follow TDD workflow (tests first)
Adhere to code style (Black + Ruff)
Keep files under 300 lines
Keep lines under 80 characters
Add type hints
Update tests and documentation
Run full test suite before submitting

Git Workflow

# Create feature branch
git checkout -b feature/your-feature-name

# Make changes and commit
git add .
git commit -m "feat: Add your feature description"

# Push and create pull request
git push origin feature/your-feature-name

📊 Performance

Current Metrics

Response Time: <200ms for API calls
Database: SQLite with WAL mode for concurrency
Rate Limiting: 5 logins/min, 30 messages/min
Workers: 4 uvicorn workers recommended (2-core)

Optimization Tips

Use --workers based on CPU cores: (2 x cores) + 1
Enable SQLite WAL mode (already configured)
Configure nginx caching for static assets
Monitor with /metrics endpoint

🔐 Security

Current Measures

Session-based authentication with secure cookies
Rate limiting (slowapi)
CORS configuration
Security headers (X-Frame-Options, CSP, HSTS)
Input validation (Pydantic)
SQL injection prevention (SQLAlchemy ORM)

Security Best Practices

Use strong SESSION_SECRET (32+ random bytes)
Enable HTTPS in production (Let's Encrypt)
Restrict database file permissions (640)
Regular dependency updates
Monitor logs for suspicious activity

📈 Monitoring

Health Check

curl https://your-domain.com/health

Response:

{
  "status": "healthy",
  "database": "connected",
  "timestamp": "2025-01-06T12:00:00Z"
}

Metrics

curl https://your-domain.com/metrics