Milestones
List view
- Update README with architecture diagram, results, and run instructions - Write up findings (what worked, what didn't, lessons learned) - Final advisor presentation / demo
No due dateQuantify everything. - Implement Elo rating system across training generations - Build tournament runner (round-robin across agent snapshots) - Run full ablation study (drafting reward on/off, blocking reward on/off) - Generate training curves, Elo progression charts, and head-to-head win rates - Record video of final agents racing (for demo/portfolio)
No due date•0/3 issues closedAdd tactical depth. - Design modular reward function with toggleable components: - Speed reward - Position/rank reward - Drafting proximity bonus - Blocking penalty/reward - Implement behavioral priors for agent archetypes ("Speedster" vs. "Blocker" vs. "Drifter") via reward weight initialization - Run initial ablation: train with/without drafting reward, compare Elo trajectories
No due date•0/2 issues closedIntroduce competition. - Extend environment to support 2+ simultaneous agents - Implement shared physics step across all agents - Define competitive reward function (position delta, lap time, proximity bonus) - Implement self-play loop: periodically snapshot agent → add to opponent pool - Implement league sampling (uniform or prioritized opponent selection) - Verify agents don't degenerate (reward collapse, spinning in place, etc.)
No due date•0/3 issues closedImplement the learning algorithm from scratch. - Implement Actor-Critic neural network (separate heads or shared trunk) - Implement GAE (Generalized Advantage Estimation) - Implement PPO clipped surrogate objective - Implement rollout buffer and minibatch updates - Single-agent training loop (agent learns to complete a lap) - Validate with TensorBoard: episode reward should trend upward
No due date•0/4 issues closedBuild the simulation core. - Set up PyBox2D + Gymnasium custom environment - Implement vehicle physics (friction, momentum, slip angle approximation) - Build a basic oval/circuit track with boundary collision - Define observation space (position, velocity, heading, opponent relative position) - Define action space (continuous: steering, acceleration, braking) - Manual control test (keyboard input to verify physics feel right)
No due date•0/5 issues closed