Maze Reinforcement Learning with TorchSharp

🏗 Overview

This project implements a Reinforcement Learning (RL) model using TorchSharp to navigate a predefined maze. The model utilizes Q-learning, a value-based reinforcement learning algorithm, to train an agent to reach a goal while avoiding walls.

✨ Features

• 🔹 2D Grid Maze: 0 (walls), 1 (walkable paths), 2 (goal)
• 🔹 Q-learning with a Torch Tensor-based Q-table
• 🔹 Customizable reward system:
  • Walls: -500
  • Floors: -10
  • Goal: 500
• 🔹 Epsilon-greedy strategy for action selection
• 🔹 Fully configurable training parameters
• 🔹 Trained model navigates the maze autonomously

📦 Installation

Ensure you have the following installed:

• .NET 9 (Download)
• TorchSharp (GitHub)

To install dependencies:

 dotnet add package TorchSharp

⚙️ How It Works

🏁 Maze Representation

• The maze is represented as a 2D integer array.
• The agent starts at (11,5), trying to reach (0,5).

📚 Q-learning Process

• Initialize Q-values as a Torch Tensor.
• Training Loop:
  • Explore maze with random and learned actions.
  • Update Q-values using Temporal Difference:

    Q(s, a) = Q(s, a) + learning_rate * (reward + discount_factor * max(Q(s', a')) - Q(s, a))

• Navigation & Testing: The trained agent navigates the maze.

🛠 Configuration

You can modify hyperparameters in Program.cs:

const float Epsilon = 0.95f;         // Exploration-exploitation balance
const float DiscountFactor = 0.8f;   // Future reward consideration
const float LearningRate = 0.9f;     // Q-value update rate
const int Episodes = 1500;           // Training iterations

🚀 Running the Program

To run the project, execute:

 dotnet run

🔮 Future Enhancements

• 🏗 Dynamic maze generation
• 📊 Visualization of agent movements
• 🤖 Deep Q-Network (DQN) for scalability