GridWorld DQN Experiments

Cooperative coverage experiments for CPSC 4420/6420 exploring single-agent and multi-agent Deep Q-Network (DQN) approaches in GridWorld environments. The repo contains both the final report and the Python training code that produced the figures/tables.

Contents

• Cooperative_Robots_DQN_GridWorld_Report.docx – final write-up describing the motivation, environment design, methodology, and results.
• main.py – single-agent multi-goal GridWorld plus a baseline DQN agent and a helper sweep across different grid sizes.
• multi_agent_gridworld.py – partially observable multi-agent GridWorld with a shared Double DQN policy and a CLI for targeted runs.
• run_multi_sweep.py – convenience script that instantiates the multi-agent environment for each configuration needed for the final table and prints the metrics rows.

Requirements

• Python 3.9+
• PyTorch, NumPy, Matplotlib (installable via pip install torch numpy matplotlib)

A simple way to set up the project locally:

python3 -m venv .venv
source .venv/bin/activate
pip install torch numpy matplotlib

Usage

Single-agent baseline

Generate learning curves for the vector-to-goal environment:

python main.py --mode single --episodes 800

This prints progress logs every 50 episodes and produces Matplotlib figures summarizing reward, goals collected, and success rate.

Multi-agent partially observable setting

Train the shared Double DQN policy with the default 30×30 map:

python main.py --mode multi --agents 3 --episodes 1000

For more fine-grained control (grid size, vision radius, replay size, etc.), call the dedicated module:

python multi_agent_gridworld.py --size 20 --goals 50 --obstacles 10 \
    --agents 4 --vision 3 --episodes 600 --batch_size 256 --replay 200000

Both entry points print running metrics and final tail-averaged statistics (last 20/100 episodes for goals, coverage, and success rate).

Reproducing the report table

The sweep helper mirrors the configurations used in the write-up (10×10 through 50×50 grids) and outputs pre-formatted rows:

python run_multi_sweep.py --episodes 800 --agents 3 --vision 4

Each block prints a TABLE ROW: line that can be copied directly into the report.

Notes

• Training can be GPU-accelerated if PyTorch detects CUDA, but the defaults run on CPU.
• Figures generated by main.py --mode single open in interactive windows; run in a local desktop session or save via Matplotlib if executing remotely.
• The repository deliberately ignores large artifacts (virtual environments, datasets, presentation decks) so only the report and core Python sources are versioned.