RL Lab · RL-002 · complete
Q-Learning Navigation in a 4×4 GridWorld
A small deterministic grid environment should make the Q-learning state-action update and learned navigation policy directly observable before moving to a neural Deep RL implementation.
Experiment setup
Environment and method.
Environment: Custom 4×4 GridWorld
Algorithm: Tabular Q-Learning
Status: complete
Outcome
Recorded result.
Trained for 500 episodes with a 16-state, 4-action Q-table. The learned greedy policy routes from state 0 to the goal at state 15 while avoiding obstacle states 5 and 10; the recorded policy corresponds to a 6-move route: 0 → 4 → 8 → 9 → 13 → 14 → 15.
What this means
Interpretation and boundary.
GridWorld made the tabular update concrete: Q-values encode long-term action value, obstacles create explicit negative feedback, and the greedy policy can be inspected directly. This experiment is deliberately a bridge into deeper value-function methods rather than a claim of broad Deep RL capability.
Metrics
Recorded metrics.
- Grid size: 4 × 4
- States: 16
- Actions: 4
- Training episodes: 500
- Greedy route: 6 moves
Reproducibility
Recorded configuration.
- episodes: 500
- alpha: 0.10
- gamma: 0.99
- epsilon: 1.00 → 0.01
- decay: 0.995
- seed: 42
- max steps: 200 / episode
Source and artifacts