GoRight

GoRight is a small reinforcement-learning environment from Talvitie et al. (2024) where the reward is hidden behind timing.

The robot can only move left or right along a hallway, and every step to the right costs −1. The light strip on the wall changes every step, cycling through off, dim and bright in a fixed pattern that does not depend on what the robot does. If the robot enters the prize cell while the strip is bright, the two prize lights turn on, and from then on every step right pays +3 instead. Arrive at any other moment and the lights stay off.

Try to time it yourself (the pattern repeats every 9 steps, and the status history under the controls shows the last few). Then compare a Random walk, which keeps paying −1 and only stumbles on the prize by luck, with the Optimal agent, which knows the rules and plans its arrival.

GoRight

Double GoRight

A longer hallway with a second prize at the far end, worth +6, so the +3 prize is now a distraction on the way. Which one the optimal agent goes for depends on the discount γ, how much it values future rewards: at 0.9 it settles for +3, at 0.99 it heads for +6.

Keep going

These demos go with the Reinforcement Learning notes, which build these ideas step by step.

• RL notesReinforcement Learning

  The full learning path, from multi-armed bandits to policy gradients.

• Chapter 2Markov Decision Processes

  How returns and discounting work, and why γ decides which prize the optimal agent goes for.

• Chapter 3Dynamic Programming

  How an agent that knows the dynamics computes an optimal policy, like the optimal agent here.

• DemoExploring a Grid World

  Drive a robot vacuum to the dust yourself, then see how long a random walk takes to find it.