In the traffic environment, reward was inbound flow change: clear cars, get positive reward; let queues grow, get punished. It sounded clean. It still needed careful state design to avoid pathological policies.
Compact state, big consequences
State was a tuple: signal phase, queue counts per approach, and whether the junction was occupied. Too little and the agent cannot distinguish situations. Too much and tabular Q-learning never fills the table. We landed at 649 unique states and 1,149 Q-entries after 10,000 training episodes.
Hyperparameters are part of the story
α = 0.1, γ = 0.6, ε = 0.1. Those numbers are not magic — they are the learning rate, discount, and exploration rate that produced the persisted Q-table. Documenting them next to the results made the experiment reproducible for future-me.
Collisions as a second scoreboard
Average wait time alone can hide unsafe behavior. Tracking collisions per episode forced a second lens on success. A fast intersection that crashes is not a win — in simulation or in product metrics.