What, Exactly, Is This Referee Looking At?