Topographic maps are common throughout the nervous system, yet their functional role is still unclear. In particular, whether they are necessary for decoding sensory stimuli is unknown. Here we examined this question by recording population activity at the cellular level from the larval zebrafish tectum in response to visual stimuli at three closely spaced locations in the visual field. Due to map imprecision, nearby stimulus locations produced intermingled tectal responses, and decoding based on map topography yielded an accuracy of only 64%. In contrast, maximum likelihood decoding of stimulus location based on the statistics of the evoked activity, while ignoring any information about the locations of neurons in the map, yielded an accuracy close to 100%. A simple computational model of the zebrafish visual system reproduced these results. Although topography is a useful initial decoding strategy, we suggest it may be replaced by better methods following visual experience.