Maps serve as a ubiquitous organizing principle in the mammalian brain. In several sensory systems, such as audition, vision, and somatosensation, topographic maps are evident throughout multiple levels of brain pathways. Topographic maps, like retinotopy and tonotopy, persist from the receptor surface up to the cortex. Other maps, such as those of orientation preference in the visual cortex, are first created in the cortex itself. Despite the prevalence of topographic maps, it is still not clear what function they subserve. Although maps are topographically smooth at the macroscale, they are often locally heterogeneous. Here, we review studies describing the anatomy and physiology of topographic maps across various spatial scales, from the smooth macroscale to the heterogeneous local microarchitecture, with emphasis on maps of the visual and auditory systems. We discuss the potential advantages of local heterogeneity in brain maps, how they reflect complex cortical connectivity, and how they may impact sensory coding and local computations.