Reading difficulties in dyslexia are often attributed to poor use of phonological categories. In this study, we tested whether poor category-use in dyslexia may result from poor learning of new sound categories in general. To do that, we asked adult participants with and without dyslexia to discriminate which of two serially-presented tones had a higher pitch. This task included two conditions, with and without a repeated reference pitch. The reference condition introduces regularity, which, when detected, facilitates the formation of “high” and “low” pitch categories above and below this reference, respectively. To study the neural processes involved in this learning, participants performed this task during fMRI scanning. Behaviorally, individuals with dyslexia learned the sound categories slower than individuals without dyslexia. Neurally, individuals without dyslexia showed adaptation to the repeated sounds that paralleled the category learning in their primary auditory cortex and in other higher-level regions. In dyslexia, no brain region showed such adaptation. We suggest that poor learning of sound regularity in sensory regions may underlie the poor representations of both speech and non-speech categories in dyslexia.