Accumulative studies suggest that people with dyslexia have impaired categorical perception of speech. However, the origin of this deficit remains unknown. We now propose that impaired sensitivity to distributional statistics in dyslexia underlies this deficit. This proposal is based on the following rationale. The literature on acquiring speech categories in the general population attributes a major role to learning distributional statistics. Sensitivity to the statistics of stimuli used in perceptual discrimination tasks is impaired in dyslexia. Specifically, dyslexics’ implicit memory of previous stimuli decays faster than that of good readers and their stimulus-specific adaptation processes are shorter. Their faster decay may limit the temporal window over which they implicitly calculate stimulus distributions, and yield slightly impoverished speech categories. Since impaired sensitivity to stimulus statistics and impaired categorisation in dyslexia are domain general, we further propose that impaired categorical speech perception is one manifestation of a broader deficit in distributional learning.