Reduced short-term memory (STM) of individuals with dyslexia (IDDs) and enhanced STM of musicians are well documented, yet their causes are disputed. We hypothesized that their STMs reflect their sensitivities to accumulative long-term stimuli statistics. Indeed, when performing an STM task, IDDs had reduced benefit from syllable frequency, whereas musicians manifested an opposite effect, compared to controls. Interestingly, benefit from sequence-repetition did not significantly differ between groups, suggesting that it relies on different mechanisms. To test the generality of this separation across populations, we recruited a group of good-readers, whose native language contains a smaller fraction of the high-frequency syllables. Their span for these “high-frequency” syllables was small, yet their benefit from sequence-repetition was adequate. These experiments indicate that sensitivity to long-term stimuli distribution, and not to sequential repetition, is reduced in IDDs and enhanced in musicians, and this accounts for differences in their STM performance.