Rodent cerebral cortex is a major model for understanding neural circuit function, including how circuits represent and process sensory information and store information during plasticity and learning. In the whisker somatosensory cortex, sensory information is represented sparsely (with very few action potentials). Sparse coding is implemented by cortical circuit mechanisms that include robust, cell-specific inhibition from GABAergic circuits. Sensory experience powerfully regulates inhibitory circuit function to both rapidly alter, and then more slowly restore, excitation-inhibition balance in individual neurons. These dynamics may allow the cortex to rapidly compensate for reduced levels of sensory input, but then to re-establish sparse sensory processing during long-term circuit reorganization. Disruption of the mechanisms that establish and maintain excitation-inhibition balance may contribute to autism, epilepsy, and other cortical circuit diseases.