Neural network models of associative memory with uniform inhibition are studied. It is shown that for sufficiently strong inhibition the system orders only partially even at low temperatures. A fraction of the neurons freezes in a quiescent state while the activities of the rest of the neurons fluctuate in time around an average level that is small compared to the saturation level. These models may help understanding the origin of the low neuronal firing rates observed in cortical recordings.