How does the brain generate a crisp image of the visual scene?

January 18, 2011

Photographers know that a snapshot, taken with a shaking hand, will produce a blurry image. To reduce the blur, some cameras include a built-in stabilizing mechanism, which senses the hand’s motion and generates a compensating shift in the path of light from the lens to the sensor. A new study, carried out by researchers from the Hebrew University and from Harvard University, suggests that the human brain faces a similar problem, and may apply a similar solution.

Human eyes are constantly moving, even when we fixate our gaze on a static point. The motion is surprisingly large in comparison with the fine features that our visual system can detect: as we read the smallest letters in an optometrist’s eye chart, for example, the projection of the letters on our retina shifts by more than a letter’s full size, yet, we see a crisp image and we are not even aware that the image on the retina is moving. Evidently, the brain must apply some sort of compensating algorithm, without which the random motion of the eyes would hopelessly blur fine features in the visual scene, such as the small differences between the letter ‘E’ and the letter ‘F’.

 A new study, headed by Prof. Haim Sompolinsky from the Edmond and Lily Safra Center for Brain Sciences at the Hebrew University, with colleagues Dr. Yoram Burak, Dr. Uri Rokni, and Prof. Markus Meister from Harvard University, offers a possible solution to this problem. The study, appearing this month in the Proceedings of the National Academy of Sciences (USA), demonstrates that the brain can apply a compensating shift in the information path from the eye to the brain, in order to stabilize the drifting image in its brain representation.

To implement such a mechanism, the brain would need to use the signals coming out of the retina not only to infer the image but also to infer how the eye moves. This information will then be used generate a representation of the image within the brain that automatically corrects for this motion.

“We do not know yet whether the brain actually uses this strategy to compensate for eye movements”, according to Haim Sompolinsky, “but we have shown that such a solution is possible, and proposed ways to test this idea in new experiments. What is clear from our work is that the constant random movements of the eye pose a serious challenge to the visual system. According to our proposal, the brain’s solution might be to stabilize the image representation, similar to image stabilization mechanisms implemented in cameras. If proved, this study will advance our understanding of how the brain copes with the fluctuations and noise inherent in the signals coming from the organism’s sense organs.

Prof. Sompolinksy
Prof. Haim Sompolinsky, a snapshot taken by the steady hand of Abir Sultan.

The full research article can be found here.