Neuronal basis of object segmentation in macaque visual cortex

Image segmentation – the process by which scenes are segmented into component objects – is a fundamental aspect of vision and a cornerstone of scene understanding; its neural basis, however, is largely unknown.  To begin to understand how early visual representations are transformed in successive stages to facilitate segmentation and scene understanding, we studied the responses of neurons in mid- and high-level processing stages along the ventral object processing pathway of the primate brain.  In one series of experiments, monkeys performed a shape discrimination task as we studied responses and shape selectivity of neurons to partially occluded shapes under different levels of occlusion.  Two classes of neurons were identified in V4 and IT based on relative responses to unoccluded and occluded shapes.  Simulations revealed that the response characteristics of both classes of neurons were consistent with a model in which the responses to the occluded shape and the occluders are weighted separately and linearly combined.  Our results indicate that when a target stimulus is salient relative to the surrounding clutter, either by featural contrast or by perceptual grouping of distractors, pooled encoding of summary statistics fails to explain responses. Overall, our results support the hypothesis that a segmented representation of salient objects begins to emerge in the mid-level processing stages in the primate brain.