1First Department of Physiology, School of Medicine, Nihon University, Tokyo 173-8610, Japan; and 2Università degli Studi di Parma, Istituto di Fisiologia Umana, Plesso Biotecnologico Integrato, 39-I43100 Parma, Italy
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Murata, Akira, Vittorio Gallese, Giuseppe Luppino, Masakazu Kaseda, and Hideo Sakata. Selectivity for the Shape, Size, and Orientation of Objects for Grasping in Neurons of Monkey Parietal Area AIP. J. Neurophysiol. 83: 2580-2601, 2000. In this study, we mainly investigated the visual selectivity of hand-manipulation-related neurons in the anterior intraparietal area (area AIP) while the animal was grasping or fixating on three-dimensional (3D) objects of different geometric shapes, sizes, and orientations. We studied the activity of 132 task-related neurons during the hand-manipulation tasks in the light and in the dark, as well as during object fixation. Seventy-seven percent (101/132) of the hand-manipulation-related neurons were visually responsive, showing either lesser activity during manipulation in the dark than during that in the light (visual-motor neurons) or no activation in the dark (visual-dominant neurons). Of these visually responsive neurons, more than half (n = 66) responded during the object-fixation task (object-type). Among these, 55 were tested for their shape selectivity during the object-fixation task, and many (n = 25) were highly selective, preferring one particular shape of the six different shapes presented (ring, cube, cylinder, cone, sphere, and square plate). For 28 moderately selective object-type neurons, we performed multidimensional scaling (MDS) to examine how the neurons encode the similarity of objects. The results suggest that some moderately selective neurons responded preferentially to common geometric features shared by similar objects (flat, round, elongated, etc.). Moderately selective nonobject-type visually responsive neurons, which did not respond during object fixation, were found by MDS to be more closely related to the handgrip than to the object shape. We found a similar selectivity for handgrip in motor-dominant neurons that did not show any visual response. With regard to the size of the objects, 16 of 26 object-type neurons tested were selective for both size and shape, whereas 9 object-type neurons were selective for shape but not for size. Seven of 12 nonobject-type and all (8/8) of the motor-dominant neurons examined were selective for size, and almost all of them were also selective for objects. Many hand-manipulation-related neurons that preferred the plate and/or ring were selective for the orientation of the objects (17/20). These results suggest that the visual responses of object-type neurons represent the shape, size, and/or orientation of 3D objects, whereas those of the nonobject-type neurons probably represent the shape of the handgrip, grip size, or hand-orientation. The activity of motor-dominant neurons was also, in part, likely to represent these parameters of hand movement. This suggests that the dorsal visual pathway is concerned with the aspect of form, orientation, and/or size perception that is relevant for the visual control of movements.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Since the early studies on the neural activity of the parietal
cortex in alert behaving monkeys, it has been known that many neurons
in the inferior parietal lobule are related to hand manipulation or
grasping as well as to visually guided reaching (Hyvärinen and Poranen 1974; Mountcastle et al. 1975
).
Recently we studied a group of neurons that were related to visually
guided hand-manipulation tasks in the anterior part of the lateral bank
of the intraparietal sulcus (area AIP) of the macaque (Sakata et
al. 1995
; Taira et al. 1990
). Many of them were
activated during fixation on objects and responded selectively to the
types of switch to be manipulated. These results suggested that these
neurons were selective for the shape and/or orientation of the object.
This appeared to contradict the general assumption of the existence of
two cortical visual pathways, i.e., a ventral stream projecting to the
inferotemporal (IT) cortex concerned with object vision and a dorsal
stream projecting to the parieto-occipital cortex concerned with space
vision (Mishkin et al. 1983; Ungerleider and
Mishkin 1982
). A number of electrophysiological studies have revealed that many neurons in the IT cortex respond selectively to
objects of simple or complex shapes (Desimone et al.
1984
; Gross et al. 1972
; Logothetis and
Sheinberg 1996
; Tanaka et al. 1991
). Although
many IT neurons were activated by three-dimensional (3D) objects, most
of them also were found to be equally sensitive to two-dimensional (2D)
stimuli. Many of these shape-selective neurons were also selective for
color or texture and often showed size invariance (Gochin
1996
; Ito et al. 1995
; Sato et al.
1980
; Schwartz et al. 1983
) or viewpoint
invariance (Booth and Rolls 1998
; Hasselmo et al.
1989
).
On the other hand, several clinical studies have suggested that the
discrimination of shape, orientation, and size are also important for
hand manipulation (Jeannerod 1994; Perenin and
Vigheto 1988
). Goodale and Milner (1992)
emphasized that the dorsal visual pathways projecting to the parietal
cortex constitute the "how system," which provides action-relevant
information regarding the structural characteristics and orientation of
objects. The results of recent functional brain imaging studies support
this hypothesis (Binkofski et al. 1998
; Faillenot
et al. 1997
). Furthermore recent electrophysiological studies
of the monkey have revealed that parietal visual neurons are not
limited to those that are sensitive to visual motion or spatial
position (Duffy and Wurtz 1991a
,b
; Galletti et
al. 1993
; Sakata et al. 1994
) but also include those that are sensitive to 3D stimuli and are selective for surface orientation or axis orientation in depth (Sakata et al.
1997a
,b
) and those that respond selectively to 2D shapes
(Sereno and Maunsell 1998
). These findings suggest that
the neural correlates of form vision are not confined to the ventral
visual pathway but also involve the dorsal visual pathway.
In our previous experiments, the shapes of the switches given to the subjects for manipulation were complex and were selected primarily to elicit different types of hand manipulation. Therefore it was difficult to determine whether the selectivity of these visual neurons was related to the 3D shape of the objects or to some other features related to a difference in the type of hand manipulation. The main purpose of the present study was to clarify whether the visually responsive hand-manipulation-related neurons, which responded to the sight of objects (object-type), in area AIP truly discriminate the shape of the objects manipulated. For this purpose, we used geometric solids (plate, ring, cube, cylinder, cone, and sphere) on the head of a pull switch covering a wide range of elementary 3D shapes while restricting the movements to uniform grasping and pulling. We also attempted to study the extent of selectivity of these neurons for the size and/or orientation of objects because these are important variables in the visual control of hand movement. In addition, we studied the object selectivity of the visually responsive neurons that did not respond to the sight of objects (nonobject-type), as well as that of motor-dominant neurons that did not show any difference in the responses between the light and dark, for comparison.
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We trained four Japanese monkeys to perform hand-manipulation and fixation tasks. During the experiments, each monkey was seated in a primate chair with its head fixed, facing a box separated into upper and lower parts by a half mirror (Fig. 1). In the lower part was a turntable separated into six sectors each containing a solid of different shape, namely, plate, ring, cube, cylinder, cone, and sphere. Before starting the tasks, the monkey was kept in complete darkness. One sector on the turntable was opened at a time, and the monkey could see the object through the half-mirror when the light was turned on in the lower part. The objects were painted white and stood out against the black background. A spot of light of a red/green light-emitting diode (LED) was reflected by the half mirror and superimposed on the object. The objects were presented in random order by turning the table with a pulse motor controlled by a computer during each intertrial interval, except during the task of manipulation in the dark, in which the same object was illuminated before the start of the task and presented consecutively through a block of trials.
|
Behavioral tasks
HAND-MANIPULATION TASK IN THE LIGHT. When the red spotlight was turned on, the monkey was trained to fixate on it and press a home key, placed at lap level, for a period of 1.0-2.0 s (Fig. 1). At this time, the light in the lower part of the box was turned on to illuminate the object. When the color of the spot changed from red to green, the monkey was trained to release the key, reach forward and grasp the object, and then pull it. The moment the key was released, the LED color changed from green back to red. The monkey was required to keep the object pulled for 1.0-1.5 s, until the spot changed color again from red to green, and then release it. Thus the monkey maintained its fixation on the spot from the beginning to the end of each trial. The manipulation task was divided into two periods: the "fixation" period, during which the monkey fixated on the spot superimposed on the object before movement onset, and the "movement" period, which was subdivided into the "initial" period corresponding to the dynamic phase of reaching and grasping and the "hold" period, corresponding to the period during which the monkey kept the object pulled. We presented the objects randomly during the hand-manipulation task in the light. The monkeys had to perform 8-10 trials for each of the objects.
HAND-MANIPULATION TASK IN THE DARK. To exclude visual responses, the monkeys also performed the hand-manipulation task in the dark guided by the color change of the spot in the half mirror to indicate the position of the object on which it was trained to fixate during each trial. We presented the same object consecutively in blocks of 8-10 trials. Before starting a block of trials, we turned on the light briefly to show the object to the monkey. The hand movements were monitored in the dark with an infrared video camera to ensure that the monkey used the same type of handgrip in the dark as in the light.
OBJECT-FIXATION TASK. To determine whether the neuron was responding to the sight of the object, the monkey also was trained to fixate on the spot superimposed on the object without manipulating it (object fixation in the light; Fig. 1). The activity due to fixation per se was determined during the task of fixation on the reflected spot in the dark (fixation in the dark). During both the fixation tasks, when the color of the spot was green, the monkey was trained to fixate on the spot while pressing a home key for 1.5 s until the spot changed color from green to red. In the fixation task, we refer to the period of fixation on the spot as the "fixation" period.
We routinely used six primitive geometric solids (plate, ring, cube, cylinder, cone, and sphere). We selected them on the basis of subroutines of "object primitives" of a system of computer graphics (Dore, Stardent Computer). We used three sets of these objects in different sizes (small, medium, and large): plate (45-mm wide, 40-mm deep, and 3-, 7.5-, or 15-mm thick), cylinder (45-mm long with diameter of 5, 15, or 30 mm), ring (diameter of 15, 30, or 50 mm), cube (face diagonal of 10, 20, or 30 mm), cone (bottom diameter of 10, 20, 30 mm and height of 15, 30, 45 mm), and sphere (diameter of 10, 20, or 30 mm). To compare the neuronal activity in response to different shapes, we routinely used medium-sized objects. To examine their size selectivity, we presented all three different sizes of the shapes preferred by the neurons. We studied the orientation selectivity of the neurons by changing the orientations of the plates and rings at 45° intervals in the frontal plane, although these objects were presented routinely in the same orientation to study the shape selectivity of the neurons. We tested the orientation or size selectivities during the manipulation task only.Unit recording and data analyses
After behavioral training of the monkeys, surgery was performed
under general anesthesia (pentobarbital) to implant stainless steel
bolts for fastening a halo ring for head fixation, a magnetic search
coil for eye-movement recording, and a stainless steel cylinder for
microelectrode recording around a trephine hole in the skull over the
anterior part of the inferior parietal lobule (IPL). Extracellular
recordings of single-unit activity were made using tungsten
microelectrodes (Frederick Haer). Microelectrode penetrations were made
primarily in area AIP (Sakata and Taira 1994;
Sakata et al. 1995
), which is located in the rostral
part of the lateral bank of the intraparietal sulcus.
To determine the differences in activity levels in the different task conditions, we performed statistical analyses on the activity levels during the manipulation and fixation tasks either in the light or dark. We compared the mean discharge rate (spikes/s) between the task period and the "reference" period (1.0 s before presentation of the LED spot) to determine task-related activity (Wilcoxon's signed-rank test, P < 0.05). In nonobject visual-motor and -dominant neurons as well as in motor-dominant neurons, we measured task-related activity during the movement period of the manipulation task because there was no activation during the fixation period. We calculated the mean net activity of neurons by subtracting the discharge rate during the "reference" period from that during the task period to examine if there were any differences in the mean net activity between the different task conditions (Mann-Whitney U test, P < 0.05). If the mean net activity during the manipulation task or fixation task was not significantly greater than that during fixation in the dark, we determined that there was no movement-related activity or visual response, respectively.
To assess the variations in the net activity level in relation to the shape, orientation, and size of the objects, we performed one-way ANOVA (2-tail, P < 0.05) for the activity during the manipulation and fixation tasks. When significant variations in the net activity levels were determined by ANOVA, we performed multiple comparisons to examine the differences in net activity levels among the six shapes by the Student-Newman-Keuls procedure (2-tail, P < 0.05). If the neuronal activity level for one object was significantly higher than that for the other objects, we classified the neurons as being highly selective. If the neuronal activity level for the most preferred object was not significantly higher than that for the other objects, we designated these neurons as being moderately selective. The last group, the activity levels of which were not significantly different for the different objects, was designated as nonselective neurons.
In the moderately selective neurons, we assessed how the neurons in
area AIP encode the similarity among the six objects using multidimensional scaling (MDS; SPSS statistical package). We
constructed a matrix in which the cells were represented by columns,
and the mean net activity for each object, during the fixation task in object-type neurons, or during the manipulation period in
nonobject-type and motor-dominant neurons, was represented by rows. We
calculated the Euclidean distance between each pair of objects
(Young and Yamane 1992, 1993
) as an index of similarity,
which is the square root of the sum of the squared differences between
the activities of individual neurons for the two objects. Based on
these distances, MDS was performed to construct a map of the location
of the objects relative to each other. In the map, objects that evoked
similar responses were placed close together.
Histological analysis
We recorded the activities of neurons from seven hemispheres of four monkeys. After completion of the recording from both hemispheres, a series of electrolytic lesions were induced along several of the penetrations. A few days after the induction of the lesions, the monkeys were anesthetized deeply with an overdose of pentobarbital and perfused. Histological sections were made along the frontal plane in all seven hemispheres. One of every four sections was stained with thionine to trace the penetrations and verify the electrolytic lesions. The unit recording sites were determined indirectly from the relative positions of the penetrations to the electrolytic lesions and plotted on an unfolded map of the intraparietal sulcus (IPS). We plotted the recording sites in five hemispheres of three monkeys in which the direction of the penetrations were parallel to the frontal plane.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Types of neurons
We recorded single-unit activities from the anterior part of the
lateral bank of the intraparietal sulcus (area AIP) and its surrounding
areas. We recorded activities of 182 hand-manipulation-related neurons
from seven hemispheres of four monkeys. We classified these neurons
into three classes according to their level of activity during the
three different task conditions, as in the previous studies
(Sakata et al. 1995; Taira et al. 1990
).
"Visual-motor" neurons were less active during manipulation in the
dark than during that in the light. This reduction in the level of
activity is attributable to the loss of the visual stimulus rather than to habituation due to the repetitive presentation of the same object in
the dark, because the impulse rasters did not show any gradual decrease
in activity in the later trials within the block. "Visual-dominant"
neurons were those that were not activated in the dark.
"Motor-dominant" neurons did not show any difference in activity
level between manipulation in the light and dark. We classified
visual-motor and -dominant neurons together as "visually responsive" neurons, which then were further subdivided into two types according to their responses in the object-fixation task in the
light. "Object-type" neurons were activated by the sight of the
object during the object-fixation task, whereas the
"nonobject-type" neurons were not activated during the
object-fixation task. The number and percentage of cells in these five
types of hand-manipulation-related neurons are listed in Table
1.
|
Typical examples of the five types of neurons are shown in Fig.
2. An object-type visual-motor neuron
(Fig. 2A) showed sustained activity during the fixation
period of the manipulation task in the light (Fig. 2A,
left), its activity was enhanced during the "initial"
part of the movement period and decreased again during the "hold"
period. The activity during the fixation period of the manipulation
task in the dark was decreased markedly compared with that in the
light, whereas the activity during the movement period in this task
remained partially decreased due to the lack of visual input (Fig.
2A, center). It should be noted that the residual
activity during the fixation period in the dark observed in other
visual-motor neurons (13/41) may be attributable to the set-related
activity as observed in the premotor cortex (Murata et al.
1996; Weinrich and Wise 1982
). The neuron fired
during the object-fixation task in the light even in the absence of
subsequent movements (Fig. 2A, right). On the
other hand, a nonobject-type visual-motor neuron (Fig. 2B)
was not activated during the fixation period of the manipulation task
in the light and started exhibiting activity only after the onset of
movements during the task (Fig. 2B, left). The
neuron exhibited a lower level of activity during the movement period
of the manipulation task in the dark than during that in the light
(Fig. 2B, central). The difference between the
activity levels in the light and in the dark could be attributable to
some visual component other than the sight of the object because there
was no activation during the object-fixation task (Fig. 2B,
right). The object-type visual-dominant neuron shown in Fig. 2C exhibited sustained activity during both the manipulation
and object-fixation tasks in the light, like the object-type
visual-motor neuron. However, it did not show any enhancement of
activity during the initial period of movement in the light nor did it
show any activity during manipulation in the dark. Thus the activity of this neuron could be attributable mainly to the visual response to the
object without any influence from the motor component. A nonobject-type
visual-dominant neuron (Fig. 2D) started exhibiting activity
just before the monkey touched the object but did not show any activity
during either the manipulation task in the dark or the object-fixation
task in the light. Finally, a motor-dominant neuron did not show any
significant difference in the levels of activity between manipulation
in the light and dark and was not activated at all in the
object-fixation task (Fig. 2E). This class of neurons was
also not activated during the fixation period of the manipulation task
in the light, like the nonobject-type neurons.
|
Recording site
We recorded the hand-manipulation-related neurons in area AIP that
is immediately posterior to the hand area of the primary somatosensory
cortex (SI). Before we recorded from the bank of the IPS in this
experiment, we mapped the hand area in the SI to determine the
positions of the penetrations as shown in Fig. 3. The area in which we found the
hand-manipulation-related neurons was limited to an area located
rostral to the lateral intraparietal area (area LIP) and was adjoined
in depth with the ventral intraparietal area (area VIP) located in the
fundus of the IPS (Gallese et al. 1994; Sakata et
al. 1995
). We did not detect any tactile response in this area.
In area VIP, tactile, visual, and bimodal neurons were recorded, so
area AIP was distinguished from area VIP by the absence of tactile
response. In the rostral-most part of the bank, mouth-movement
(reaching with lips or grasping with mouth, etc.) -related activity or
tactile responses on the face combined with visual responses were
recorded, and this region probably corresponds to the "associative
face area" (Leinonen and Nyman 1979
), adjoining the
area in which we recorded hand-manipulation-related neurons. Finally,
the caudal border of area AIP was identified by the presence of visual
fixation or saccade-related neurons and by the absence of the
hand-manipulation-related neurons.
|
Five hemispheres of three monkeys were sectioned, and the recording sites in three left hemispheres of different monkeys are shown in Fig. 3. The object-type and nonobject-type visually responsive neurons are denoted by different symbols. The neurons of three different types (object-type, nonobject-type, and motor-dominant neurons) were intermingled in area AIP, although the motor-dominant neurons were predominant in the first hemisphere as seen from Fig. 3; only a few motor-dominant neurons were found in the other two hemispheres.
Selectivity for 3D object
To examine whether the hand-manipulation-related neurons exhibit any selectively for the 3D shapes of the objects manipulated, we compared the activity levels of hand-manipulation-related neurons during the manipulation task in the light using the set of six geometric solids routinely. We excluded those neurons that were not studied with the routine set of six objects (50/182) from the database for further analysis. The remaining 132 neurons were classified into five types as shown in Fig. 2 and subdivided further by their degree of selectivity for the objects (Table 2). We classified them as highly selective neurons when the net activity of the neurons for one object was significantly higher than that for the other objects or as moderately selective when the neuronal activity level for the most preferred object was not significantly higher than that for the other objects. The rest of the neurons that did not show any significant difference in the activity level for the six objects that were classified as nonselective (Table 2, the Student-Newman-Keuls procedure, 2-tail; P < 0.05). The number of highly selective neurons (n = 53) was comparable with the number of moderately selective neurons (n = 55). Seventy-seven percent (101/132) of the hand-manipulation-related neurons were visually responsive, whereas the motor dominant neurons constituted less than a quarter (31/132). The number of the object-type neurons (n = 66) was nearly twice as high as that of the nonobject-type neurons (n = 35) in the present study. Therefore we first describe the shape selectivity of the object-type neurons.
|
OBJECT-TYPE VISUAL-MOTOR NEURONS. Highly selective neurons. Figure 4 illustrates an example of a highly selective object-type visual-motor neuron. This neuron started to exhibit activity soon after the onset of fixation, and the activity was sustained during the movement period. However, the response for the ring or cube was much weaker than that for the plate, and inhibition was observed for the cylinder. Figure 5 shows the activity of the same neuron shown in Fig. 4 during the object-fixation task. Also under this task condition, the cell was highly selective for the plate, and inhibition was observed during fixation on the cylinder. Thus there was a precise correspondence of object preference between the manipulation and object-fixation tasks. Therefore this neuron visually preferred the plate among the six geometric solids, and its activity was enhanced by the movement suited to the manipulation of the preferred object. Among the 132 "hand-manipulation-related" neurons, 41 neurons were object-type visual-motor neurons of which 23 (56%) showed highly selective activity during the manipulation task (Table 2). We examined whether there was correspondence in the selectivity of the neurons between the manipulation and the object-fixation tasks.
|
|
|
|
|
SELECTIVITY OF OBJECT-TYPE VISUAL-DOMINANT NEURONS. Object-type visual-dominant neurons were activated mainly in response to the sight of an object but did not show any activity during manipulation of the object in the dark. Figure 9 illustrates the activity of a typical object-type visual-dominant neuron, shown in Fig. 2C, during the fixation task. The neuron showed vigorous sustained activity during fixation on the ring. There was also some activity, but at a much lower level, during fixation on the cylinder and no significant activity during fixation on the other objects.
|
|
|
SELECTIVITY OF OBJECT-TYPE VISUALLY RESPONSIVE NEURONS DURING THE OBJECT-FIXATION TASK. The activity of object-type visually responsive neurons during object fixation was considered to be predominantly a visual response to the object with minimal influence from the sustained hand movement during key pressing. Therefore we analyzed the activities of 55 of the 66 object-type neurons composed of visual-motor and -dominant neurons together during the object-fixation task. Of these 55, 25 neurons were highly selective during the object-fixation task, and among them (n = 25), 8 neurons preferred the plate, another 8 preferred the ring, 6 preferred the cylinder, and 3 preferred among the cube, sphere, and cone. It should be emphasized that we always presented both the plate and the ring in the same orientation, either horizontal or vertical, and these highly selective neurons for the ring or plate discriminated between these two objects despite this same orientation, suggesting that these neurons as well as other highly selective object-type neurons were primarily selective for the shapes of the objects.
To test whether the moderately selective neurons (n = 28) were selective for common geometric features of objects as described above, we performed MDS of these neurons on the basis of their responses to the sight of objects, similar to the analysis of the similarity of different faces by Young and Yamane (1992
|
SELECTIVITY OF NONOBJECT-TYPE VISUALLY RESPONSIVE NEURONS. The selectivities of the nonobject-type visual-motor and -dominant neurons were analyzed together because the number of these neurons was relatively small (visual motor, n = 20; visual dominant, n = 15). These neurons showed lower levels of activity during manipulation in the dark than during that in the light or no activation at all in the dark as shown in Fig. 2, B and D, respectively, suggesting that they respond to some visual input, although there was no activation during object fixation. Figure 13 illustrates an example of the activity of a nonobject-type visual-motor neuron during the manipulation task in the light. This neuron started to exhibit activity just before the monkey touched the plate and its hand came into the center of its visual field, and the activity was sustained during the hold period. This was a highly selective neuron because there was almost no activation for other objects except the ring, which activated the cell slightly toward the end of the manipulation task. Because this neuron was not activated during the fixation period, we considered that the visual response of the neuron was not related to the object and was most probably related to the particular pattern of handgrip for the plate. Because this neuron was a visual-motor neuron and was partly activated during manipulation in the dark, similar to the neuron shown in Fig. 2B, both visual and motor components might contribute to the selectivity of this neuron. Among the 35 nonobject-type visually responsive neurons, 13 (37%) were highly selective (Table 2), and 16 (41%) were moderately selective.
|
|
|
|
SELECTIVITY OF MOTOR-DOMINANT NEURONS. Figure 17 shows an example of a moderately selective motor-dominant neuron. This neuron became activated during the initial period of manipulation of the small cube, cone, and sphere and showed peak activity just before the object was touched, the activity being sustained during the hold period. The activity levels during the manipulation of these three objects did not show any significant differences, but the activity level for the plate was less than that for the other three objects. This is the same neuron as the one shown in Fig. 2E, the activity of which probably was related only to motor components without any contribution from the visual input. As described in the section on the selectivity of nonobject-type neurons, although the shapes of the cube, cone, and sphere are dissimilar, the movement pattern of the hand for grasping these objects was similar but different from the pattern for manipulating the plate for which the lateral surfaces of the index finger and thumb formed a flat surface. Of the 31 motor-dominant neurons, 8 (26%) were highly selective, 11 (35%) were moderately selective, and 12 (39%) were nonselective neurons. The proportion of nonselective neurons in this class of neurons was higher than that in the other types.
|
Selectivity for orientation
In our previous experiments, we found some
hand-manipulation-related neurons that were selective for the axis
orientation of a pull lever switch (Taira et al. 1990).
In the current experiment, we studied the orientation selectivity of
the neurons that showed strong activity during manipulation of the
plate and/or the ring (n = 20). Figure
18A shows the object-type
visual-motor neuron shown in Fig. 4. The neuron was activated
vigorously during the fixation period and the movement period in the
manipulation task for the vertical plate but exhibited a lower level of
activity for the plate in other orientations. This neuron was not
activated during manipulation of a vertical ring although its
orientation plane was the same as that of the vertical plate. On the
other hand, Fig. 18B illustrates a nonobject-type
visual-dominant neuron that was also selective for the vertical plate
but did not show any activity for the vertical ring. However, it was
activated, though only during the movement period, so that its activity
probably was related to the sight of a particular handgrip and hand
orientation.
|
Orientation selectivity was examined in 11 object-type neurons that preferred the plate and/or the ring. Ten of the 11 neurons were orientation-selective, and 5 of these 10 were highly selective for either the ring or the plate and exhibited much a lower level of activity for other objects. Therefore these neurons were selective for both the shape and the orientation of the object. On the other hand, three neurons were activated by both the plate and ring in the same orientation. We assumed that the activity of these three neurons was related to the orientation per se. At least two of them were tested for orientation selectivity with both the plate and ring and showed the same orientation tuning. In addition, three of four nonobject-type neurons tested were orientation-selective; two of them were highly selective for the plate and the third preferred the plate and ring in the same orientation. Four of five motor-dominant neurons tested were orientation-selective of which two were highly selective for the plate and the remaining two preferred the plate and ring in the same orientation.
Selectivity for size of object
In the present experiment, we studied the selectivity of neurons for the size of the object, using three different sizes (small, medium, and large) of the preferred objects usually. For most of the objects (sphere, cone, cube, and ring), the size change was uniform, but in the case of plate and the cylinder, only the thickness was changed.
Figure 19A shows an example of a size-selective neuron that was activated by the small cone with very low levels of activity for large objects. This is the moderately selective motor-dominant neuron shown in Fig. 17 that preferred the cone, sphere, and cube. On the other hand, Fig. 19B illustrates an example of an object-type visual-motor neuron that was activated by a large cube. The neuron showed moderately selective activity during manipulation of the large- and medium-sized cubes, plates, and cylinders.
|
Thirty-one of 46 neurons tested were selective for size. Seven cells preferred small objects, 16 preferred medium-sized objects, and 8 cells preferred large objects. Sixteen of 26 object-type neurons were also selective for the size of the preferred objects. They showed shape selectivity (nine highly selective and seven moderately selective). Thus these neurons were selective for the size and shape of objects. Among the 10 object-type neurons that were not selective for size or thickness, 9 neurons were selective for shape (4 highly selective and 5 moderately selective neurons). These neurons were selective for shape independent of their selectivity for size. Of the 12 nonobject-type neurons tested, 7 were selective for size, and 6 of them showed selectivity for shape; these latter 6 seemed to be selective for shape and size of the handgrip. The remaining five nonobject-type neurons preferred the plate but were not selective for its thickness, perhaps because there was not much difference in the shape of the handgrip with changing thickness in the case of the plate. All eight motor-dominant neurons tested showed selectivity for both size and shape. Some of the size-selective neurons (8/31) showed a gradual change in the level of discharge depending on the size of the objects, as shown in Fig. 19, A and B, suggesting that the rate of discharge was related to the size of the stimulus.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The large majority of hand-manipulation-related neurons in area
AIP of the parietal cortex are visually responsive (Sakata et
al. 1995; Taira et al. 1990
). In the present
study, we grouped two types of visually responsive neurons, the
visual-dominant and -motor neurons, together and further classified
them into two groups, i.e., the object- and nonobject type, according
to the presence or absence of activity, respectively, during object fixation in the light. We mainly studied the visual selectivity of
these neurons, in regard to the shape, size, and orientation of the
objects, using simple geometric solids as targets for manipulation and fixation.
Object shape representation in AIP
The main finding of the present study is that the large majority
of object-type visually responsive neurons were selective in their
response to the sight of 3D objects in elementary (primitive) geometric
shapes. The set of six geometric solids used in this study covered a
fairly wide range of primitive 3D shapes because we selected them on
the basis of subroutines of "object primitives" of a system of
computer graphics. About half of the object-type neurons (32/66) were
highly selective for the shape in the manipulation task in the light.
More than 80% of them (55/66) were examined with the same set of six
objects during the object-fixation task, and nearly half of them
(25/55) were highly selective. The high proportion of neurons with
sharp tuning to a particular geometric solid strongly suggests that the
visual responses of these neurons represent elementary 3D shapes. There
may have been sampling bias because we recorded only those neurons that
were sensitive to the six objects used in this study. In a previous
study, some neurons in area AIP responded to more complicated objects
such as a plate in groove, knob in groove, square button on a plate etc. (Sakata et al. 1995, 1999
). Some others may respond
to primitive geometric shapes other than those of the six objects, such
as a square column (Sakata et al. 1999
), circular disk
(Shikata et al. 1996
), etc. Nevertheless, the highly
selective AIP neurons recorded in the present study provide positive
evidence that they are sensitive to the 3D shape. The activity of the
moderately selective object-type neurons during the fixation task was
also likely to be related to the geometric shape because at least a proportion of them appeared to respond to common geometric features. The MDS-based on the activity during the object-fixation task showed
clustering of two distinct groups; one including the cylinder, cone,
and sphere (round objects) and the other including the plate and ring
(flat objects).
If the object-type visually responsive neurons specifically represented
the 3D shapes of objects, their response to 3D objects should be
viewpoint-invariant (Booth and Rolls 1998) and
size-invariant (Ito et al. 1996
; Schwartz et al.
1983
). However, as mentioned in the following text, many
neurons that responded selectively to the plate or ring also showed
selectivity for the orientation plane, suggesting that they had
combined selectivity for shape and orientation. Likewise, the majority
of the object-type neurons tested (16/26) showed size selectivity, and
only 10 neurons did not. The shape selectivity of these 10 neurons was
size-invariant. However, it must be emphasized that the shape
selectivity of the object-type neurons was verified even when it was
combined with the selectivity for size or orientation because we kept
other conditions constant while examining shape selectivity.
A number of electrophysiological studies have revealed that many
neurons in the IT respond selectively to simple or complex shapes of
visual objects (Desimone et al. 1984; Gross et
al. 1972
; Tanaka et al. 1991
). Although many IT
neurons were activated by 3D objects, most of them also were found to
be sensitive to 2D stimuli, such as disks, triangles, and stars, or
various Fourier descriptors (Schwartz et al. 1983
).
Tanaka et al. (1991)
attributed the selectivity of these
neurons for complex objects to their response to a combination of 2D
visual patterns derived by simplification of the original preferred
objects. Some IT neurons were more sensitive to highly complicated
shapes such as faces and hands than to simple geometric stimuli
(Desimone et al. 1984
; Perrett et al.
1982
). The use of these complex visual stimuli for IT neurons
was in sharp contrast to the use of simple geometric solids by us in this study of AIP neurons.
On the basis of psychophysical studies, two different mechanisms for 3D
object recognition have been proposed. One presumes that the visual
system recognizes 3D objects through 2D view interpolation rather than
viewpoint-invariant 3D representation (Bülthoff and Edelman 1992). The other hypothesis postulates that a 3D object is recognized by an arrangement of simple geometric components called
"geons" (Biederman 1987
). The objects used by us in
this study were comparable with geons (cube, sphere, and ring
corresponded to block, ellipsoid, and curved cylinder, respectively),
although objects such as the wedge, curved brick, and truncated cone
etc., were not used. Although many of the object-type neurons showed orientation selectivity and therefore were not viewpoint-invariant, the
highly selective object-type neurons were more likely to depend on
processes that encode 3D geometric shapes like geons than on processes
that encode 2D shapes. Moderately selective, object-type neurons may
respond to the common features of two or three geometric solids
corresponding to the intermediate stage of processing before geons,
termed by Marr (1982)
as the surface geometry.
Selectivity in pattern of grip
Nonobject-type visually responsive neurons were identified in a
previous investigation (Sakata et al. 1995). However, it
was not clarified what kind of visual input was important for the nonobject-type neurons, although the sight of hand movement is the most
plausible. Our present study demonstrated more clearly than previous
studies that the activity of these neurons was more related to the
shape of handgrip than to the object itself, because this class of
neurons was activated during the movement period when the grasping hand
was in view but not during fixation period in the manipulation task,
when the object alone was in view. We therefore estimated the degree of
selectivity of neurons of this type based on their activity levels
during the movement period rather than during the fixation task.
There were many nonobject-type neurons that were highly selective, suggesting that these neurons responded to the sight of a handgrip matching the shape and size of a particular object or the sight of the handgrip combined with the grasped object. We were able to recognize a similarity in the pattern of handgrips for two or three preferred objects in the case of moderately selective neurons. For example, the nonobject-type neurons that preferred the cube, cone, and sphere appeared to prefer similar handgrips, although the shapes of these objects were different. Actually these three objects were placed very close together by MDS based on the responses of the nonobject-type neurons.
Before the actual contact with an object that a subject attempts to
grasp, the hand forms a different spatial configuration according to
the shape, size, and orientation of the object, termed preshaping
(Jeannerod 1986). It is known that visual feedback signals are very important for the control of hand movements
(Jakobson and Goodale 1991
), especially to achieve
terminal accuracy in the preshaping phase (Jeannerod
1988
). Neurophysiological studies have revealed that neurons in
some cortical areas respond to the hand presented as a visual stimulus.
For example, neurons in the inferotemporal cortex were activated by
hand shapes as the visual stimuli (Desimone et al. 1984
;
Gross et al. 1972
) and neurons in the superior temporal
sulcus (area TEa) responded to the view of the hand action of an
experimenter (Perrett et al. 1989
). Neurons in area F5
of the ventral premotor cortex that were related to hand-movement
control (Rizzolatti et al. 1988
) also were activated when the monkey observed the hand action of the experimenter as well as
when the animal itself performed a hand action. They designated these
neurons as "mirror" neurons (Gallese et al. 1996
).
Because we did not record any neurons in area AIP that responded to the hand action of the experimenter, the nonobject-type neurons are considered to be different from the mirror neurons of F5. However, it
is plausible that they receive visual signals of hand actions from area TEa.
The selectivity of the motor-dominant neurons, which are not visually
responsive, is likely to be related to the pattern of active hand
movements per se and not to proprioceptive feedback from the joints of
the hands and fingers. The presence of the highly selective
motor-dominant neurons in area AIP suggests that they may represent
particular patterns of hand movements, which may be appropriate to
grasp particular objects. For the moderately selective motor-dominant
neurons, the overall configuration of the MDS map was similar to that
in the corresponding map for the nonobject-type neurons. The cube,
sphere, and cone were grasped with similar patterns of hand movement
especially when they were small. Slight discrepancy, such as the
clustering of the ring with the cylinder for the motor-dominant but not
the nonobject-type neurons, may be related to the fact that the
nonobject-type neurons could respond to a combined view of the handgrip
and object. However, we need more direct evidence such as the response
to images of the hand on video as used by Iriki et al.
(1996) to clarify the source of the visual responses of
nonobject-type neurons and the difference between the nonobject-type
neurons and the motor dominant neurons.
Orientation selectivity
Arbib (1981) proposed that the perceptual schema of
the orientation of an object in space is linked to the motor schema of hand rotation in his model of the coordinated control program for
visually directed reaching to grasp an object. The hypothesis was
supported by a study demonstrating that patients with parietal lobe
lesions had deficits in adjusting their hand orientation to that of a
target (Perenin and Vighetto 1988
). The results of our
previous experiments were compatible with these results because some
AIP neurons were selective for the axis orientation of the lever switch
that the monkey was grasping (Taira et al. 1990
). Recently, axis-orientation-selective (AOS) neurons that represent the
orientation of the longitudinal axis of objects in depth were found in
area c-IPS that is located in the caudal part of the lateral bank of
the IPS (Kusunoki et al. 1993
; Sakata and Taira 1994
; Sakata et al. 1997a
). Area AIP probably
receives visual input from area c-IPS concerning the axis orientation
of objects (Luppino, personal communication).
A visually agnostic patient with damage in the cortical ventral visual
pathway could not adjust hand orientation to a tilted plate in 3D space
without binocular visual cues (Dijkerman et al. 1996).
This suggests that the parietal cortex may be involved in the
discrimination of surface orientation depending on binocular visual
cues. In area c-IPS, surface-orientation-selective (SOS) neurons also
were found to represent the 3D orientation of flat surfaces
(Sakata et al. 1998
; Shikata et al. 1996
;
Tsutsui et al. 1997
). In the current experiment, we
found many object-type neurons that were selective for the orientation
of a plate and/or ring. These neurons probably receive visual input
from the SOS neurons in area c-IPS. In addition, some neurons in area
AIP were selective for both the shape and orientation of these objects. A similar feature was found in some neurons in area c-IPS
(Sakata et al. 1997b
, 1998
), suggesting that information
regarding the shape is also conveyed from area c-IPS to area AIP.
The nonobject-type or motor-dominant neurons also showed orientation selectivity. As described earlier, the activities of these types of neurons were more related to the pattern of hand movements than to the object shape. The same may be true for the case of grasping an object with different orientation. This orientation-selective activity in the nonobject-type or motor-dominant neurons probably represents the orientation of the hand. Therefore area AIP may play an important role in adjusting hand orientation to the orientation of the 3D objects.
Size selectivity
The majority of object-type neurons tested (16/26) were selective
for the size of objects and often showed selectivity for shape as well
as size, and more than half (9/16) were highly selective. This tendency
was somewhat different from IT neurons because at least half of the IT
neurons tested showed size-independent shape selectivity, although a
considerable number also showed size-selectivity (Ito et al.
1995). Thus most of the size-selective object-type hand-manipulation-related neurons represented the size together with
shape of the objects.
The majority of the nonobject-type neurons tested (7/12) also showed
size selectivity, and almost all of them were also selective for shape.
All the motor-dominant neurons tested were selective for size (8/8),
and almost all of them were also selective for shape. Most of these two
classes of neurons probably represent grip size combined with grip
shape. The maximum grip aperture is influenced by the size of the
objects grasped (Jakobson and Goodale 1991;
Jeannerod 1986
). Patients with lesions in the parietal cortex showed impairment in the control of grip size but no deficit in
the perceptual matching to object size (Jeannerod et al.
1994
). A patient who had damage in the ventral visual pathway
showed impairment in the matching of size but not in the grasping of objects (Goodale et al. 1991
). These results suggest
that visual size information is represented both in the dorsal and
ventral visual pathways for different purposes. In the former, it may play an important role for the control of grip aperture, whereas in the
latter it may be related to the size perception.
It was reported that the estimation of the length of bars was impaired
in patients with parieto-occipital lesions (Von Cramon and
Kerkhoff 1993). In the monkey parietal cortex,
axis-orientation-selective neurons in area c-IPS exhibited increasing
levels of activity with increasing length of the stimuli, suggesting
that rate of discharge is related to the length of the stimuli
(Sakata et al. 1997a
). This finding may correspond to
the gradual change in discharge rate according to the size of the
objects in the case of some size-selective AIP neurons.
Anatomic organization of area AIP
We recorded hand-manipulation-related neurons from the anterior
part of the lateral bank of the IPS, designated as area AIP. We used
somatotopic mapping of the hand area in the SI as landmarks because
area AIP is located immediately posterior to this area (Gallese
et al. 1994; Sakata et al. 1995
). Previously, we
could not clearly define the antero-posterior distribution of
hand-manipulation-related neurons in the lateral bank of the IPS, but
the distribution of these neurons was found to be closer in the present
experiments than in previous ones. The area is more anterior than area
LIP, which is related to saccadic eye movements (Andersen
1989
) and adjacent in depth with area VIP located in the fundus
of the IPS. In area AIP, we did not find any tactile neurons or bimodal
neurons that responded both to visual and somatosensory stimuli as in area VIP (Colby et al. 1993
) or area 7b (Leinonen
et al. 1979
). Thus area AIP is distinct from area VIP and area
7b (Sakata et al. 1995
).
Area AIP has reciprocal connections with area F5 in the ventral
premotor cortex (Matelli et al. 1986), which is related
to visually guided hand movements (Rizzolatti et al.
1988
). The interconnection between areas AIP and F5 was
confirmed further by recent anatomic studies (Gallese et al.
1997
; Luppino et al. 1999
; Matelli et al.
1994
; Rizzolatti et al. 1998
). Results for the
injection of neural tracers into area AIP showed that area AIP also has
some connection with area c-IPS, in which AOS and SOS neurons were found (Luppino, personal communication). The results of these anatomic
studies support our hypothesis that one of the major sources of visual
input to area AIP is area c-IPS. Furthermore some connection is known
to exist between the anterior part of the lateral bank of the IPS
(including area AIP) and area TE (Webster et al. 1994
),
which contains neurons that object shape based on monocular visual cues
(Desimone et al. 1984
; Tanaka et al.
1991
). Thus some of the signals concerning object shape may be
sent from the inferotemporal cortex to area AIP.
Functional role of AIP neurons in the visual guidance of hand manipulation
In the previous study, we proposed a model in which area AIP sends
visual signals of objects to area F5 for selection of the pattern of
hand movements, and area F5 sends back the efference copy of the
selected motor command to area AIP (Sakata et al. 1995).
This feedback loop plays an important role in matching the pattern of
handgrip to the spatial characteristics of objects. The results of the
present study, which show good correspondence in object selectivity of
the object-type visual-motor neurons between the manipulation and
fixation tasks, support this hypothesis.
Recently the functional properties of F5 neurons were studied using the
same apparatus and behavioral tasks as those used in the present study
of AIP neurons (Murata et al. 1997). In general, the
functional properties of the neurons were similar in these two areas;
however, there were no neurons in area F5 that were homologous to the
visual-dominant neurons in area AIP. The motor-dominant neurons were
somewhat more numerous in area F5 than in area AIP, and the
visual-motor neurons were common to both areas. These findings support
the assumption that the object-type visual-motor neurons of area AIP
send trigger signals to F5 and receive converging input from both the
motor- and visual-dominant neurons. Although it previously was proposed
that the visual-dominant neurons receive visual signals from the
prestriate cortex to recognize object structure, it is more plausible
that the 3D shape as well as other features, such as the axis and
surface orientation in depth and volumetric size, are processed in more
posterior parts of the parietal cortex, perhaps in and around area
c-IPS (Sakata et al. 1997a
, 1998
), and the
visual-dominant neurons simply receive these signals. Recently, some
visual neurons in area LIP were found to be selective for 2D shapes
(Sereno and Maunsell 1998
). However, in a recent
psychophysical study, Castiello et al. (1996)
demonstrated that the 2D silhouette of a spherical object elicited
preshaping for a flat object. Therefore it may be concluded that the
shape selectivity of AIP neurons is primarily 3D in nature and relevant to the visual control of hand movement. On the other hand, the selective responses of the nonobject-type visually responsive neurons
appeared to be related to the grip shape, grip size, or hand
orientation and relevant to the visual feedback for adjustment of the
handgrip for an object. However, more direct evidence is required to
support this hypothesis.
![]() |
ACKNOWLEDGMENTS |
---|
We are grateful to Prof. G. Rizzolatti for making many valuable comments on the manuscript.
This study was supported by Grants 04NP0101.11145238 and 1078512 from the Japanese Ministry of Education, Science, Sports and Culture, and a research grant from the Human Frontier Science Program 1993-1995.
![]() |
FOOTNOTES |
---|
Address for reprint requests: A. Murata, Nihon University, School of Medicine, First Dept. of Physiology, Oyaguchi-kamimachi 30-1, Itabashi-ku, Tokyo 173-8610, Japan.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
Received 7 April 1999; accepted in final form 30 December 1999.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|