Evidence for an Eye-Centered Spherical Representation of the Visuomotor Map

Philipp Vetter, Susan J. Goodbody, and Daniel M. Wolpert

Sobell Department of Neurophysiology, Institute of Neurology, London WC1N 3BG United Kingdom


    ABSTRACT
Top
Abstract
Introduction
Methods
Results
Discussion
References

Vetter, Philipp, Susan J. Goodbody, and Daniel M. Wolpert. Evidence for an eye-centered spherical representation of the visuomotor map. During visually guided movement, visual coordinates of target location must be transformed into coordinates appropriate for movement. To investigate the representation of this visuomotor coordinate transformation, we examined changes in pointing behavior induced by a local visuomotor remapping. The visual feedback of finger position was limited to one location within the workspace, at which a discrepancy was introduced between the actual and visually perceived finger position. This remapping induced a change in pointing that extended over the entire workspace and was best captured by a spherical coordinate system centered near the eyes.


    INTRODUCTION
Top
Abstract
Introduction
Methods
Results
Discussion
References

To reach a visually perceived target, the CNS must transform visual information into appropriate motor commands (Andersen et al. 1985; Flanders et al. 1992; Ghilardi et al. 1995; Kalaska and Crammond 1992; Soechting and Flanders 1989). This transformation from visual to motor coordinates is known as the visuomotor map. Plasticity of the visuomotor map is essential, as sensorimotor discrepancies inevitably arise throughout life, for instance due to body growth (Held 1965; Howard 1982). This plasticity has been studied extensively, demonstrating the remarkable ability of the visuomotor map to adapt, at least partially, to a wide variety of stable remappings (for a review, see Welch 1986).

To assess the natural coordinate system of the visuomotor map, we have used a paradigm in which subjects were exposed to a single novel visuomotor (visuoproprioceptive) pairing. Such a single-point remapping can be captured by a shift in almost any coordinate system. However, the pattern of generalization, that is the change in pointing at other points in the workspace, will be determined by the particular coordinate system in which the visuomotor map is represented. In contrast, previous studies of visuomotor adaptation generally have used prisms to alter the visuomotor map over a large region of the workspace. This is equivalent to providing a set of training data in the form of many visuoproprioceptive pairs. From such studies it is difficult to infer the natural coordinate system of the map as the set of visuoproprioceptive pairs experienced may be in conflict with the visuomotor map's natural coordinate system, leading to an ambiguous adaptation.

We compared predicted and actual changes in pointing after such a single-point remapping based on five a priori hypotheses of the coordinate system of the visuomotor map: Cartesian coordinates based at the shoulder and eye, and spherical coordinates based on both shoulder and eye and joint-based coordinates. This work builds on previous studies of spatial generalization in one (Bedford 1989, 1993b) and two dimensions (Ghahramani et al. 1996), suggesting a Cartesian coordinate system, and generalization in the velocity domain suggesting a decay of adaptation at novel velocities (Kitazawa et al. 1997). The present study used pointing in three-dimensional space, which allowed natural pointing movements and joint angle measurements.


    METHODS
Top
Abstract
Introduction
Methods
Results
Discussion
References

Subjects

Eight right-handed subjects (5 men; 3 women; ages 21-33) gave their informed consent and participated in the study. Subjects were naive to the purpose of the experiment. They participated in a remapping and a control session on separate days in a balanced order. Control sessions were identical to the remapping session except that no visuomotor remapping was introduced.

Apparatus

A schematic of the setup is shown in Fig. 1. The subject's arm position was monitored with infrared emitting diodes (IREDs) the positions of which were detected by an Optotrak 3020 motion analysis system (Northern Digital, Waterloo, Ontario) at 90 Hz. 18 IREDs were mounted on three rigid bodies (RB) placed on the subject's fingertip (8), forearm (6), and upper arm (4). To measure joint angles, the center of the shoulder rotation (shoulder position) was determined by pivoting the elbow around a fixed shoulder and calculating the point relative to the upper arm RB whose positional variance in Cartesian space was minimal. The elbow position was determined by rotating the upper arm and forearm and calculating the point relative to the upper arm RB whose positional variance relative to the forearm RB was minimal, that is, the elbow's center of rotation. Joint angles were calculated from the two 4 × 4 homogeneous transformation matrices, which define the position and orientation of the upper arm and forearm RBs. From the orientation of the upper RB, the joint angles gamma , beta , alpha , which represent successive rotations of the upper arm about fixed Cartesian x, y, and z axes, respectively, were calculated (see Fig. 1). The zero angular position for the upper arm was taken as the upper arm pointing downward aligned with the vertical z axis and the forearm pointing along the positive y axis when the elbow was bent to 90°. The elbow angle phi  was the angle between the forearm and the upper arm and was calculated from the relative orientations of the upper arm and forearm RBs.



View larger version (38K):
[in this window]
[in a new window]
 
Fig. 1. Apparatus for measuring unconstrained 3-dimensional arm movements under 3-dimensional virtual visual feedback. Looking down at the mirror through field sequential glasses, the subject sees virtual images of the finger and targets, which are projected onto the rear projection screen from the computer. Shuttered glasses alternately blanked the view from each eye in synchrony with the display, allowing each eye to be presented with the appropriate planar view---subjects therefore perceived a 3-dimensional scene. Shaded area shows the workspace in which the targets appeared. The exposure target, at which a visuomotor remapping was introduced, is shown.

A three-dimensional virtual visual feedback setup was used to overlay images on to the arm's workspace (for details of this setup, see Goodbody and Wolpert 1998). The images were generated with the OpenGL graphics package, which uses projective geometry to adjust the size of the image appropriately with its distance and angle from the eye. Therefore the subject sees a perspective view in which the size of the object on the retina reduces as the object moves further away. The system was calibrated for each subject as the perspective algorithm depends on the subject's interocular distance. Subjects could not see their arm but were shown their finger location as a 1 cm green cube. A computer controlled discrepancy between the finger and cube position could be introduced. Targets were displayed as red, 4 mm radius spheres.

Procedure

Each session had four phases---familiarization, preexposure, exposure, and postexposure---interspersed with rest periods every 50 movements. Pointing movements were made to 36 targets in the three-dimensional workspace. One of these targets was the exposure target, at which the visuomotor remapping was introduced (Fig. 1).

Each trial consisted of a pointing movement to one of the targets. The trial started when the finger moved behind a notional fronto-parallel plane 12.2 cm in front of the eyes, at which time a target appeared. Subjects were asked to assume a similar starting position with the finger close to their midline.

In the familiarization phase, subjects pointed to targets with continuous veridical feedback of their finger position. The exposure target was presented 12 times, and all the other targets were presented twice in a pseudorandom order. In the preexposure phase, subjects' pointing errors were assessed before the remapping. Subjects pointed to targets without visual feedback of their finger location. The exposure target was presented 18 times and all other targets 3 times in a pseudorandom order. Each trial ended when the subject's finger velocity dropped below 1 cm/s.

In the exposure phase, subjects repeatedly pointed to the exposure target (x = 8.1 cm, y = 36.2 cm, z = -27.6 cm; origin between the eyes) 50 times. During this phase, a visuomotor remapping was introduced, tailored for each subject based on their average preexposure pointing position at the exposure target. This ensured a similar remapping for all subjects independent of their preexposure pointing biases. The remapping required subjects to point 6 cm to the right (positive x) of this average position to perceive their finger on target. The location and direction of the remapping were chosen so as to maximize the differences in the predictions of the hypotheses tested. To limit the visuoproprioceptive exposure, visual feedback of finger position (green cube) was only displayed when subjects were within 3 cm of the target. The trial ended when subjects had held their finger on target continuously for 2 s. The remapping was introduced gradually in the exposure phase, incrementing on each trial so that the full perturbation was present on trial 17. In the control condition, visual feedback was altered so that subjects had to point to their average preexposure position to see their finger on target.

In the postexposure phase, the changes in pointing due to the exposure phase was assessed. As in the preexposure phase, subjects pointed to the targets without visual feedback. The exposure target was presented 18 times and all other targets 3 times in a pseudorandom order. To prevent any decay of learning, an exposure trial was presented after every three trials.

Analysis

For each subject and target, average pre- and postexposure pointing positions were calculated. The difference between pre- and postexposure represented the generalization of the remapping over the workspace. These changes were compared with predictions based on the five hypotheses about the natural coordinate system of the visuomotor map. The first was a Cartesian coordinate system with a fixed origin between the eyes. Second, a Cartesian coordinate system with origin at the shoulder was considered. This coordinate system differs from the eye-centered system as the eye is fixed in external space whereas the shoulder is free to move by several cm. Thus the Cartesian shoulder coordinate system represents the finger position relative to the shoulder. Third, a spherical coordinate system centered about the eyes (r, phi , theta ) was tested, in which r, phi , and theta  represent distance, azimuth, and elevation, respectively. Fourth, a spherical coordinate system with the origin at the average shoulder position was considered. Finally a joint-based coordinate system was examined (see Apparatus for joint-angle definition).

For each point, a vector omega  was calculated representing the location of the finger in a particular coordinate system. For each hypothesis, the observed change at the exposure target is domega  = omega postexposure - omega preexposure. For each hypothesis and nonexposure target, predictions were made by adding domega to the preexposure pointing coordinates: omega prediction omega preexposure + domega and then transforming all the predictions into Cartesian space. Thus the change in pointing at the exposure target created a single, global offset in the coordinate system (e.g., domega  = (dr, dphi , dtheta ) in spherical coordinates). In other words, to predict the change in pointing, the offset calculated from the exposure target was added to all preexposure pointing coordinates. For spherical coordinates, we also examined the possibility that the distance r was altered by a gain (k) change mechanism, such that rprediction = k × rpreexposure.

The prediction error for each target was calculated as the magnitude of the vector difference between the predicted and actual changes in pointing. Average preexposure and postexposure positions were used to calculate the actual change in pointing. A repeated measure analysis of variance was performed on the prediction errors as a function of hypothesis and target number.


    RESULTS
Top
Abstract
Introduction
Methods
Results
Discussion
References

Changes in pointing between the pre- and postexposure phases in the control condition were not significant along any of the Cartesian coordinate axes (Fig. 2A). However, remapping of a single point in space induced significant changes in pointing over the whole workspace (Fig. 2B). The changes were significant along the x and y but not the z axis (2-tailed t-test; P values = x: 0.0001, y: 0.0011, z: 0.41). On questioning, only one subject suspected a remapping during the experimental condition. The pattern of generalization resembled a colinear shift in the coronal plane (xz), whereas in the horizontal plane (xy) changes looked rotational. The average magnitude of change in pointing was 4.54 ± 0.32 cm (mean ± SE) and did not decay significantly with distance from the exposure point (Fig. 3A).



View larger version (33K):
[in this window]
[in a new window]
 
Fig. 2. Change in pointing behavior between pre- and postexposure phases in the control (A) and remapping conditions (B; pooled data from all subjects). Plots are 2-dimensional slices through the workspace with the orientation of each row indicated by the schematic of the subject. right-arrow, change in pointing, with the arrow pointing from the preexposure to the postexposure position. Error ellipses show 95% confidence limits for the change in pointing at each target. , veridical target positions in the control condition. , exposure target. Coronal slices (top plots at fixed y) and horizontal slices (bottom plots at fixed z) are shown. Note that for the y values that are split (e.g., 22.2/16.2), the larger value is for the bottom row of targets.



View larger version (28K):
[in this window]
[in a new window]
 
Fig. 3. A: magnitude of change in pointing with distance from the exposure target. B: relative decrement of the mean prediction errors (±SE) over the hypothesis based on spherical coordinates around the eyes. C: comparison of predicted changes in pointing based on spherical coordinates around the eyes (gray) with actual changes (black) in the same format as Fig. 2. Predictions are based on the change in pointing at the exposure target (highlighted by a square). Pooled data from all subjects are shown.

The actual changes in pointing were best predicted by spherical coordinates centered around the eyes (Fig. 3B). These predictions were significantly better than spherical coordinates about the shoulder (P < 0.05), joint angles (P < 0.05), and Cartesian coordinates (P < 0.001) as well as the hypothesis that there is no generalization (P < 0.05). The absolute prediction errors for each subject and hypothesis are summarized in Table 1. This shows that spherical coordinates about the eye produced the best prediction for five of the eight subjects and produced the second best prediction for the three remaining subjects. As shown in Fig. 3C, spherical coordinates about the eyes captured the pattern of changes in the pointing observed. An analysis of the prediction errors for the spherical coordinate system about the eyes (that is the vector differences between the black and gray arrows of Fig. 3C) showed no obvious trends and in particular showed no correlation along any of the Cartesian axes (P > 0.05). The predictions made by the hypothesis of a scaling of the distance (r) component of the spherical coordinate, as opposed to a single offset, were systematically worse than for the single offset hypothesis (data not shown).


                              
View this table:
[in this window]
[in a new window]
 
Table 1. Average prediction errors for individual subjects and hypotheses


    DISCUSSION
Top
Abstract
Introduction
Methods
Results
Discussion
References

A three-dimensional virtual reality setup was used to expose subjects to a highly localized remapping between actual and displayed finger position. This induced significant changes in subjects' pointing behavior over the entire workspace which did not decay significantly with distance from the remapped location. Several hypotheses as to the natural coordinate system of the visuomotor map were tested by comparing predicted changes in pointing with actual changes. The hypothesis of spherical coordinates with the origin at the eyes best captured the observed changes. These predictions were significantly better than those based on spherical coordinates about the shoulder, joint angle coordinates, or Cartesian coordinates.

Our results are consistent with Bedford's (1989, 1993a,b) findings that changes in pointing did not decay with distance from the remapping and were approximately linear along a fixed radius of arc. Ghahramani et al. (1996) found a decaying pattern of generalization in their planar two-dimensional study. Their study was limited in two important respects that could account for these differences. First they had no control over the starting position of the hand, a factor that is thought to exert an influence over the visuomotor remapping. In our study, subjects were confined to executing movements from a limited region of space in front of their body. Second, the nature of their apparatus constrained the subject to make unnatural pointing movements, forcing subjects to point at the height of their shoulder.

Previous generalization studies focusing on movement dynamics have found joint-based generalization (Shadmehr and Mussa-Ivaldi 1994). However, we show in this study that for the visuomotor map, the natural coordinate system is not joint-based. Imamizu et al. (1995) examined pointing behavior with a 75° rotatory remapping and, consistent with our data, showed that subjects learned the rotation for movements in one direction and generalized this to movements in other directions.

Flanders et al. (1992) had subjects perform targeted arm movements to remembered positions of virtual targets in three-dimensional space. They suggested that retinocentric coordinates gradually evolve through head-centered to become shoulder-centered coordinates (Flanders et al. 1992; Soechting et al. 1990). McIntyre et al. (1997) found evidence for an eye-centered frame of reference by analyzing variable errors and constant errors in a three-dimensional pointing task to remembered positions with visual feedback of the finger position. This finding was independent of the hand used, its starting position, and head orientation. In a pointing task without visual feedback of finger position, Baud-Bovy and Viviani (1998) found evidence for a representation in spherical coordinates by analyzing the variable errors. Our results show that the process of visuomotor learning also has a natural coordinate system based on spherical coordinates centered near the eyes.

A neurophysiological study in monkey by Lacquaniti et al. (1995) suggests that the superior parietal lobule (Brodmann area 5) might represent a neural substrate for an ego-centric spherical representation of reaching to a visual target. Analysis of electrical discharge of parietal neurons during three-dimensional reaching revealed a specific neural tuning along the distance, azimuth, and elevation axes. Both shoulder- and eye-centered spherical frames fit the neural data, but the eye-centered frame fitted slightly better.

In conclusion, by studying a highly limited visuomotor remapping, we could examine the natural coordinate system of the visuomotor map under natural pointing movements in three-dimensional space. On the basis of a comparison of the prediction of several a priori hypotheses, we have determined that the pattern of generalization seen is best captured by a spherical coordinate system centered near the eyes.


    ACKNOWLEDGMENTS

This project was supported by the Wellcome Trust, the Biotechnology and Biological Sciences Research Council, and the Royal Society. P. Vetter is funded by the Wellcome four-year PhD Program in Neuroscience at University College London.


    FOOTNOTES

Address for reprint requests: D. M. Wolpert, Sobell Dept. of Neurophysiology, Institute of Neurology, Queen Square, London WC1N 3BG, UK.

1 The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Received 20 July, 1998; accepted in final form 3 November 1998.


    REFERENCES
Top
Abstract
Introduction
Methods
Results
Discussion
References

0022-3077/99 $5.00 Copyright © 1999 The American Physiological Society