Gaze Anchoring to a Pointing Target Is Present During the Entire Pointing Movement and Is Driven by a Non-Visual Signal

S.F.W. Neggers1,3 and H. Bekkering2,3

 1Department of Psychonomics, Helmholtz Institute, Utrecht University, 3584 CS Utrecht;  2Department of Experimental and Work Psychology, University of Groningen, 9712 TS Groningen, The Netherlands; and  3Department of Cognition and Action, Max-Planck-Institute for Psychological Research, 80802 Munich, Germany


    ABSTRACT
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Neggers, S.F.W. and H. Bekkering. Gaze Anchoring to a Pointing Target Is Present During the Entire Pointing Movement and Is Driven by a Non-Visual Signal. J. Neurophysiol. 86: 961-970, 2001. A well-coordinated pattern of eye and hand movements can be observed during goal-directed arm movements. Typically, a saccadic eye movement precedes the arm movement, and its occurrence is temporally correlated with the start of the arm movement. Furthermore, the coupling of gaze and aiming movements is also observable after pointing initiation. It has recently been observed that saccades cannot be directed to new target stimuli, away from a pointing target stimulus. Saccades directed to targets presented during the final phase of a pointing movement were delayed until after pointing movement offset ("gaze anchoring"). The present study investigated whether ocular gaze is anchored to a pointing target during the entire pointing movement. In experiment 1, new targets were presented at various times during the duration of a pointing movement, triggered by the kinematics arm moment itself (movement onset, peak acceleration/velocity/deceleration, and offset). Subjects had to make a saccade to the new target as fast as possible while maintaining the pointing movement to the initial target. Saccadic latencies were increased by an amount of time that approximately equaled the remaining pointing time after saccadic target presentation, with the majority of saccades executed after pointing movement offset. The nature of the signal driving gaze stabilization during pointing was investigated in experiment 2. In previous experiments where ocular gaze was anchored to a pointing target, subjects could always see their moving arm, thus it was unknown whether a visual image of the moving arm, an afferent (proprioceptive) signal or an efferent (motor control related) signal produced gaze anchoring. In experiment 2 subjects had to point with or without vision of the moving arm to test whether a visual signal is used to anchor gaze to a pointing target. Results indicate that gaze anchoring was also observed without vision of the moving arm. The findings support the existence of a mechanism enforcing ocular gaze anchoring during the entire duration of a pointing movement. Moreover, such a mechanism uses an internally generated, or proprioceptive, nonvisual signal. Possible neural substrates underlying these processes are discussed, as well as the role of selective attention.


    INTRODUCTION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

To interact effectively with objects in the everyday world, humans frequently make combined eye and hand movements. The control of these movements is not independent. Typically, saccades (rapid ballistic eye movements) bring the new area of interest to the fovea, the central area of the retina where visual acuity is high, shortly before a pointing movement to the area (or target) is initiated (Prablanc et al. 1979). The reaction times of saccadic and manual movements toward visual targets are known to be moderately correlated on a trial-by-trial basis (Bekkering et al. 1995; Frens and Erkelens 1991; Gielen et al. 1984) and somewhat higher for somatosensory targets (Neggers and Bekkering 1999) or auditory targets (Mather and Fisk 1985). Pointing accuracy suffers when participants do not foveate to a target during the homing-in phase of aiming movements (Bekkering et al. 1995; Neggers and Bekkering 1999; Prablanc et al. 1979; Vercher et al. 1994). Also, when several visual targets are presented, eye and hand movements usually move to the same target (Gielen et al. 1984). Thus the CNS apparently enforces a co-alignment for the ocular and manual motor systems in space and time.

Whereas coordinative behavior of saccades and pointing initiation has been the subject of many studies, the behavior of the saccadic system during pointing has not received much attention. Recently, we reported that sacades away from the fixated pointing target could not be made to new targets during the ongoing hand movement when the new targets were offered shortly after the arm movement reached peak velocity (Neggers and Bekkering 2000). Saccades were severely delayed so that they occurred shortly after the hand reached the fixated target. Apparently the CNS enforces ocular fixation of a pointing target, or inhibits secondary saccades away from a pointing target, until pointing is completed, an effect that will be referred to as "gaze anchoring" in the present study. The saccadic delay correlated on a trial-to-trial basis with the duration of the pointing movements, suggesting that the execution of an arm movement directly causes gaze anchoring.

Interestingly, other studies using a double-step paradigm, where the participant's eyes and hand followed a target that jumped to another position during pointing, reported corrective saccades to the new target position that appeared to occur with close-to-normal reaction times (Goodale et al. 1986; Lunenburger et al. 2000; Prablanc and Martin 1992). The eye and hand movement traces in the mentioned reports clearly show saccades that are not delayed much and occur during the arm movement. The main difference between the double-step paradigm and our previous study (Neggers and Bekkering 2000) is that in our study subjects had to maintain pointing to the initial target, whereas subjects had to point to the new target in the previously mentioned studies. Thus saccades had to be executed away from a pointing target in our previous study, and not toward it, as in the other double-step experiments. The findings are therefore not contradictory, but can be interpreted as two manifestations of the same coordinative mechanism of the eyes and arm, forcing the eyes to be directed to the target of an arm movement. Such a mechanism would guarantee a stable visual target representation for arm movements with respect to the torso during pointing, undisturbed by saccades.

Arm movements rely on a shoulder-centered target representation (Baud-Bovy and Viviani 1998; Flanders et al. 1992; Soechting et al. 1996), which is suggested to be constructed by the superposition of retinal signals with extraretinal eye position (Abrams et al. 1990; Gentilucci et al. 1997; Hallett and Lightstone 1976; Mays and Sparks 1980), in addition to head position signals when the head is not fixed. A saccade may disrupt such an internal target representation by changing the eye position signal; however, after a short interval, the internal target representation could be updated by combining the new retinal image with the new eye position signal. Evidence for such processes underlying spatial representations is provided by the "phantom array" illusion (Jordan and Hershberger 1994), in which subjects see a trace of dots shortly before their eyes move, although in actuality a stationary stimulus was blinking during the interval between target presentation and saccade execution. These results suggest that the "update" process may already be started even before the eye movement is executed, with an amount that equals the desired eye displacement. Many visual areas in the primate brain are known to reflect such spatial updating processes prior to eye movements. Interestingly, it has been found that prior to saccades, neurons in the lateral intraparietal sulcus (LIP) show a remapping of the visual space that predict the information that will fall on their receptive fields after the saccade (Colby et al. 1995; Duhamel et al. 1992), which is suggested to be based on an efference copy of the saccadic signal. To prevent distortions during pointing and obtain foveal vision of the pointing target, co-alignment of the gaze and the pointing target would be desirable for the whole duration of an arm movement. Furthermore, it has been found that the visuomotor space for pointing movements is systematically distorted in a retinotopic fashion when gazing at eccentric locations one does not point to, resulting in relatively large pointing errors reflecting large misalignments of ocular gaze and pointing targets (Henriques and Crawford 2000). Apparently, an optimal strategy for pointing is to align ocular gaze to the place where the hand moves to.

The present study is designed to investigate the mechanisms underlying saccadic inhibition during pointing movements in more detail. We previously demonstrated (Neggers and Bekkering 2000) the occurrence of gaze anchoring during the final phase of a pointing movement (from pointing peak velocity to pointing movement offset). In experiment 1 of the present study, new saccadic targets are presented at the time of hand movement initiation, maximum acceleration, peak velocity, maximum deceleration, and at movement offset. Participants were instructed to make a saccade to the new target as quickly as possible, while continuing the pointing movement to the initial target. It is investigated whether saccades toward the new targets are delayed during the whole pointing movement or only during the final stages of the pointing movement.

Importantly, in our previous experiment reporting gaze anchoring during pointing movements, subjects could see their arms (Neggers and Bekkering 2000). Therefore gaze anchoring during pointing could result from a mechanism using one of three signals carrying information about the movement of the arm. One mechanism could use a retinal image of the moving arm. An alternative mechanism could use an internally generated efferent signal directly related to arm movement control. A third candidate providing information on arm movement status could be an afferent signal carrying information on the dynamic status of the arm. The latter signals can be provided by proprioceptive receptors in the muscle spindles and by tactile stimulation of receptors in the skin (Jones 1994; McCloskey 1978). To determine whether a visual, efferent, or proprioceptive signal drives gaze anchoring during pointing, a second experiment was conducted, where vision of the moving arm was manipulated by simply switching off the illumination in the room in some blocks of trials, and leaving it on during other blocks. Similar as in experiment 1, saccadic targets were presented at arm movement initiation, maximum acceleration, peak velocity, maximum deceleration, and at movement offset. If vision of the moving arm is the principal signal used to drive gaze anchoring during pointing, no saccadic delay would be expected during pointing without vision of the moving arm. However, if either an efferent signal related to motor control or an afferent proprioceptive signal inhibits the ability of the eyes to saccade away during pointing, a similar delay of saccadic eye movements is expected regardless of visibility of the hand.


    METHODS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Apparatus

Pointing and saccadic targets were produced by illuminating light-emitting diodes (LEDs) fixed in a table (see Fig. 1A). The LED board contained a matrix of 12 × 8 two-colored LEDs. In the horizontal position, the surface of the LED board was 79 cm above floor level and could be tilted out of the horizontal position along the x-axis (from the left to the right side, perpendicular to the long side of the board). To establish a frontal view, the LED-board surface was tilted 20° out of the horizontal plane during the experiments. The board's position was recorded and calibrated with the hand movement data for analysis (see next paragraph). A black sheet of paper covered the LED-matrix, with circular holes of 2 mm diam above each LED. The LEDs radiated red (660 nm) or green (565 nm) light with a luminance of 80 and 55 cd/m2, respectively. The voltage over an illuminated LED was 5.0 V. The movements of both eyes were tracked at a rate of 250 Hz with the head-mounted EyeLink system (Sensomotoric Instruments) using infrared (IR) cameras. An additional camera mounted on the headband of the EyeLink helmet measured head position by tracking the position of four IR LEDS that were attached to the corners of the LED board (see Fig. 1A). Head position and the angles of the eyes in the head allowed for the calculation of the ocular gaze coordinates on the LED board in millimeters. The viewing distance of the LED board relative to the head was approximately 600 mm but was not restrained. A tube with a narrow elongated tip was placed over the participant's index finger and served as a pointing device (see Fig. 1B). Three IR markers were attached to the pointing device and tracked at 50 Hz by the OPTOTRAK system (Northern Digital).



View larger version (23K):
[in this window]
[in a new window]
 
Fig. 1. The experimental setup with the light-emitting diode (LED) table and the infrared tracking devices OPTOTRAK from Northern Digital for measuring 3-dimensional (3D) arm movements and the EyeLink System from SMI for measuring eye movements. A pointing device (a narrow tube with a sharp tip) with 3 OPTOTRAK infrared markers (IREDs) attached to it was placed over the participant's index finger.

For spatial analyses, the coordinates of the pointing device's tip were calculated from the measured three-dimensional (3D) positions of the three markers attached to the tube. To measure the orientation of the LED board in space, three OPTOTRAK IR markers were attached to the front of the board. Both ocular gaze and hand position could therefore be measured in a coordinate frame fixed to the LED board, with the origin at the lower left corner. A dimmed spotlight attached to the ceiling illuminated the room. The luminance reflecting from the table surface as measured from the participant's viewing angle, was approximately 2 cd/m2.

Participants

Eighteen healthy, right-handed, participants took part in the experiments (9 in experiment 1 and 9 in experiment 2). All had normal or corrected-to-normal vision, and were mostly college students or doctoral students. Most of the participants had prior experience participating in behavioral experiments. All participants were selected from the age range 18-32 yr. Participants were informed beforehand about the experimental procedures and were paid 6.14 EURO/h. Subjects gave informed consent, and the procedures were approved by the Max-Planck-Society.

Procedure

EXPERIMENT 1. A trial started when the participants fixated and touched the lit central fixation point that was situated at a central location on the board, approximately (since head nor trunk were fixated, and could move freely) at the body midline. A red target appeared 2,300 ms after the eyes fixated the central fixation point (always 100 or 200 mm to the left of the initial fixation point, see Fig. 2), and participants had to initiate a saccade and a pointing movement to that target as fast as possible. After participants visually fixated on the target and during the pointing movement, a green target appeared on each trial (except when pointing was too slow, see the real-time data analysis, paragraph below). The time of the second target appearance was triggered by the onset, maximum acceleration, peak velocity, maximum deceleration, or the offset of the hand movement (see Fig. 2 for an example). Such onset times of the second target were chosen to establish a full coverage of the movement time of the arm, and not meant to investigate specific characteristics of kinematic triggering of eye movements during pointing, since the primary objective was to investigate whether saccades are locked to the target of a pointing movement during the entire movement time. The five target onset times occur during the different phases of a bell-shaped movement and are appropriately spaced throughout the movement.



View larger version (12K):
[in this window]
[in a new window]
 
Fig. 2. The pointing movement and the moments the onset of the 2nd target was triggered during pointing. The 2nd target in this example appears at 100 mm distance from the 1st, pointing amplitudes were either 100 or 200 mm. ON, pointing movement onset; amax, maximum acceleration; upsilon max, maximum velocity; amin, minimum acceleration; OFF, movement offset; F, initial fixation point.

The green target appeared either 100 mm to the left or 100 mm above the red target on which the eyes were fixating and the hand was moving toward, thus participants could not anticipate with 100% reliability were the new target would appear. Participants were instructed to execute a saccade to the green target as quickly as possible, while the pointing movement should stay directed toward the red, initial target. Accuracy was emphasized, although movement speed had to be moderate (>1 m/s) to detect peak velocity. Trials in which the velocity was too low, resulting in the nonappearance of the second target, were excluded from analysis (see the triggering paragraph below). Twenty practice trials were presented before the data collection started.

Pointing amplitudes (100 or 200 mm), saccadic directions (away from the fixation target either upward or leftward), and the five trigger types were randomized throughout a block. Participants made pointing movements and saccades in four blocks of trials, with each block containing 80 trials [5 (Trigger) × 2 (amplitude) × 2 (saccade direction) conditions, 4 repeats], resulting in a total of 320 trials per subject.

EXPERIMENT 2. In two out of four blocks, the room illumination was the same as in experiment 1, thus exactly the same paradigm was used. During the other two blocks of trials, however, room illumination was switched off, and participants executed pointing movements and saccades in total darkness (except for the lit targets). To compensate for the high contrast between the lit stimulus and background in a darkened room, the intensity of the LEDs was reduced (4.5 V over the LEDs instead of 5.0 V) such that two test participants performed saccades with latencies that were identical in dark and in light.

Real-time data analysis

The following procedure was used to determine movement kinematics in real-time, that is, during the on-line recording of the pointing movement. As described above in the procedure paragraph, the occurrence of a kinematic parameter (e.g., peak velocity) triggered the onset of the second target to saccade to. Onset times of the second target were as follows: movement onset (ON), the maximum acceleration (amax), the maximum velocity (upsilon max), the minimum acceleration (or maximum deceleration; amin), and movement offset (OFF). The on-line procedure resulted in approximate estimates of the kinematic moments (see 1st paragraph of RESULTS), since due to buffering requirements in real-time movement processing, the sample frequency of our OPTOTRAK system could not exceed 50 Hz for the current experiments.

Velocity and acceleration were calculated as follows. A regression line was fitted through xt, t coordinate pairs of the most recent four position samples {xt, xt-1, xt-2, xt-3} sampled with an interval of 20 ms. The slope of that line was taken as a measure for (dx/dt)t, the velocity at time t, or upsilon t. Because this slope is an estimate of the velocity at a time between t - 1 and t - 2, a delay of 1.5 sample time, or 30 ms, was induced at this level. To determine (d2x/dt2)t, the acceleration at time t, or at, the slope of the line through the upsilon t, t coordinate pairs of the four most recently calculated velocity samples {upsilon t, upsilon t-1, upsilon t-2, upsilon t-3} was determined. Again, a delay of about 30 ms was induced by this estimate, together resulting in a delay of 60 ms with respect to the x-coordinate samples. Accordingly, the change in acceleration with respect to time (d3x/dt3)t, was estimated by the regression line through {at, at-1, at-2, at-3}, resulting in a total delay of 90 ms. Movement onset time was defined at the time t where the value of upsilon t became larger than 0.05 m/s for the first time after pointing target onset, and movement offset as the time when the value of upsilon t was below 0.05 m/s again. The peak velocity was defined as the time t where -0.01 m/s2 < at < 0.01 m/s2 (close to zero = reversal point for upsilon t) and upsilon t > 1 m/s. The maximum acceleration was defined as the moment in time where -0.001 < (d3x/dt3)t < 0.001 m/s3 and at > 0.5 m/s2. The minimum acceleration is defined as the moment in time where -0.001 m/s3 < (d3x/dt3)t < 0.001 m/s3 and at < -0.5 m/s2. Finally, the five times of occurrence of the mentioned kinematic properties had to occur successively, first movement onset, then amax, upsilon max, amin, and movement offset. In other words, the real-time detection algorithm of a kinematic property was first activated after the previous one occurred. For example, when peak velocity was detected, the algorithm for detecting peak deceleration was applied in real-time.

Off-line data analysis

To analyze performance, a number of movement parameters were calculated using algorithms, which were developed in our lab (MATLAB 5.3 scripts). The tangential velocity was calculated along the three-dimensional trajectory for hand movements, and a velocity threshold of 0.05 m/s was used to detect hand movement onset and offset. Movement onset was defined as the first sample time the tangential velocity exceeded the threshold after stimulus onset, and movement offset was defined as the first sample time after movement onset where the velocity was lower than the threshold. Saccadic onset was calculated as the sample time when the ocular velocity, measured in degrees relative to the straight-ahead gaze (in head-referenced coordinates), exceeded 35°/s and the ocular acceleration exceeded 9,500°/s2; saccadic offset was defined as the sample time when the saccadic velocity and deceleration dropped below these thresholds. The latencies of saccades were defined as the time between the second target presentation and onset of the second saccade. Accordingly, the remaining pointing time (the time the 2nd target was present during pointing) was defined as the time between second target onset and pointing movement offset. The more accurate kinematic moments (e.g., peak velocity) that could be calculated off-line (i.e., after the experiment) were not used to determine the reaction and movement times since they did not correspond to the actual target presentations. The differences between the "true" kinematics (as calculated off-line) and the actual real-time triggers were analyzed (see RESULTS).

Pointing deceleration time was defined as the time from peak velocity to movement offset. Pointing accuracy was defined as the Euclidian distance between the position of the tip of the pointing device and position of the target.

Statistics on the movement parameters was performed using a two-way ANOVA with repeated measures. Trials with saccadic latencies less than 100 ms or larger than 700 ms were excluded from analysis. In a few trials, the real-time detection algorithm could not detect a dynamic property, primarily due to a pointing velocity that was too slow to fulfill the minimum velocity criterion. The percentage of analyzed trials always exceeded 60% of total trials for all participants, and at least 10 repeated measures per subject per condition (of a maximum 16 in case of 100% analyzable trials).


    RESULTS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Experiment 1

PERFORMANCE OF THE REAL-TIME DETECTION ALGORITHM. The mean velocity profile of hand movements across all subjects as well as the "true" (as calculated off-line) and "actual" (as calculated in real-time) times of the kinematic properties of the hand movement that were used to trigger second target (ON, amax, upsilon max, amin, OFF) are presented in Fig. 3. A thick line marks where the kinematic landmark occurred as calculated off-line from the velocity and acceleration traces (after 2nd-order Butterworth filtering, with a cutoff frequency of 100 Hz), which can be considered the true time of occurrence. A thin line denotes the moment the second targets were actually presented after the real-time detection algorithm signaled that the kinematic property had occurred. The delay between both lines is the accumulative delay caused by the algorithm (see METHODS section) and other soft- and hardware-related delays in the laboratory. Saccadic reaction times (RTs) were calculated as the time between the actual target onset and saccade initiation.



View larger version (19K):
[in this window]
[in a new window]
 
Fig. 3. Mean velocity profile averaged across all subjects. Thick vertical lines indicate the "true" times of occurrence of the kinematic properties ON, amax, upsilon max, amin, and OFF as calculated off-line, after Butterworth filtering. Thin vertical lines reflect the times the 2nd targets actually appeared during the experiments, with the software- and hardware-related delays resulting from the real-time detection algorithm.

TRACES OF TYPICAL TRIALS. The traces from five typical pointing and eye movement trials plotted against time for each of the saccadic target presentations are presented in Fig. 4. In this example, the pointing target was 200 mm to the left of the fixation dot, and the second target was 100 mm to the left of the first target. Only the x-coordinates are shown, as only minute movements were made in the y-direction. It is evident that the second saccade is delayed until pointing is completed for all trigger conditions, a finding that is typical of the overall behavior.



View larger version (19K):
[in this window]
[in a new window]
 
Fig. 4. Single trial movement traces of eye (solid gray lines) and hand (solid black lines) movements (x-coordinates) against time are shown, one panel for each trigger condition. The 2nd saccade was evoked by a stimulus presented at (from top to bottom): ON, amax, upsilon max, amin, and OFF. It is evident that 2nd saccades were executed only after the pointing movement had been terminated. Dotted vertical lines denote stimulus onset.

SACCADIC REACTION TIMES. Saccadic RTs decreased as a function of the successive trigger types used to elicit an eye movement during pointing, F(32,4) = 49.0; P < 0.0001. Importantly, saccades were, in general, delayed until pointing was over, even when the saccadic target was offered at pointing movement onset. Figure 5A depicts the average saccadic latencies for nine participants, and in Fig. 5B are presented all single-trial values of saccadic RT and remaining pointing time (with respect to target onset) depicted as coordinate pairs, pooled for all conditions and subjects. The delay of second saccades was largest when the target was offered early during pointing, and smallest when offered at pointing movement offset. The latter condition elicited saccades with close-to-normal latencies, presumably since no arm movement was executed at that moment.



View larger version (27K):
[in this window]
[in a new window]
 
Fig. 5. A: mean saccadic initiation times for experiment 1. Saccadic reaction times (RTs; open circle ) and the time from 2nd target onset to pointing movement offset () for each of the kinematic pointing movement properties that triggered stimulus onset during pointing: ON, amax, upsilon max, amin, and OFF. Left panels show data for horizontal saccades, right panels for vertical saccades. Top panels show data for 200-mm trials, bottom panels for 100-mm movement trials. Error bars denote 2 times standard deviation (from top to bottom). B: the coordinate pairs (remaining pointing time after 2nd target onset, saccadic RT) for all trials pooled across all participants. Negative values on the x-axis denote that the target was offered shortly after pointing movement offset. Most values (85.1%) lie above the line y = x solid line, implying that saccades are usually delayed until after pointing offset. RTs suddenly start to increase along the line y = x after t = 170 ms, and are mainly constant prior to 170 ms. The function f(t) (see Eq. 1), with a constant value before this point, and a slope of 1 thereafter, is fitted to the data (- - -). Free parameters: constant a and point of sudden increase tC.

The delay of the second saccade (or saccadic RT) was smaller when participants pointed to the nearer target [TRIG * AMP1; F(32,4) = 32.4; P < 0.0001]. Since arm movement time is, in general, shorter for movement amplitudes of 100 mm than for 200 mm, the latter observation of shorter second saccadic RTs for 100 mm than for 200 mm pointing amplitudes is congruent with the notion that the prolonged saccadic RTs were caused by the moving arm.

TIME FROM POINTING OFFSET TO SACCADE ONSET. The time between the occurrence of the second target and hand movement offset (the time the 2nd target was presented during pointing) is plotted in Fig. 5A. It is evident that the majority of saccades (84.1%) were delayed until after pointing offset (the saccadic RT curve lies above the curve for remaining pointing time). The time difference between pointing termination and saccade onset will be referred to as "residual" reaction time, or residual RT. It is interesting to note that the initiation of the second saccade followed shortly after pointing offset (after 47 ± 74 ms, mean ± SD) when the second target emerged at pointing movement onset. Figure 5B demonstrates that most saccades were executed shortly after pointing ended, if the time available during pointing (from target onset to pointing offset) was sufficient to prepare a saccade (a normal saccadic RT is assumed to take approximately 150-200 ms) (Prablanc et al. 1979). When less time was available during pointing, the residual RT was larger, resulting in almost normal saccadic RTs for targets offered during the final 170 ms of the pointing movement, as can be seen from the close-to-zero slope in Fig. 5B for this time period (left of the intersection of both thick lines). A sharp increase in saccadic RTs can be seen for saccades evoked earlier than 170 ms before pointing ends (for t > 170 ms). If saccadic preparation processes were inhibited during pointing, one would have expected that the increase in saccadic RT in Fig. 5B started at t = 0, since after movement offset the saccade would have to be prepared in its entirety. This is clearly not the case, suggesting that saccade execution processes are inhibited in the present experiments. To verify this statement statistically, the function f(t) has been fitted to the data of Fig. 5B, where
<IT>f</IT>(<IT>t</IT>)<IT>=</IT><FENCE><AR><R><C><IT>a</IT></C><C>:t<IT><</IT><IT>t<SUB>C</SUB></IT></C></R><R><C>t<IT>−</IT><IT>t<SUB>C</SUB></IT><IT>+</IT><IT>a</IT></C><C>:t<IT>></IT><IT>t<SUB>C</SUB></IT></C></R></AR></FENCE> (1)
In Eq. 1, t is the time from target presentation to pointing movement offset (negative values represent targets offered shortly after pointing offset), a is a constant value for the left section of the data where RTs are more or less constant, tC is the time when RTs start increasing with slope 1. The free parameters that were fitted are tC and a. To calculate the best fit of the free parameters, a Quasi-Newton gradient descent line-search algorithm was used to minimize the sum-of-squared residuals. In Fig. 5B the fit was performed for all pooled data. Best-fit lines were also performed for the data of each participant separately, and it was found that on average, tC = 170 ± 33 ms, significantly larger than 0 (t-test: T = 5.1, P < 0.00001).

In general, the time a saccade was initiated after hand movement offset was substantially shorter when the trigger was delivered earlier during the hand movement [TRIG; F(32,4) = 68.5; P < 0.0001], regardless of the direction the saccade was made.

POINTING DECELERATION TIME AND ACCURACY. The deceleration time of the pointing movements was comparable for all trigger types [TRIG; F(32,4) = 1.84, P = 0.15] and was on average 0.331 ± 0.044 s. The pointing accuracy, on average 15.3 ± 8.2 mm, was also unaffected by the time that the second target was offered during pointing [TRIG; F(32,4) = 0.5]. The presentation of the second target during pointing neither influenced endpoint distributions nor pointing dynamics. Therefore the dynamic feedback loops that are known to control the final phase of pointing movements are apparently not altered by the appearance of the second target, which would have complicated a direct comparison of the residual RTs among trigger types.

Experiment 2

REACTION TIMES. As in experiment 1, saccadic RTs decreased as a function of the successive trigger types used to elicit an eye movement during pointing [TRIG: F(32,4) = 98.9, P < 0.0001; see Fig. 6]. The RTs of saccades executed with vision were slightly smaller than when executed without vision [ILL: F(8,1) = 12.9; P < 0.01], probably due to the different intensities of the LEDs in both conditions, that were adapted to yield similar saccadic RTs in light and in darkness (see METHODS). An important result is that the slight overall increase of saccadic RTs did not affect the main effect that was observed in experiment 1: the decrease of saccadic RTs with respect to the successive trigger types was observed for saccades executed in an illuminated room as well as in a darkened room, where no vision of the moving hand was available [TRIG * ILL: F(32,4) < 1]. In other words, saccades were delayed during pointing, irrespective of the availability of vision of the hand. As in experiment 1, smaller saccadic delays were observed for saccades evoked during pointing movements with a short amplitude (100 mm) compared with a long amplitude (200 mm) [TRIG * AMP1: F(32,4) = 7.4; P < 0.0005].



View larger version (21K):
[in this window]
[in a new window]
 
Fig. 6. Mean saccadic initiation times for experiment 2. Saccadic RTs (open circle ) and the time from 2nd target onset to pointing movement offset () as a function of the dynamic landmark that triggered stimulus onset during pointing: ON, amax, upsilon max, amin, and OFF. Left panels show that data for trials in an illuminated room, right panels for trials in darkness. Top panels show data for trials with movement amplitude of 200 mm, bottom panels for 100-mm pointing amplitude. Data for horizontal and vertical saccades are pooled. Error bars denote 2 times standard deviation (from top to bottom).

TIME FROM POINTING OFFSET TO SACCADE ONSET. The initiation of the second saccade followed shortly after pointing offset when the second target emerged at pointing movement onset. The residual RT, the time between pointing offset and saccadic initiation (i.e., the difference between both curves in Fig. 6) increased to values in the range of normal saccadic reaction times when the second target emerged later during pointing [TRIG: F(32,4) = 73.9; P < 0.0001], both with and without visibility of the hand. The latter findings imply that the signal that is used to prevent saccades from being executed during pointing movements cannot be derived from a visual image of the moving hand, but is more likely related to proprioceptive or efferent sources involved in arm movement execution.

POINTING DECELERATION TIME AND ACCURACY. The deceleration time of the pointing movements was not influenced by the presence of a second target [TRIG * ILL: F(32,4) = 1.35, P = 0.27], regardless of vision for any of the trigger types. Furthermore, the availability of vision of the moving hand did not influence deceleration time [ILL: F(32,4) = 3.30; P = 0.11]. The pointing accuracy, on average 14.1 ± 3.9 mm, was also unaffected by the time the second target was offered during pointing [TRIG: F(32,4) = 0.95], irrespective of room illumination [TRIG * ILL: F(32,4) < 1]. Pointing was slightly more accurate when vision of the moving hand was available (13.6 mm) compared with pointing without vision of the hand (14.5 mm) [ILL: F(32,4) = 11.2, P < 0.02].


    DISCUSSION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

The present experiments provide evidence for a coupling of the gaze to the target of an arm movement. Saccades away from the pointing target during arm movements in the present task were very rare, with the majority of saccades delayed until after pointing offset (experiment 1). When the second saccadic target was switched on at pointing movement onset, abnormally long saccadic RTs of more than 400 ms were observed; in this case, however, the eyes saccaded to the new target position immediately after pointing termination, resulting in short residual RTs (~47 ms). The fit in Fig. 5B demonstrates that when the time available between target presentation and pointing termination was larger than 170 ms, which is typically a sufficient time for a normal saccade to be prepared and initiated, saccades were delayed and executed shortly after pointing termination. Saccades that were evoked during the last 170 ms of pointing were hardly delayed and had regular reaction times (±220 ms, see Fig. 5B). The behavior depicted in Fig. 5B can therefore be explained with the following mechanism: during manual aiming, saccades away from the target of the arm movement can be prepared, although the execution of the saccade is prevented. When the arm movement ends, the prepared saccade is then "released" and executed almost immediately.

The second experiment replicated the findings of the first experiment in an environment where no vision of the moving arm was available. The results indicate that an internal signal directly related to the arm movement control or proprioception, rather than a visual signal related to the image of the moving arm, is integrated in the oculomotor system, presumably to coordinate the behavior of the manual and oculomotor system.

The present experiments provide new evidence for a mechanism that not only forces the line of gaze to move to the target of an arm movement, but also anchors the gaze to this target during the entire course of the pointing movement (the eye-hand co-alignment mechanism). Other double-step experiments have been reported, in which a pointing movement had to change its course in-flight and move on toward a target that changed its position around hand movement onset (Goodale et al. 1986; Lunenburger et al. 2000; Prablanc and Martin 1992). The mentioned studies reported a second saccade that was hardly delayed, toward the new target position during pointing. However, since the second saccade realigns the gaze and pointing movement after the target jump, these nondelayed saccades are also in agreement with such a co-alignment mechanism.

It should be noted that some evidence exists that small saccades are observed between areas of high contrast within a larger object that is being grasped at that moment (Johansson and Flanagan 1999). The latter findings, together with the findings on jumping targets mentioned above, indicate that one should not infer from the present study that all saccades are impossible during pointing, but merely that ocular gaze is actively co-aligned with the target or object being reached to or grasped. Small saccades between details of objects do not violate such a co-alignment principle. In our previous study (Neggers and Bekkering 2000) we showed that the saccadic delay during pointing is less when the pointing target is not gazed at (participants gazed at some other location), which is also congruent with that notion.

Smooth pursuit eye movements are smoother (i.e., less corrective saccades occur during pursuit) (Koken and Erkelens 1992), or more accurate (Gauthier and Hofferer 1976; Steinbach 1969) when participants are allowed to track a moving stimulus with their hand as well as their eyes, providing further evidence that eye and arm movements are not independent but forced to co-align.

Taken together, accumulating evidence suggests a mechanism actively co-aligning ocular gaze to the target of a pointing movement, which is reflected as a temporally and spatially correlated movement pattern where the eye is "forced" to the pointing target, and as a gaze anchoring to the pointing target thereafter. The following paragraphs discuss a possible role of spatial visual attention in an eye-hand co-alignment mechanism, and suggest neurophysiological correlates for such a mechanism.

Role of visual attention

It is known that visual attention and goal-directed movements are heavily interdependent. Previous research has shown that shortly before a saccade, the performance in recognizing quickly flashed items improves at the location where the saccade will go, compared with items flashed at other positions (Deubel and Schneider 1996). Similar effects are found for items at the future location of a pointing movement (Deubel et al. 1998). Distractors suddenly appearing shortly before the onset of a saccade can delay the saccade to the target considerably (Walker et al. 1997), or can even evoke reflexive saccades to such distractors (Theeuwes et al. 1998). It is generally concluded that spatial attention and goal-directed action are obligatory coupled, and it has been suggested that spatial attention is nothing more than preparation of a movement that may or may not be executed (Clark 1999; Rizzolatti 1983; Rizzolatti et al. 1987).

If during pointing movements, and not only before, as in Deubel et al. (1998), spatial visual processing can be enhanced at, or even limited to, the target region, new targets at other locations than the pointing target region might not be recognized during pointing, and logically no saccade could be made, consistent with the present findings.

In the present study, however, sacades are evoked by simple illuminated dots and not by letter recognition, with results demonstrating that saccades evoked early during pointing are initiated almost immediately after pointing ends (47 ms). Therefore in our study the stimulus LED had only to be localized; there was no recognition of the stimulus characteristics required to perform the task. If visual attention is "free" to shift to another position only after pointing offset, all the processes thought to contribute to a normal saccadic latency (target detection, movement planning, initiation) would have to be executed, resulting in long saccadic RTs. The results indicate that saccadic preparation can apparently be done during pointing when evoked at pointing onset, at least when no recognition of target properties is required. Therefore it seems likely that in the present study a more direct interaction between the manual and oculomotor systems prevents saccades from being executed during pointing movements, although they still can be prepared.

Physiological correlates

Saccades can be evoked directly by electrostimulating neurons in the frontal eye fields (FEF) (Robinson and Fuchs 1969) or in the superior colliculus (SC) in the midbrain (Robinson 1972). The SC transmits the "movement vector" (direction and amplitude) of an upcoming saccade to the brain stem saccade generator (Robinson 1972; Schiller and Koerner 1971; Sparks 1986; Wurtz and Goldberg 1971, 1972). More recent studies also report reach-related activity in the SC (Kutz et al. 1997; Stuphorn et al. 1999, 2000; Werner 1993; Werner et al. 1997a,b). Reach-related neurons have never been found in the FEF, although attempts have been made (Mushiake et al. 1996). It has therefore been suggested that the reach activity in the SC, besides perhaps controlling arm movements, may also play an important role in coordinating ocular gaze to manual aiming movements (Werner et al. 1997a).

Interestingly, the direction of reach movements in SC reach neurons is coded in a retinal reference frame (i.e., with respect to the actual gaze position) for 40% of SC reach-related neurons, lying mainly in intermediate SC layers intermingled with the saccadic neurons (Stuphorn et al. 2000). If the vector coding of reach neurons and saccadic neurons somehow interact, they might account for the coupled behavior of eyes and arm during arm movements, as observed in this study. It is known that SC neurons in these layers are interconnected by inhibitory interneurons (Meredith and Ramoa 1998). Reach neurons in SC probably obtain their activity from motor-related areas, by means of projections from the (pre)motor cortex (Fries 1984, 1985; Werner et al. 1997a,b). The observed gaze anchoring during pointing is also driven internally, as shown in experiment 2 of this study.

Further evidence for a role of the midbrain in coupling eye and hand movements is a case study on a patient with slowly progressive bilateral parietal lobe degeneration, showing "magnetic misreaching" (Carey et al. 1997). When the patient was asked to point to a peripheral target on which she was not currently fixating, her hand automatically went to the point at which she was gazing, whereas when allowed to foveate on the target, her performance was normal. Since access to the parieto-frontal loop for oculomotor control via the FEF is affected by the brain damage in the patient discussed above, the only intact pathway that remains to generate eye movements is via the midbrain SC. If SC is responsible for the coupling of ocular gaze and pointing, coupling should indeed become obligatory.

Altogether, the present experiments show that, during pointing, ocular gaze is anchored to the pointing target. The observed characteristics of the anchoring and physiological findings suggest a role for the primate SC in coordinating saccadic eye and goal-directed hand movements. Electrophysiological recordings during combined eye and arm movements may shed light on the suggested role of the midbrain in eye-hand coordination.


    ACKNOWLEDGMENTS

The authors thank J. Garbotz for assistance in conducting the experiments; F. Banci, K.-H. Honsberg, and H. Milewski for technical support; and S. Jordan, J. Pratt, and two anonymous reviewers for helpful comments on earlier versions of this paper. We also thank D. Romero and C. Dijkerman for careful proofreading and for suggestions regarding the manuscript.

This study was supported by German Research Foundation (DFG) Grant BE 1873/2 as part of the interdisciplinary project "Sensorimotor Integration" and by Dutch Organization for Fundamental Research (NWO) Grant 440-20-000.


    FOOTNOTES

Address for reprint requests: S.F.W. Neggers, Dept. of Psychonomics, Helmholtz Institute, University of Utrecht, Heidelberglaan 2, 3584 CS Utrecht, The Netherlands (E-mail: B.Neggers{at}fss.uu.nl).

Received 8 September 2000; accepted in final form 17 April 2001.


    REFERENCES
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

0022-3077/01 $5.00 Copyright © 2001 The American Physiological Society