Max-Planck-Institute for Psychological Research, Department of Cognition and Action, 80802 Munich, Germany
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Neggers, S.F.W. and H. Bekkering. Ocular Gaze is Anchored to the Target of an Ongoing Pointing Movement. J. Neurophysiol. 83: 639-651, 2000. It is well known that, typically, saccadic eye movements precede goal-directed hand movements to a visual target stimulus. Also pointing in general is more accurate when the pointing target is gazed at. In this study, it is hypothesized that saccades are not only preceding pointing but that gaze also is stabilized during pointing in humans. Subjects, whose eye and pointing movements were recorded, had to make a hand movement and a saccade to a first target. At arm movement peak velocity, when the eyes are usually already fixating the first target, a new target appeared, and subjects had to make a saccade toward it (dynamical trial type). In the statical trial type, a new target was offered when pointing was just completed. In a control experiment, a sequence of two saccades had to be made, with two different interstimulus intervals (ISI), comparable with the ISIs found in the first experiment for dynamic and static trial types. In a third experiment, ocular fixation position and pointing target were dissociated, subjects pointed at not fixated targets. The results showed that latencies of saccades toward the second target were on average 155 ms longer in the dynamic trial types, compared with the static trial types. Saccades evoked during pointing appeared to be delayed with approximately the remaining deceleration time of the pointing movement, resulting in "normal" residual saccadic reaction times (RTs), measured from pointing movement offset to saccade movement onset. In the control experiment, the latency of the second saccade was on average only 29 ms larger when the two targets appeared with a short ISI compared with trials with long ISIs. Therefore the saccadic refractory period cannot be responsible for the substantially bigger delays that were found in the first experiment. The observed saccadic delay during pointing is modulated by the distance between ocular fixation position and pointing target. The largest delays were found when the targets coincided, the smallest delays when they were dissociated. In sum, our results provide evidence for an active saccadic inhibition process, presumably to keep steady ocular fixation at a pointing target and its surroundings. Possible neurophysiological substrates that might underlie the reported phenomena are discussed.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
When humans interact with their environment, they
often make a combination of saccadic eye movements and goal-directed
hand movements. Saccadic eye movements, for instance, are made
repeatedly to targets of interest to use the high foveal receptor cell
density on the retina. After a minimum fixation period of ~200 ms, a
new saccade can be generated to foveate another target
(Feinstein and Williams 1972). Various eye-hand
coordination studies showed that a saccadic eye movement usually
precedes a goal-directed hand movement. Because of the short saccadic
movement duration, the eyes, typically, even arrive at the target
before the hand starts to move (Bekkering et al. 1994
,
1995
; Frens and Erkelens 1991
;
Prablanc et al. 1979
, 1986
). It is also known that
pointing accuracy suffers when a visual target is not fixated
(Abrams et al. 1990
; Neggers and Bekkering
1999
; Prablanc et al. 1979
; Vercher et
al. 1994
). Also the disappearance of a target after it already is fixated with the eyes results in less accurate pointing movements (Prablanc 1986
) to this now invisible target position
even when no vision of the moving arm is available. Together these
results indicate that the fixation of a visible pointing target is a
favored and perhaps even necessary precondition for accurate aiming
movements. The neural mechanisms that are regulating goal-directed hand
movements might profit from the accurate spatial information of the
foveated target. Presumably to achieve foveal control of the pointing
target, the eyes are directed to a pointing target before pointing
takes place, and might logically be forced to keep a stable fixation thereafter. It generally is believed that during pointing deceleration visual, on-line, closed loop control of pointing takes place to guide
the hand to the target, and that the first part of the pointing movement, where the pointing movement is accelerating, is ballistic and
more or less "preprogrammed" and "open-loop" (Keele
1981
; Paillard 1982
). Particularly
during pointing deceleration, visual stability could be a prerequisite
for accurate pointing.
Oculomotor behavior during active pointing has been measured by some
researchers in so-called double-step saccade paradigms (Blouin
et al. 1995; Goodale et al. 1986
;
Prablanc and Martin 1992
). That is, a second
target was presented during the execution of the first saccade,
typically around the onset of the pointing movement directed to the
first target, and because the first target then disappeared, a target
"jump" was created. Because during a saccade visual perception is
suppressed (Bridgeman et al. 1975
, 1994
), subjects are
not aware of this target jump (Goodale et al. 1986
). In
these paradigms, subjects not only had to saccade to the second target
but also had to point to the second target instead of the first as
initially planned. In general it was found that the pointing movement
could smoothly change its course and land at the second target, proving
evidence for the notion that pointing is not completely preprogrammed.
Blouin et al. (1995)
, using "free" (not ending with
target contact) speeded pointing responses, failed to find corrections
in the initial pointing movement; corrections first were observed in
after-movements.
The mentioned studies observed (second) saccades with normal latencies
during pointing under these circumstances. Only in the mentioned study
of Blouin et al. (1995) the pointing movements were so
fast (210 ms, a regular saccadic reaction time) that the second
saccades on average were executed at pointing movement offset.
Importantly, however, in these studies, both eye and hand needed to be
directed to the second target. Therefore to investigate the hypothesis
that the fixation of a visible pointing target is a favored and perhaps
even necessary precondition for accurate aiming movements, it would be
interesting to instruct the pointing movement to stay directed to the
first, visible target, while instructing the eyes to fixate as fast as
possible on the second target.
Furthermore there are several recent neurophysiological findings that
suggest an influence of arm movements on the oculomotor system.
Interestingly, in single-cell recording studies (Stuphorn et al.
1999, 2000
; Werner et al. 1993
,
1997a
,b
), it is reported that neurons in the deep and
intermediate layers of the monkey midbrain structure superior
colliculus (SC), well known to be involved closely into the generation
of saccadic eye movements (Robinson 1972
;
Schiller and Koerner 1971
), are active shortly before
and during arm movements as well. Some of these neurons also have
saccadic components. The saccadic neurons in SC are organized in an
anatomic map with caudal regions coding for large saccades and more
rostral regions for small saccades and fixation (Munoz and Wurtz
1993a
). Interactions between saccades or active fixation and
arm movements therefore could arise at the level of the SC. In the
DISCUSSION, the possible neural mechanisms underlying eye-hand coordination are discussed in more detail and to what extent
they are able to explain the findings in this study.
The aim of the present investigations is to examine oculomotor behavior
during goal-directed pointing. The first experiment in this study was
set up to investigate the ability to make saccades away from the
pointing target during pointing. The paradigm required the eye and hand
to move from a central fixation point to a common first target. Because
the eyes in general fixate the first target before the hands start to
move, no further instructions on eye movements were given. A second
target was flashed on when the pointing movement reached peak velocity
(referred to as dynamic trials). This moment was chosen
because by doing so saccades, if any, will be executed during the
deceleration phase of the pointing movement, where visual stability
might be required, as mentioned before, and the largest delay can
probably be found. Latencies of saccades to this new target were
compared with latencies of saccades toward target stimuli, which were
presented when a subject had just completed pointing (referred to as
static trials). Also if saccades indeed are delayed even
until after pointing in the dynamic trials, the time between the offset
of the pointing movement and the initiation of a saccade is compared
with "normal" saccadic RTs in the static conditions. Latencies of
visually evoked movements, such as the saccades in this experiment, in
general are thought to reflect a preparation stage, where the metrics of a saccade are determined, and finally an execution stage, i.e., the
generation of a motor program that can drive the eyes to a target
(Carpenter 1988). If only the execution stage of
saccadic eye movements is inhibited until pointing is over but the
saccadic metrics can be prepared in its entirety during pointing, a
saccade can be executed immediately after pointing stops. If, however, the planning of a saccade is delayed as well and occurs after pointing
is completed, the saccade probably is generated after pointing with a
delay that is comparable with a regular saccadic RT.
Because it is known that saccades that are evoked shortly after a
previous saccade have longer reaction timesreferred to as the
saccadic refractory period (Feinstein and Williams
1972
)
and interstimulus-intervals in static and dynamic trials
were different in experiment 1, experiment 2 tested the
saccadic refractory hypothesis as the underlying cause of the
hypothesized delay of saccades in the dynamic condition. If saccadic
RTs turn out to be similar for trial types with a short
interstimulus-interval (ISI) in a sequence of two saccades, as for
trial types with long ISIs, this will indicate that if a saccadic delay
during pointing is observed in experiment 1, this cannot
be contributed to different ISIs.
In addition, we want to focus on saccadic delays during pointing under different types of pointing-gaze configurations. It is investigated specifically whether saccades away from a target that is being pointing at are prevented, whereas other saccades are not. If a visual control mechanism (requiring a stable gaze at the pointing target) that is regulating the final pointing phase is the underlying cause of saccadic inhibition, one might expect that saccadic delay during pointing decreases when less foveal information about the final pointing phase is available. Therefore experiment 3 investigated if saccades are inhibited to the same extent (as in experiment 1), when the ocular gaze during pointing is not directed toward the pointing target, but 50 or 100 mm above it. The experiments are described in the following text in more detail.
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Apparatus
In Fig. 1A, a schematic overview of the laboratory, the experimental setup, and the equipment used is given. Pointing and saccadic targets were selected from a matrix of 12 × 8 two-colored light-emitting diodes (LEDs), mounted in 5-mm cylindrical holes in a Plexiglas board (1,200 × 500 mm). The board was mounted on a stand, at a rotatable joint, at the height of a normal desk (800 mm from floor to surface), and could be tilted and rotated along a vertical axis perpendicular to the floor. The distance between two LEDs was 50 mm, the whole matrix spanned 600 × 400 mm. A black sheet of paper covered the LED-matrix, with circular openings of 2 mm in size above each LED, creating a target size of 2 mm. Directly on top of this sheet was a 2.5-mm opaque Plexiglas plate that prevented vision of the inactive LEDs. The LEDs could be illuminated in red (660 nm) and green (565 nm). When both colors (1 light pitch contains 2 diodes) were illuminated, a yellow light could be observed. The LEDs were switched on and off by a 192 × digital IO card. Both eye (gaze) and hand movements were measured in a coordinate frame that coincided with the board with the origin at the lower left corner.
|
Eye movements were measured with the SMI EyeLink system. Both eyes were measured with infrared digital cameras attached to small tubes that were mounted on a helmet. The cameras were positioned below the eyes at the cheek level. On the camera, two IR sources were mounted the reflection of which on the cornea was measured. By choosing the proper threshold, the pupil position could be isolated, and the orientation of the eyeball in the head could be calculated. A camera mounted on the headband of the EyeLink helmet, at the forehead, measured the position of four IR LEDs on top of the stimulus board, from which head orientation and position in space could be calculated. Head position and the angles of the eyes in the head allowed the calculation of the ocular gaze coordinates on the LED board in mm, resulting in x, y coordinate pairs with the origin in the upper left corner. In this way, the gaze of both eyes was tracked at a rate of 250 Hz. The viewing distance of the head relative to the LED board was ~600 mm but was not restrained. Before each block, the eye system was calibrated by offering nine saccadic targets across the board that the subject was required to fixate.
The position and orientation of the fingertip were measured with the OPTOTRAK infrared tracking system of Northern Digital. A tube with a narrow elongated tip was placed over the index finger (see Fig. 1B). Three OPTOTRAK markers were attached at the tube and tracked with 250 Hz. They formed a coordinate frame that moved with the index finger. The coordinates of the tip of the tube in this reference frame were precalibrated. For analysis, the tip position was calculated in LED-board coordinates, using the traces of the three markers. This procedure was used to enable a free view on the tip (no markers attached on it).
To measure the orientation of the LED board in space, three OPTOTRAK markers were attached to the front of the board. The fingertip position was transformed from lab-fixed coordinates to a LED-board-fixed coordinate frame. By doing so, the fingertip position was measured in exactly the same coordinate system as the ocular gaze.
The SMI EyeLink tracker and the LED board were controlled by a Pentium PC, the OPTOTRAK system by another Pentium PC. Both systems were synchronized over a parallel interface. The experimental procedures to control the hardware and generate stimuli were programmed in C. In all experiments the room was dimly lit by a central spotlight attached at the ceiling. The arm and the pointing device were clearly visible.
Subjects
For each experiment, 10 subjects were asked to participate. Subjects were healthy, right-handed, had normal or corrected to normal vision, and were mostly students or doctoral students. Most subjects had experience in participating in behavioral experiments. All subjects were selected from the age range 18-32 yr. Subjects were informed beforehand about the experimental procedures, volunteered to participate and were paid 6.14 EURO/h.
Stimulus presentation
Before laying out the exact design of the experiments, the concept of dynamic and static triggering that is used to compare saccades during pointing with normal saccades, will be explained. Those trial types were used in experiments 1 and 3. First the so-called dynamic-trigger trials are discussed (50%); see Fig. 2. The start position for fixation and pointing was a yellow fixation point that was illuminated throughout the whole trial. It was situated at the midline of the LED board directly in front of the subject. Initially the subject fixated the central fixation point and touched the surface of the LED board near it (Fig. 2A). This was controlled; that is, a trial first starts when subjects fixate properly. Subjects were instructed to make an eye movement and point to the red target that emerged (Fig. 2B) at the left side of the fixation point. No further instruction about eye movements was given for the first movement. Subjects were told to point accurately, the tip of the pointing device (<1 mm diam) had to land within the target region (2 mm diam), no constraints on speed were induced for pointing or the first saccade. A second eye movement was triggered by a green dot that was switched on (Fig. 2C) when the hand movement reached its peak velocity and the eye was directed to the red target. For more details on the on-line calculation of peak velocity, see Data analysis. The static-trigger trials (50%) only differ with respect to the triggering of the second target. The second target for the eye movement appeared when the hand reached the first red target and was at rest (Fig. 2C) instead of moving with peak velocity, as in the dynamic-trigger condition.
|
Subjects were instructed to make a saccade to the green target as quickly as possible (Fig. 2D). The hand could stay on (static trials) or continue to (dynamic trials) the red target. When the second saccade had been made and pointing was completed, the subject could move back to the fixation point; all other targets then extinguished. Subjects had to fixate the fixation LED and put their pointing tip back on it, which was controlled. If these conditions were satisfied, a new trial started. The differentiation between saccades in static- and dynamic-trigger conditions made it possible to study saccades that were performed under similar conditions except that the hand is at rest in the former condition and moving in the latter condition.
Experiment 1
A trial started 300 ms after the subjects fixated the central fixation point. The first (red) target, for pointing and the first saccade, appeared at 200 or 100 mm to the left of the fixation point. The second (green) target, for the second saccade, emerged above the first target (away from the subject, creating a 2nd movement direction at an angle of 90° relative to the first) or to the left of the first (red) target, at a distance of 50, 100, or 150 mm (see Fig. 3A). This was done to investigate if saccades were inhibited independent of their direction and amplitude. Only leftward movements were chosen because by doing so the moving arm will never prevent vision of emerging stimuli (all subjects were right-handed). It was unfortunately not possible to offer targets at more spatial positions because the combination of conditions and repeats necessary for statistical reliability already required experiments with a duration of ~1 h. Longer experiments usually result in concentration problems and make further testing unreliable.
|
The paradigm consisted of 24 trial types (2 pointing distances × 2 trigger types × 3 saccadic amplitudes × 2 saccadic directions). The 24 trial types were ordered at random. Three blocks of trials were executed in one experimental session. Each block contained 96 trials. Subjects were to perform 10 practice trials before the session.
Experiment 2
A similar procedure was used as in experiment 1 except that no hand movements had to be executed. A trial started 300 ms after the subjects fixated the central fixation point. Again a first target was offered at the left side of the fixation point, and now only a saccade had to be executed toward this target. In 50% of all trials, after a delay of 640 ms (comparable with the mean ISI in a dynamic trial of experiment 1) a second target was offered. A second saccade had to be initiated. In another 50% of the total number of trials, a delay of 920 ms (comparable with the mean ISI of the static trial type in experiment 1) was used between the appearance of both targets. The target positions were identical to experiment 1 as were target size, target colors, block, and trial numbers.
Experiment 3
A trial started when the subjects fixated the central fixation point and moved the pointing tip to the surface of the LED board within 25 mm of the central fixation point. Then after a 100-ms delay, a green dot (always at a vertical axis 100 or 200 mm to the left of the initial fixation point) appeared, that signaled the subject where to fixate (see Fig. 3B). This green dot could appear at three different positions on a vertical axis: at the pointing target (not illuminated yet) or located on a position 50 or 100 mm above it. In the figure these three positions are shown for large pointing targets. After 1500 ms, the red pointing target appeared, either 100 or 200 mm to the left of the fixation target at the same vertical axis as mentioned in the preceding text. It could cover the green target the eye was already fixating on when that green target appeared at the lowest of the three possible positions (Fig. 3B, top). To create comparable conditions for the three initial fixation conditions (0, 50, or 100 mm above the pointing target), the previously green fixation target always turned red when the pointing target appeared. Subjects were instructed to initiate a pointing movement while they fixated at the pointing target or 50 or 100 mm above the pointing target. Again the appearance of a second target was triggered by peak velocity or movement offset of the hand movement (dynamic or static trials). The green second target now could appear 100 mm to the left or above the target the eyes were fixating. Subjects again were instructed to execute a saccade to this target as quickly as possible.
Pointing amplitudes (100 or 200 mm), fixation-pointing target dissociation (0, 50, or 100 mm), second saccadic directions (upward or leftward), and trigger type (dynamic or static) were randomized throughout a block. There were four blocks of trials. Each block contained 72 trials.
Data analysis
ON-LINE. The peak velocity of a hand movement, as used to trigger a second saccadic target during pointing (see Stimulus presentation) was determined on-line. The OPTOTRAK system allows to request a data sample of each marker on-line, i.e., during sampling of the movement. The velocity of the moving finger was calculated as the three-dimensional distance between the most current sample of fingertip position and the previous one, divided by the time between two samples (4 ms). This method was accurate enough to detect a threshold velocity; no filtering was necessary. Peak velocity is defined as the moment where the velocity exceeds a threshold of 0.35 m/s, and the change in velocity (= acceleration) drops <0.01 m/s2. The latter constraint determines whether velocity is bound to change from increasing to decreasing. Calculations were performed by the c-routine that was running during OPTOTRAK data-collection.
OFF-LINE. The data were analyzed off-line using MATLAB 5.0 scripts. To analyze the performance of the subjects, a number of movement parameters was calculated. For hand movements, the tangential velocity was calculated along the trajectory. For detection of the hand-movement onset and offset time, a relative velocity threshold was used, 5% of peak velocity. Onset of movement was defined as the moment the tangential velocity exceeded this threshold, and a minimum displacement, 25% of movement amplitude, took place within 300 ms after this onset. The offset time of a trial was the reverse of this procedure, taking the time of the first sample after the detected movement onset where the velocity decreased to a level lower than the threshold. Saccadic onset was defined as the moment when the ocular velocity measured in degrees relative to the straight-ahead gaze (in head-referenced coordinates) exceeded 35°/s and the ocular acceleration exceeded 9,500°/s2, saccadic offset was the time when the saccadic velocity and deceleration dropped below these thresholds. The reaction time (RT) of a saccade or pointing movement was defined as the time between stimulus presentation and movement onset. The so called "residual reaction time" (residual RT) of a saccade was defined as the time between the offset of a pointing movement and the onset of a second saccade that was triggered during pointing (see METHODS, Stimulus presentation).
The accuracy of a pointing movement (or a saccade), was defined as the two-dimensional (= projection on the board-plane) spatial distance between the x, y coordinates of the tip of the pointing device (or the ocular gaze coordinates) at movement offset time, and the x, y coordinates of target position on the board. For all movement parameters, a tolerance window was defined. When one of the movement parameters was outside this window, the whole trial will be excluded from further analysis. The window sizes were very large and were only meant to exclude out errors made in executing the task. The window sizes are: saccadic RTs: [0.10, 0.70] s, saccadic durations: [0.01, 0.1] s, saccadic accuracy: [0, 70] mm, hand-movement RTs: [0.1, 0.8] s, hand-movement durations: [0.1, 0.8] s, hand-movement accuracy: [0, 40] mm. The experimental factors used in the different conditions were used to group the various movement parameters and are abbreviated as follows. Experiments 1 and 2: TRIG (dynamic or static), AMP1 (pointing and 1st saccadic amplitude, 200 or 100 mm), DIR (direction 2nd saccade, upward or to the left), and AMP2 (amplitude 2nd saccade, 50, 100, or 150 mm). Experiment 3: TRIG, AMP (pointing amplitude, 200 or 100 mm), DIR, and DISS (pointing-fixation target dissociation distance, 0, 50, or 100 mm). The reported F values are obtained with a repeated measurements ANOVA, and reflect "within subjects" effects. ![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Experiment 1
TYPICAL TRIALS. In Fig. 4, a pointing and eye-movement trace are plotted in time for the static condition (A and B) and for the dynamic condition (C and D). This trial required a second saccade to a target at 100 mm left of the first target (see Fig. 4, E and F, for a spatial top view). The graphs illustrate that the second saccade in the dynamic condition was delayed severely compared with the second saccade in the static condition and was executed well after the fingertip landed on the target.
|
ISI. An overview of the resulting ISIs (time between stimulus onsets 1 and 2) can be found in the top row of Table 1 averaged over all subjects. The two different pointing amplitudes that were used resulted in significantly different ISIs for static trials [AMP1: F(9,1) = 6.77; P < 0.02], but ISIs were similar for dynamic trials [AMP1: F(9,1) = 0.368].
|
REACTION TIMES OF PRIMARY SACCADES. The reaction time of primary saccades (i.e., not including corrective saccades) toward the second target was analyzed with respect to the experimental conditions. A mean increase of 155 ± 79 ms (mean ± SD) was observed for saccades RTs to targets that were offered during pointing, the dynamic condition, compared with saccades to targets that were offered after pointing [TRIG: F(9,1) = 68.9; P < 0.0001; see Fig. 5].
|
ONSET OF THE SECOND SACCADE WITH RESPECT TO THE HAND-MOVEMENT OFFSET. The residual RTs (the time between hand-movement offset and saccadic onset) of saccades in dynamic trials were compared with the RTs of saccades in static trials. If saccades can be prepared during pointing but not executed, the resulting residual RTs should be substantially shorter than normal saccadic RTs, i.e., saccades are executed immediately after pointing stops. However, if saccades cannot be prepared at all during pointing, the whole saccadic initiation process starts when pointing is completed and should result in residual RTs that are comparable to normal, or statically triggered, saccadic RTs. Importantly, no significant difference was observed [TRIG: F(9,1) = 2.1; P = 0.18], implying that the residual RTs of dynamically triggered saccades were comparable with normal, or statically triggered RTs. See Table 1 for the observed residual RTs. No interaction was observed for saccadic amplitudes [TRIG*AMP2: F(18,2) = 1.07; P = 0.36] nor for saccadic directions [TRIG*DIR: F(9,1) = 2.12; P = 0.18] nor for pointing amplitudes [TRIG*AMP1: F(9,1) = 2.06; P = 0.19].
ACCURACY OF SACCADES. Primary saccades were as accurate in the dynamic condition as in the static condition, for both horizontal and vertical saccades, and all pointing amplitudes [TRIG: F(9,1) = 3.2; P = 0.11]. No interactions with other conditions were observed. See Table 1 for the observed saccadic accuracies.
POINTING DURATION AND ACCURACY. The pointing accuracy was the same in dynamic trials and static trials for all directions and amplitudes of the second saccade (see Table 1). In other words, pointing performance did not deteriorate when during pointing a target for a new saccade was flashed on while fixating the pointing target [TRIG: F(9,1) = 0.07]. No interactions with other experimental conditions were observed, implying that neither vertical saccadic targets offered during pointing nor horizontal saccadic targets resulted in deviated pointing landing position for both pointing amplitudes. Also deceleration times of the moving hand (from peak velocity to movement offset) were not changed when a saccadic target is offered during pointing compared with trials where the target first appeared after landing of the finger [TRIG: F(9,1) = 0.83] nor did any other experimental conditions interact. Pointing deceleration times were longer for pointing target amplitudes of 200 mm (0.348 ± 0.048 s) than for 100 mm [0.270 ± 49 s; AMP1: F(9,1) = 166, P < 0.0001].
CORRELATIONS BETWEEN POINTING DURATION AND SACCADIC RTS. To examine how the delay of saccades that are evoked during pointing is influenced by pointing movements, the correlation (Pearson product moment) between the duration of the pointing deceleration time (from the onset of the 2nd target, shortly after the hand reaches peak velocity, to movement offset) and saccadic RTs in the dynamic trials was calculated on a trial-by-trial basis. A significant correlation was found for most subjects. The correlation coefficients of all subjects are given in Table 2. In Fig. 6, the values of saccadic RT and deceleration time of single trials are plotted as coordinate pairs for three subjects.
|
|
Experiment 2
The RTs of double-step saccades to a sequence of two targets with different ISIs were analyzed. Saccades to the second target had RTs that were on average 29 ± 11 ms longer when the ISI was short (640 ms) compared with trials with a long (920 ms) ISI [F(9,1) = 43.3; P < 0.0001]. The observed increase in RTs of the second saccade for short ISIs was not influenced by the size of the first saccade [TRIG*AMP1: F(9,1) = 1.73; P = 0.22] nor by the direction [TRIG*DIR: F(9,1) = 1.55; P = 0.24] nor the amplitude [TRIG*AMP2: F(18,2) = 1.02; P = 0.38] of the second saccade. The delay apparently was only depending on the time interval between both target presentations and therefore on the time interval between the execution of the first saccade and the presentation of the second target (see Table 1 for the observed RTs).
Experiment 3
SACCADIC REACTION TIMES.
The RTs of saccades toward the second target were in general longer in
the dynamic condition compared with the static condition [TRIG:
F(9,1) = 33.6; P = 0.0003].
Importantly, an interaction effect was observed for the dissociation of
fixation target and pointing target [TRIG*DISS:
F(18,2) = 8.1; P = 0.003]. In
other words, RTs of saccades that were evoked during pointing were
again longer than RTs of saccades that were evoked after pointing;
however, this difference decreased in size when subjects fixated a
target that did not coincide with the pointing target. The size of the difference was smallest for a large dissociation distance (100 mm) and
biggest when fixation and pointing targets coincided (see Fig.
7). A t-test comparison of
the measure RTdynamic RTstatic (for the
2nd saccade) between trials with coinciding targets
(d = 0) on the one hand and 50- or 100-mm target
dissociation on the other hand yielded a significant difference
(P < 0.05 for both d = 50 mm
and d = 100 mm). However, a comparison between trials with 50 or 100 mm target dissociation did not result in a
significant difference for this measure (t-test:
P = 0.52).
|
|
ONSET OF THE 2ND SACCADE WITH RESPECT TO THE HAND-MOVEMENT OFFSET. As in experiment 1, the time between hand-movement offset and saccadic initiation was analyzed. Overall, the residual reaction time of saccades was now significantly smaller (145 ms) than static RTs [230 ms; TRIG: F(9,1) = 104; P < 0.0001]. See Table 1 for the values of the residual RTs. This difference tended to be larger for dissociated targets [TRIG*DISS: F(9,1) = 3.36; P = 0.058].
POINTING DECELERATION TIME AND ACCURACY. Pointing deceleration times did not change as a result of the target dissociation or trigger type (dynamic vs. static). No interaction was found either [DISS: F(9,1) = 1.06; P = 0.37; TRIG: F(9,1) = 1.74, P = 0.22; TRIG*DISS: F(18,2) = 0.35]. Pointing accuracy was invariant under target dissociation or trigger type (dynamic vs. static) as well. No interaction was found either [DISS: F(9,1) = 0.7; TRIG: F(9,1) = 3.2; P = 0.59; TRIG*DISS F(18,2) = 3.1, P = 0.072]. See Table 1 for the observed accuracies and deceleration times.
CORRELATIONS BETWEEN POINTING DURATION AND SACCADIC RTS. As in experiment 1, correlations between the remaining movement time of the hand after the second target had been switched on, and the RT of the saccade toward that target were calculated. Again significant correlations were found. However, these correlations decreased as a function of target dissociation. In Fig. 9, all trials for all subjects were pooled and plotted in three panels, corresponding with the three dissociation distances of fixation target and pointing target. The increase of saccadic RT with pointing deceleration time is clearly visible in the top graph (coinciding targets), this coherence is obviously less in the middle and bottom graphs (dissociated targets).
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
First, the main findings of the three experiments are summarized, then a general discussion is provided.
Experiment 1
Experiment 1 has revealed three important conclusions about saccadic delay during pointing to a fixated target.
First, typically subjects did not make an eye movement when the hand still moved toward the target although the instruction was to make an eye movement as quickly as possible to the emerging target. This observation is best illustrated in Fig. 6; the reaction time for the second saccadic eye movement was larger than the deceleration time of the hand movement (best depicted by the fact that almost all points lie above the line y = x) for all subjects. The amplitude and direction of the planned saccade (see Fig. 5) did not affect the delay of the saccades in the dynamic condition compared with the static condition.
Second, in dynamic trials, the time between the termination of the hand
movement and the onset of the second saccade (residual reaction time)
was comparable with saccadic RTs in the static condition. In other
words, the residual RT reflects a normal saccadic initiation process,
starting after the offset of the hand movement. This finding implies
that not only the execution of saccades was inhibited during pointing
but also the perceptual processes related to target localization and
the planning of the saccadic eye movement (Carpenter
1988). If only the execution stage would have been inhibited,
significantly shorter RTs than for normal saccades should have been observed.
Third, the finding that saccades are delayed until the hand movement is terminated is stressed by the following observations.
A within-subject correlation was observed between the deceleration time of the hand movement and the reaction time of dynamically triggered saccades, see Fig. 6 and Table 2.
Also, a larger saccadic delay is observed when pointing movements were executed toward a target at a larger amplitude (200 compared with 100 mm) from the starting point (Fig. 5), a finding consistent with the notion that deceleration time is directly related to movement amplitude (for bell-shaped hand movements) and that longer deceleration times in its turn influences saccadic RT during pointing.
Altogether these findings provide evidence for the hypothesis that the preparation and execution of a saccade can only start after the fingertip has arrived at the target.
Experiment 2
Experiment 2 controlled for the fact that in experiment 1, the ISI in static trials (640 ms) was shorter than the ISI in dynamic trials (920 ms). No hand movements were executed in experiment 2. However, an increase in RTs of the second saccade of only 29 ms was found for short ISIs compared with long ISIs, indicating that the biggest part of the effect of experiment 1 was due to the accompanying hand movement and not to a saccadic refractory period.
Experiment 3
In general, saccadic delays during pointing decreased when the
starting point of the saccade did not coincide with the pointing target. However, a considerable delay in generating a saccade also was
observed when subjects were not gazing at the pointing target. It has
to be noted that oculomotor behavior varied substantially between
subjects in this experiment. Some subjects executed saccades without
any delay when they gazed at a different position as where they were
pointing to; some executed saccades that were delayed to the same
amount irrespective of the dissociation between pointing and fixation
targets, others showed intermediate behavior. Note that the double-step
saccade-pointing studies mentioned in the INTRODUCTION
(Goodale et al. 1986; Prablanc and Martin
1992
) observed saccades during pointing but toward a
pointing target and not away from it. Moreover because the decrease of
saccadic delay observed in this experiment for saccades away from
targets that is not currently being pointed to, one can conclude that
the ocular gaze-anchoring during pointing, as observed in this study,
is specific for ocular gaze directed at or near a pointing target.
It is in general believed that processes that are involved in visual
control of pointing consume time. For example, when visual targets are
switched off during pointing, pointing durations are known to decrease
(Prablanc et al. 1986). Abrams et al.
(1990)
, however, found that hand-movement durations did not
change as a result of less foveal control. The dissociation between
fixation location and pointing target neither affected deceleration
times nor pointing accuracy in this experiment. Therefore the observed modulation of the delay time of saccades by target dissociation, as
observed in experiment 3, cannot be caused indirectly by
a changing deceleration time. This is an important point to make because in experiment 1, pointing deceleration times
were found to correlate positively with the delay time of saccades
elicited during pointing.
The correlation between remaining pointing time (after 2nd target onset) and saccadic RT, as observed in experiment 1, was observed in experiment 3 as well. However, the correlation was much weaker when the fixation position and pointing target were dissociated.
The residual reaction time of a saccade was now significantly shorter than a normal (statically triggered) saccade unlike in experiment 1. This implies that mainly motor processes were inhibited here. This difference with experiment 1 is discussed in more detail in the following text.
General discussion
DELAY OF SACCADES DURING POINTING. The most important finding of the present study was the observation that ocular orientation was locked to a target during goal-directed reaching to that same target. That is, although instructed to do so, subjects were not able to initiate a saccade to a new target when the hand was reaching for the first target. Interestingly, the residual RT of these saccades, that is the time needed to initiate a saccade after pointing offset was comparable to normal "single" saccadic RTs (experiment 1). This implies that the time that was available in the so-called dynamic trials, if the target was present before hand-movement offset, apparently could not be used to prepare a saccade. The present results provide evidence for the idea that not only saccadic execution is inhibited during goal-directed pointing movements but also the preparation of a saccadic eye movement toward a new visual target stimulus. If response preparation (i.e., the perceptual analysis of the target for the second saccade) would have been partially completed during pointing, the residual RT should have been smaller than normal RTs.
Interestingly, shorter residual RTs were observed in experiment 3. However, in this experiment, targets for the inhibited saccade could only emerge at two positions: 100 mm to the left or above the initial fixation target. In contrast, in experiment 1 the second target could appear at six possible positions. Presumably, anticipatory effects play a role here as have been reported to play a role in saccade generation (Findlay 1981Saccadic initiation and visual attention
The notion that the second saccade could not be prepared at all
during ongoing hand movements (in case of a high number of potential
target positions), resulting in residual RTs comparable with normal
saccadic RTs, links the current findings to visual attentional
mechanisms and extends the underlying mechanism beyond an inhibition
between two motor systems. Various studies in the past decade
concentrated on the strong connection between visual attention and
saccades. It has been shown that perceptual identification improves
dramatically for objects presented at a future saccade target location
shortly before saccade execution compared with objects at other
positions in the visual field (Deubel and Schneider 1996; Hoffman and Subramaniam 1995
;
Kowler et al. 1995
; Shephard et al.
1986
). It also is found that the recognition reaction time decreases when the to be recognized object and saccadic target coincided. This pattern is observed even when subjects had advance information about where the object will appear (Deubel and
Schneider 1996
). Recently Deubel et al. (1998)
have broadened this approach to goal-directed hand movement as well and
show that similar recognition enhancements can be found for pointing
targets even when the eyes fixate a central fixation-dot-fixation point.
In the present study, the homing in of the hand on the target region
presumably required spatial on-line movement control (e.g.,
Abrams et al. 1990). This homing-in mechanism might have tied visual attention to the fixated region, thereby preventing the
shift of attentional focus to the new saccadic target. The premotor
theory of attention (Rizzolatti et al. 1987
) assumes that a shift of visual attention is obligatory to prepare a saccade. If
visual attention indeed was tied to the pointing location, a necessary
condition for preparing a saccade could thus not have been fulfilled
during pointing, and as a consequence subjects postponed the initiation
of a saccade until pointing is completed. This attentional construct of
processes that might underlie saccadic inhibition during pointing is
able to explain why the observed residual RT was comparable with normal
single saccades in case of multiple possible target locations.
The findings from experiment 3 showed that the observed
delay was modulated by the distance between the target of the
goal-directed hand movements and the fixation target. The largest
saccadic delay was found for coinciding targets, but again the increase
in saccadic RT in dynamic trials was independent of saccadic direction
and amplitude. The hand-movement deceleration times were unchanged by
target dissociation, therefore the observed modulation effect was
caused directly by the fact that the pointing endpoint was not
foveated. The findings of Walker et al. (1997) bear some
resemblance with the results of this study. They offered a visual
distractor at the same time as a target for a saccade, holding target
position constant (4 or 8° eccentricity), and varying the distractor
position (from
10 to 10° with a step of 2°, experiment 1a). A
delay of saccadic initiation was observed (
40 ms), only when
distractors were offered in a large section of the contra lateral
hemifield, and the delay was found to be modulated by the distance
between distractor and fixation position, with a maximum for a
distractor at fixation location, and was not depending on the
eccentricity of the target location (and therefore the saccade) nor the
direction (left or right). An opposite effect, the "gap-effect,"
emerges when a fixation point disappears before target onset. In this case, saccadic latencies then were found to be shorter than when the
fixation point was continuously visible (e.g., Saslow
1967
). Also more express saccades, with a latency <100 ms,
were observed under such conditions (Fischer and Ramsperger
1984
).
However, the sole presence of a visual target at fixation position in
our study cannot explain our findings because in the dynamic trials as
well as in the static trials, the target at fixation was always
present; the only difference was the absence of an ongoing pointing
movement. Also the size of saccadic delay observed in this study was
more than three times as large as in Walker's study. Nevertheless the
underlying mechanisms might be related, resulting in similar
characteristics in saccadic RT increase, as the independence of the
saccadic vector and its modulation with the distance between
"inhibitor" and ocular fixation position. The inhibitor was a
target being pointed at, close to fixation position, in our study,
equivalent to the onset of a distractor close to current fixation in
the reports of Walker et al. (1997). Furthermore the
resemblance with saccadic delay effects that, in the study of
Walker et al. (1997)
, were induced visually is an
indication that the underlying mechanisms could be related to visual
selection processes (or attention). When the hand was approaching the
fixated target, maximum visual control of homing in was possible,
probably restricting visual processing, or attention, elsewhere in the
visual field. When the coupling of saccades and visual attention is
obligatory and hand and eye movements share the same attentional
mechanisms (as shown in Deubel et al. 1998
), this could
prevent saccadic preparation during homing in on the fixated target.
Recently, Bekkering et al. (1996)
found that the gap-effect is present for goal-directed hand movements as well, providing more evidence that saccades and hand movements rely on
similar attentional mechanisms. When the pointing target was not
fixated, visual processing might have been relatively free to shift to
a new saccadic target position because on-line foveal visual control of
pointing was reduced. This would yield a weaker inhibition of saccades
as observed when the fixation and pointing targets were dissociated.
Physiological correlates of eye-hand coordination
There are various neurophysiological findings that shed some light
on the organization of ocular behavior, i.e., saccades and fixation.
The SC in the primate midbrain is known to be involved closely in the
generation of saccades (Robinson 1972; Schiller and Koerner 1971
), smooth pursuit (Krauzlis and Mile
1998
), and head movements (Cowie and Robinson
1994
; Segraves and Goldberg 1985
). In the
cerebral cortex of primates, exist at least two oculomotor fields with
similar saccadic and fixation-related activity as observed for the SC,
the frontal eye fields (FEFs) in the lateral surface, directly and
bilaterally connected with the SC (Segraves and Goldberg
1987
; Sommer and Wurtz 1998
), and the
supplementary eye fields (SEFs) in the medial edge of the frontal cortex.
The superficial and intermediate layers of the SC contain cells that
are sensitive to multimodal sensory signals, including somatosensory
(Groh and Sparks 1996; Stein et al.
1976
), auditive (Jay and Sparks 1987
),
and visual information (Goldberg and Wurtz 1972a
;
Humphrey 1968
), with retinotopic receptive fields. In
the intermediate layers, mainly sensorimotor buildup neurons are found that increase their firing rate shortly before a saccade with a
particular vector (amplitude and direction), also referred to as the
movement field. These neurons lose their premotor activity when no
target is present within this movement field (Goldberg and Wurtz
1972b
). The deep layers' neurons increase their activity shortly before a saccade with a particular movement field, irrespective of the presence of a target; i.e., they will also be active for saccades in dark (Schiller and Koerner 1971
;
Sparks 1986
; Wurtz and Goldberg 1971
).
The movement fields are aligned in an anatomic map; neighboring neurons
have movement fields for saccades with neighboring vectors. The
position of rostral SC neurons in this map imply a premotor activity
associated with small saccades or ocular fixation, which is indeed
observed here (Munoz and Wurtz 1993a). Electro stimulation of neurons in the rostral SC inhibits saccades or forces
saccades to halt (Munoz and Wurtz 1993b
).
Interestingly, Werner et al. (1993, 1997a
,b
) and
Stuphorn et al. (1999
, 2000
) reported activity in deep
SC before and during arm movements as well. Macaque monkeys performed a
saccade-reaching task to a target, both movements separated in time and
triggered by cues. SC neurons that exhibited reach activity also could
obtain visual (6%) or saccadic components (3%) or both (10%). Werner et al. discussed that SC reach cells play a role in the direct control
of arm-movements and/or the interaction of arm movements with other
motor systems in which the SC is involved (including saccades) by means
of existing projections from skeletomotor-related cortical areas
[possibly arm movement activity from the (pre)motor cortex] (see
Fries 1984
, 1985
) into SC. For a detailed functional and
anatomic discussion, see Werner et al. (1997a
,b
). The
inhibition of saccades during pointing, as observed in this study, is
an interaction that could arise at the level of SC. If rostral SC (fixation) cells have considerable hand-movement components, they might
inhibit a saccade that should be executed during a hand movement,
irrespective of the saccadic vector. This independence of the saccadic
vector is indeed found for the saccadic delay during pointing of this
study. However, a deep layer motor activity in the SC can inhibit only
the execution of saccades by increasing rostral SC
fixation cell activity. Therefore the observation that saccades had
normal residual RTs (time from hand-movement offset to saccade
execution) if a saccadic target was presented during arm movements
(experiment 1) cannot be explained completely by such a
motor interaction, which would have yielded saccades that are executed
immediately after pointing stops. That is, also saccadic preparatory
processes involving perception could have been inhibited during arm movements.
The FEF is known to be involved in the control of saccades as well
(Bruce and Goldberg 1985; Robinson and Fuchs
1969
), and, more specifically, is responsible for
selection of competing visual stimuli as a target for a saccade
(Schall and Hanes 1993
). The SC long has been thought to
be responsible for more reflexive saccades to a single target
(Schiller et al. 1987
), although it should be noted that
Glimcher and Sparks (1992)
showed that target selection
processes also can be observed in primate SC. A possible way for the
FEF to anchor ocular gaze to a pointing target, as observed in this
study, would be that a reach target somehow gets a higher priority in
the FEF as a nonreach target and delays the buildup of a motor program
for a saccade until the pointing is over. Unfortunately, no such
influences of hand movements on FEF neuronal activity have been
observed so far, although attempts have been made (Mushiake et
al. 1996
). In the SEF, however, also known to be involved in
the generation of saccadic eye movements (Schlag and Schlag-Rey
1985
, 1987
), the same study (Mushiake et al.
1996
) observed that a neuron belonging to a subpopulation of
~50% (of all observed neurons in FEF) is preferentially active either for a single saccade and not for a saccade-reach task, or
vice-versa, and the other 50% of the neurons is equally active in both
tasks. The SEF therefore also can play a role in the coordination of
eye and hand movements.
Cell recording under similar conditions as in this study could clarify the proposed role of neurons in the SC, FEF and the SEF on the phenomenon of saccadic inhibition during pointing movements that is reported in this study.
![]() |
ACKNOWLEDGMENTS |
---|
The authors thank J. Garbotz for assistance in conducting the experiments, F. Banci and K.-H. Honsberg for technical support, and S. Jordan and two anonymous reviewers for helpful comments on an earlier version of this paper. The authors thank Dr. Veit Stuphorn and Prof. Dr. Klaus-Peter Hoffmann from the Neurobiology and Zoology department at the University of Bochum, partners in the same project, for helpful discussions with respect to neurophysiology and interpretations of their recent data on single cell recordings.
This study was supported by Grant BE 1873/2 from the German Research Foundation (DFG) as part of the interdisciplinary project "Sensorimotor Integration."
![]() |
FOOTNOTES |
---|
Address reprint requests to: S.F.W. Neggers, Max-Planck-Institute for Psychological Research, Dept. of Cognition and Action, Amalienstrasse 33, D-80799 Munich, Germany.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
Received 3 June 1999; accepted in final form 1 September 1999.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|