1Center for Neuroscience and Section of Neurobiology, Physiology and Behavior, University of California, Davis, California 95616; and 2Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland 20892
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Recanzone, Gregg H. and Robert H. Wurtz. Shift in Smooth Pursuit Initiation and MT and MST Neuronal Activity Under Different Stimulus Conditions. J. Neurophysiol. 82: 1710-1727, 1999. The activity of neurons in extrastriate middle temporal (MT) and medial superior temporal (MST) areas were studied during the initiation of pursuit eye movements in macaque monkeys. The intersecting motion of two stimuli was used to test hypotheses about how these direction- and speed-sensitive neurons contribute to the generation of pursuit. The amplitude and direction of the initial saccade to the target and the initial speed and direction of pursuit were best predicted by a vector-average model of the underlying neuronal activity with relatively short time and spatial separation before a visual pursuit target and a distracter stimulus crossed in the visual field. The resulting eye movements were best described by a winner-take-all model when the time and spatial separation between the two stimuli was increased before the stimuli crossed. Neurons in MT and MST also shifted their activity from that best described by a vector average to a winner-take-all model under the same stimulus conditions. The changes in activity of neurons in both areas were generally similar to each other during these changes in pursuit initiation. Thus a slight alteration in the target motion produced a concurrent shift in both the neuronal processing and the movement execution. We propose that the differences in the oculomotor behavior can be accounted for by shifts in the overlap of active neuronal populations within MT and MST.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
A central question in neuroscience is how neurons
in the brain represent the sensory-motor events that determine the
metrics of movements. This representation in the primate brain has been found to lie in the activity of a population of neurons that are active
before a range of movements. Such population coding has been observed
in the primate motor cortex during hand-reaching movements
(Georgopoulos 1996; Georgopoulos et al.
1986
) and in the superior colliculus during saccadic eye
movements (Lee et al. 1988
; McIlwain
1991
; Munoz and Wurtz 1995
;
Sparks and Mays 1990
). In both cases, the model that
best represented the translation of neuronal activity into movement was
one that took the vector sum or average of the activity of each neuron
across the population (Georgopoulos et al. 1986
;
Lee et al. 1988
).
Such a population of active neurons, each tuned to a range of stimulus
parameters, also has been found in the sensory processing on which
movement frequently depends. Nowhere is this population activity
clearer than in the direction- and speed-selective neurons in the
extrastriate cortex of the primate, particularly the middle temporal
(MT) and medial superior temporal (MST) areas (Allman et al.
1985; Desimone and Ungerleider 1986
;
Maunsell and Van Essen 1983
; Tanaka et al.
1989
; Van Essen et al. 1981
). The use of this visual information to guide smooth pursuit eye movements is also well
established and is based on the change in activity of single neurons
during pursuit, the pursuit deficits following ablation, and the
alteration of pursuit by electrical microstimulation (Dursteler et al. 1987
; Groh et al. 1997
; Komatsu
and Wurtz 1988
, 1989
; Newsome et al. 1985
;
Schiller and Lee 1994
; Yamasaki and Wurtz
1990
). Although several different classes of models could be
used to predict the neuronal activity of these cortical areas and
pursuit behavior, the two models that have been most successful have
been the vector-averaging and winner-take-all models. The
vector-average models assume that all the vectors representing each
active neuron are averaged, whereas the winner-take-all models assume
that only the neurons with the highest activity contribute to the
movement generation (Ferrera and Lisberger 1995
, 1997
;
Groh et al. 1997
; Lisberger and Ferrera
1997
).
In testing which of these models best predict the observed pursuit
behavior, recent experiments have compared the pursuit generated when
one stimulus moved across the visual field with that produced when two
stimuli moved across the visual field. The introduction of the second
stimulus tests how the underlying neuronal populations could interact
to produce the metrics of the initial eye movement: do they combine
(vector averaging) to produce an averaged eye movement or does one
population predominate (winner take all) to produce an eye movement
similar to that when only one stimulus is presented. Experiments using
two visual stimuli (Lisberger and Ferrera 1997) or one
stimulus coupled with electrical stimulation of MT (Groh et al.
1997
) found that a vector-averaging model best predicted the
pursuit behavior. Similarly the responses of MT and MST neurons were
best predicted by the vector-average model of two stimuli moving
through the receptive field in fixating monkeys (Recanzone et
al. 1997
). However, in another experiment using a different
two-stimulus task, pursuit was better predicted by a winner-take-all
model (Ferrera and Lisberger 1995
, 1997
).
In the present experiments, we directly investigated the activity of neurons in MT and MST that form the basis for these model predictions. We first verified the alterations in pursuit behavior in the presence of two moving stimuli. By increasing the time and spatial separation of two moving stimuli before the onset of pursuit, we found that the best prediction of the pursuit behavior shifted from a vector average to a winner-take-all model. We also found that the activity of neurons in MT or MST also shifted from a vector-average to a winner-take-all model as the stimulus conditions changed. We propose that the behavioral changes are consistent with shifts in the overlap of active neuronal populations on a conceptual velocity map of MT and MST.
A brief report of this study has appeared previously (Recanzone
et al. 1993).
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The same two monkeys used in these experiments also were studied
in a previous experiment (Recanzone et al. 1997), and
the experimental procedures were the same. All procedures were approved by the Institute Animal Care and Use Committee and complied with Public
Health Service policy on the humane care and use of laboratory animals.
Behavioral tasks
Two adult male rhesus monkeys (Macaca mulatta) were used in this study (monkeys N and P). Each monkey was trained to perform several versions of a pursuit eye movement task, and the two tasks used in most experiments are illustrated in Fig. 1. Each trial was initiated when the monkey looked at a central fixation stimulus (fs) that was one of two different shapes (Fig. 1A). The shape of the fixation stimulus cued the monkey as to which of two moving stimuli it was required to acquire with a saccade and track with a smooth pursuit eye movement to obtain a liquid reward. The monkey maintained fixation for a random interval between 100 and 500 ms (Fig. 1B) to within ±2° of the center of the fixation stimulus. On all trials, two visual stimuli were then presented in motion in one of two possible stimulus configurations. Because all of the behavioral results described in this report were collected while recording from a single neuron in either cortical area MT or MST, the stimulus conditions were set so that at least one of these two stimuli always moved through the receptive field (RF) of the neuron under study (Fig. 1, - - -). Trials consisted of either one stimulus moving through the RF of the neuron and the other stimulus moving near the same horizontal eccentricity in the other visual hemifield (referred to as uncrossed trials; Fig. 1, left) or the trials consisted of two stimuli moving through the RF of the neuron in which one stimulus moved in the best direction of the neuron and the other stimulus moved in one of seven different, non-best directions (referred to as crossed trials; Fig. 1, right).
|
The monkey had to maintain fixation until the fixation point was extinguished 150 ms after the onset of the motion of the two stimuli (Fig. 1D). The monkey was rewarded for making a saccade to the visual stimulus that was the same shape as the fixation stimulus and then accurately pursuing that target stimulus for 300-900 ms (Fig. 1E). The initial saccade had to be made within 450 ms of the fixation stimulus offset (Fig. 1D) and had to be within 10 deg of the target stimulus. The eye position also had to be within ±5° of the target during the pursuit phase of the trial starting 300 ms after the initial saccade. These windows were necessarily large due to the size of the stimulus objects and the accuracy of the initial saccade during the crossed trials as described in RESULTS. On uncrossed trials, one stimulus was located at the center of the RF at the time that the fixation stimulus was extinguished, and the stimulus in the opposite visual field was located at the mirror image location. On crossed trials, both stimuli were at the center of the RF at the time that the fixation point was extinguished and were superimposed with the same luminance as a single stimulus. This gave the perception to human observers of two superimposed opaque objects moving across the visual field and crossing through each other at the location corresponding to the center of the RF. These two trial types allowed us to determine the effects of a distracter stimulus by comparing the eye movements and single-neuron activity in trials where the target was the only stimulus moving through the RF with trials where the distracter stimulus was also moving through the RF. In both cases, the motion of the target stimulus was the same, and the monkey had to make the same shape discrimination.
A third and fourth set of trial types, termed the long-duration trials, were identical to the crossed and uncrossed trial types described in the preceding text except that both visual stimuli initially were displaced in space such that 450 ms elapsed before the fixation point was extinguished after stimulus motion onset, and at that point the stimulus objects were at the same location as in the first two tasks (e.g., at the center of the RF; Fig. 1D). These trials provided a greater separation in space between the two stimuli and allowed the monkey an additional 300 ms to perform the shape discrimination before the stimuli reached the RF of the neuron under study. Again, the starting location of the stimuli was dependent on the response properties and the RF size of the neuron under study on that session. The stimulus velocity on the long-duration trials was the same as for the short-duration trials, resulting in different distances in the starting locations of the stimuli between sessions, but the stimuli always reached the center of the RF 450 ms after stimulus motion onset.
Stimuli were moved in eight different directions (0, 45, 90, 135, 180, 225, 270, and 315°). For uncrossed trials, the stimulus that was the target could move in any of the eight directions in any trial as could the stimulus in the other visual hemifield. For the crossed trials, the target stimulus also could move in any of the eight directions and the distracter moved in one of the other seven directions. All of these stimulus conditions (crossed trials, uncrossed trials, different target directions) were presented on randomly interleaved trials, and when long-duration trials were used, they also were introduced with the other trial types in randomly interleaved order. A complete data set consisted of at least eight correct trials for each stimulus type, although most of the data of this report are from 10 to 15 correct trials for each stimulus type.
Several factors that were checked in an initial analysis of the
behavior were found not to be statistically significant and will not be
considered further. We found no significant differences, or apparent
trends, between either the eye metrics or the neuronal responses on the
short-duration trials when they were interleaved with long-duration
trials and when they were not. The behavioral performance for the
short-duration trial types were not significantly different between the
two monkeys nor was there any significant difference in the percent of
correct trials (trials in which the target was accurately tracked for
300 ms after the initial saccade) depending on which stimulus was the
target for the eye movement (repeated measures ANOVA, all P
values > 0.1). The best performance was for both the short- and
long-duration uncrossed trials (86% correct), and the lowest
performance was for crossed trials (77%). These behavioral
observations are in good agreement with previous studies using similar
one- and two-stimulus configurations while the monkey performed a speed
discrimination task (Treue and Maunsell 1996).
Monkey N completed 80 sessions and monkey P completed 65 sessions of the short-duration task in which both crossed and uncrossed trials were presented (Fig. 1). For the long-duration task, monkey N completed 60 sessions, but monkey P frequently was unable to reliably suppress the initial saccade for a sufficient period to complete at least eight trials under all conditions for the long-duration trials (only 7 sessions with only 3-7 successful trials per stimulus per session). Although these limited data were consistent with those obtained in monkey N, this dataset is not sufficiently extensive to report.
Visual stimuli
Visual stimuli were back-projected onto a tangent screen placed 57.4 cm from the monkey using a video projector (Electrochrome, SVGA, 1024 × 768 pixel resolution). Each pixel subtended a visual angle of 0.13° horizontally and 0.12° vertically. Images were created by a PC and were presented at a frame rate of 72 Hz. Stimulus objects were brighter (1.8 cd/m2) than the background (0.2 cd/m2). Two of five different objects (circle, square, diamond, plus sign, and triangle) were used in each behavioral session, with each object of equal luminance and size (same number of pixels/object) subtending a maximum visual angle of 1.8°. Objects were moved across the screen by displacing each illuminated pixel by 1 or 2 pixels each frame in either the horizontal, vertical, or both directions. Three stimulus speeds were produced by pixel shifts of 1, 1.5, or 2 pixels/frame corresponding to ~9, 13.5, or 18°/s along the horizontal and vertical directions and 12, 18.5, or 25°/s along the obliques. For speeds using 1.5 pixels/frame, all stimulus pixels were displaced by 1 and 2 pixels on alternate frames. These stimulus motions were perceived by human observers as continuous, smooth motion.
Data collection procedures
Each monkey underwent magnetic resonance imaging (MRI) scans in
the saggital and frontal planes to aid in aligning the recording cylinders and in reconstructing the electrode penetrations. Under general anesthesia, a scleral search coil was implanted in each eye and
a restraining head post was implanted on the skull using dental acrylic
following procedures described in Duffy and Wurtz (1995). After behavioral training was completed, a recording
cylinder was implanted over the parietal cortex directly over cortical areas MT and MST in the stereotaxic vertical plane. Monkey P
had one cylinder and monkey N had a second cylinder
implanted over the opposite hemisphere after recordings from the first
hemisphere were completed.
Horizontal and vertical eye position was recorded using the magnetic
search coil technique (Fuchs and Robinson 1966;
Judge et al. 1980
). Eye position was digitized at 500 Hz
with a resolution of 0.1°. Data acquisition and control of the video
display computer was controlled by a real-time experimental system
(REX) (Hays et al. 1982
) and run on a PDP11/73. Both
target and eye position were monitored on-line as well as stored for
off-line analysis.
Tungsten microelectrodes were advanced into the region of the superior
temporal sulcus through guide tubes positioned in a grid within the
recording cylinder (Crist et al. 1988). Neuronal signals
were amplified, filtered, and displayed on an oscilloscope and audio
monitor using conventional methods. Single neurons were isolated using
a time and amplitude window discriminator (Bak). The occurrence of each
neuronal spike was digitized and time stamped at 1 kHz and stored for
off-line analysis.
Neurons recorded in this study met the same three criteria described
previously (Recanzone et al. 1997): the activity of the neuron was altered by the presence of visual stimuli (but not necessarily moving stimuli), isolation was sufficient to be confident that only a single neuron was being recorded, and the center of the RF
was between 5 and 25° of eccentricity. This last requirement was
necessary as the monkeys were unable to maintain fixation if a target
or distracter stimulus moved through the fixation point before the
fixation point offset and the monkeys were unable to reliably make the
visual shape discrimination at eccentricities >25°.
RFs were defined using hand held bars, spots, and patterns of light. RF
edges were defined using both visual and audio criteria as locations
where the neuron no longer responded to moving, flashed, or stationary
visual stimuli. We categorized each neuron as being located within
cortical area MT or MST based on the location and depth of the
electrode within the recording cylinder relative to the MRI images and
the characteristics of the visual stimuli required to maximally
activate the neuron (Allman et al. 1985; Baker et
al. 1981
; Lagae et al. 1993
). MT neurons
responded best to small moving stimuli and had smaller RFs than MST
neurons. MST neurons usually responded better to larger moving stimuli such as patterns than to moving spots. We only rarely encountered neurons with very large RFs that responded best to large patterned stimulation similar to those described in the dorsal region of MST
(MSTd) (Komatsu and Wurtz 1988
; Tanaka and Saito
1989
; Tanaka et al. 1986
, 1989
, 1993
), and the
vast majority of our sample of MST neurons were characteristic of those
located in the lateral region (MSTl). However, it is possible that some
of the neurons in this study were located within MSTd.
The stimulus speed and shape used in a particular behavioral session
were based on the best responses recorded during preliminary characterization of the cell's tuning and RF properties and were adjusted so that the stimulus would traverse as much of the RF as
possible before reaching the center at 150 ms after motion onset on the
short-duration trials. RF sizes ranged from diameters of 5-24° for
MT neurons and 5-35° for MST neurons and increased in diameter with
increasing eccentricity as described previously (e.g., Desimone
and Ungerleider 1986; Komatsu and Wurtz 1988
). We noted no differences in the response properties of our sampled neurons when the stimulus onset was slightly within, at, or slightly outside the edge of the RF, and thus all data were pooled.
At the end of the experiments, histological sections through the
superior temporal sulcus were obtained as described previously (Recanzone et al. 1997). MT was identified on the
posterior bank by its dense myelination. Orientation for the general
region of MST was provided by the region of dense myelination on the
anterior bank. Drawings of the sections showed that the guide tubes
were directed toward the superior temporal sulcus and that the
electrode tracks passed through or below the densely myelinated area on the anterior bank (MST) and the posterior bank (MT). The categorization of each cell to MT or MST was based on the physiological criteria described above and was consistent with the cytoarchitectonic definition of these two cortical areas.
Data analysis and model predictions
Smooth pursuit metrics were measured for 40 ms starting 20 ms
after the completion of the initial saccade. This period was chosen as
it does not contain corrective saccades (Lisberger and Westbrook
1985) and follows the same analysis period reported recently by
Groh et al. (1997)
. The onset and offset of the saccades were defined based on the velocity, acceleration, and duration of the
eye movement using a saccade-detection algorithm. All accepted individual eye movements then were inspected visually to ensure that
only eye movements that were initiated between 100 and 450 ms after the
fixation point offset and that contained only a single saccade before
the pursuit measurements were included in this analysis.
The saccade amplitude was defined as the length of a vector (in deg) from the fixation point to the eye position 20 ms after the initial saccade. The saccade direction was defined as the direction of this vector. The initial pursuit direction was defined as the direction of a vector drawn from the eye position starting at 20 and ending at 60 ms after the saccade, and the initial pursuit speed (°/s) was defined by the length of this vector divided by 0.040 s.
The eye metrics (initial saccade amplitude and direction and initial
pursuit speed and direction) were compared with the predictions of
winner-take-all and vector-average models of the behavior. For these
two predictions, the eye metric was measured for each stimulus
direction on the uncrossed trials, and each was described as a vector
with both magnitude and direction. The winner-take-all model predicts
that the eye movement should follow one vector or the other when the
two stimuli were presented inside the RF, and we compared the eye
movement on each of the two uncrossed trials with the eye movement on
crossed trials when the same target stimulus was presented. To predict
the vector-average result, the vectors for the two stimuli presented on
the uncrossed trials were averaged, taking into account both the
direction and velocity of each eye movement; the average therefore
represents a true vector average and is not the linear average of the
two directions or speeds. We also compared the results with those
predicted by a vector sum model by summing the two component vectors.
The vector sum model was never a better predictor than either of the
other two models (as was also the case in previous experiments of
Ferrera and Lisberger 1995; Groh et al.
1997
; Lisberger and Ferrera 1997
), and we will
not consider the predictions of this model any further.
To determine qualitatively which model made the best prediction, the
initial pursuit speed and direction and the initial saccade amplitude
and direction were averaged across all correct trials for a given
stimulus condition. Each of these points across all sessions for a
given monkey then were subjected to regression analysis with two
dependent variables (Mosteller et al. 1983; see also
Recanzone et al. 1997
). For analysis of the direction data, we removed any artificially large differences in target directions as, for example, between 358 and 3°, which could be taken
as either a difference of 363 or 5°. When we compared the difference
between the measured and predicted directions, we always used the
difference that was smallest, that is, 5 rather than 363° in this
example. To quantitatively measure the difference in the predictive
power of the two models, the absolute value of the error between the
measured eye metric and that predicted by each of the three different
models was measured. These errors then were subjected to a paired
t-test to determine if one model consistently better
predicted the eye movement metric across stimulus conditions and
behavioral sessions.
For the responses of individual MT and MST neurons, the firing rate was
measured in the period from the onset of the moving visual stimuli to
the onset of the initial saccade (Fig. 1B). The firing rate
before the onset of the moving stimulus also was calculated
(spontaneous activity) and subtracted from the driven activity. For
each neuron, the best direction was defined as the stimulus direction
that gave the largest response during the uncrossed trials. The null
direction was defined as the direction 180° from the best direction.
As with the eye metric measurements, the winner-take-all prediction was
defined as a response equivalent to that of the target stimulus during
the uncrossed trials, where a single stimulus was moving through the RF
of the neuron. The vector-average model was predicted by the averaged
response for the two stimulus directions taken during the separate
uncrossed trials and thus is a scalar average of the responses (see
Recanzone et al. 1997). The errors between the predicted
and observed responses were calculated as a contrast ratio,
|(predicted
observed)|/(predicted + observed) because the
activity of individual neurons varied widely so that there were large
differences in the total activity (e.g., 10 spikes from neurons with a
peak activity of 12 spikes) and small differences (e.g., 10 spikes
where the peak activity was 80). The absolute value was used to prevent
positive and negative differences from summing near zero because
deviations of the model predictions of either sign were equally important.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Our major observation was that the presence of a distracter stimulus that crossed the trajectory of the target stimulus affected both the metrics of the initial pursuit eye movement and the responses of single neurons in areas MT and MST. The eye movement metrics and the neuronal activity were consistent with either a vector-averaging or winner-take-all models of the activity of the MT and MST neuronal population, but which model was most appropriate changed between the short- and long-duration trials. We first will describe the changes in the initial pursuit eye movements, then compare these behavioral observations with the predictions of the vector-average and winner-take-all model, and finally describe the changes in single-neuron activity recorded as the pursuit behavior changed in comparison with the activity predicted by the same models.
Eye-movement metrics
In each experiment, we compared the pursuit initiation with only one stimulus in each visual hemifield with that with two stimuli in the visual hemifield and varied the time and distance over which the stimuli traveled in each condition. Figure 2 shows the eye movements from one experiment recorded during these four stimulus conditions with a target that was in the left hemifield and moved leftward. The two columns in Fig. 2 show the eye movements to the same target stimulus on the uncrossed trials (left -when the 2 stimuli were in opposite visual fields) and crossed (right -when the 2 stimuli crossed in the same visual field; see METHODS and Fig. 1). The rows in Fig. 2 show the effect of changing the duration and distance of stimulus motion from a relatively short-duration and stimulus path (top) to a relatively long duration and length (bottom). For brevity, we will refer to these spatial-temporal changes in the stimulus motion across trials as short- and long-duration trials. The time at which the stimuli crossed the center of the field are aligned vertically (Fig. 2, A and C and B and D) so that the target stimulus trajectories during the last 700 ms are aligned on the time axis.
|
On the short-duration trials (Fig. 2, A and B), the eye movements on the uncrossed trials (Fig. 2A) were characterized by an initial saccade that brought the eye close to the target and an initial pursuit movement where the eye immediately moved at nearly the same speed and direction as the target with occasional corrective or catch-up saccades. On crossed short-duration trials (Fig. 2B), the initial saccade was not to the target (solid gray line) and the initial direction and speed was not matched to the target direction and speed as indicated by the relatively flat portions of the eye traces immediately after the first saccade. Normal pursuit was not generally resumed until after the first corrective saccade, and by that time, the two stimulus objects had moved away from each other. Thus the uncrossed case shows the usual pattern of saccades and pursuit for a single target, whereas the crossed case shows a saccade directed between the target and the distracter coupled with a reduced initial pursuit speed.
On the long-duration trials (Fig. 2, C and D), where the period of time before the release from fixation was extended 300 ms, the eye movements on the uncrossed trials (Fig. 2C) were similar to the short-duration uncrossed trials (Fig. 2A). For the crossed long-duration trials (Fig. 2D), the initial pursuit eye movements were much more like those on the uncrossed trials than the short-duration crossed trials. The eye movements were now closer to the target and generally had a direction and speed similar to the target. Thus there was a significant alteration in the initial saccade and speed of pursuit for the short-duration but not the long-duration crossed trials compared with the uncrossed trials.
To see whether these changes in pursuit and saccades were consistent on
a trial-by-trial basis, we analyzed the eye movements across all 145 behavioral sessions for both monkeys. We first normalized the eye
metrics in each session to allow for the different target directions,
speeds, and eccentricities that resulted from tailoring the stimuli to
the RF properties of the neuron under study. To do this normalization
for the trials in each session, we first took the average of all
correct eye movements for uncrossed trials where the target stimulus
moved in the best direction of the neuron under study and set the value
of the horizontal and vertical components of the averaged eye metrics
(initial pursuit speed and direction or initial saccade amplitude and
direction) to +1.0. We then did the same for the uncrossed trials where
the stimulus moved in the direction opposite to the best direction and
assigned these horizontal and vertical components a value of 1.0.
These eye movements to the single target were generally very accurate
so that these averages gave good estimates of the monkey's ability to
follow the target moving in a given direction and speed in that
session. We then measured the horizontal and vertical values of these
same eye movement metrics in the crossed stimulus trials and plotted
these values on the axes based on the normalized values.
Figure 3 shows a plot of initial
horizontal and vertical pursuit speed for sample trials from each of
the 145 sessions and compares this pursuit on short-duration (Fig.
3A) with those on long-duration (Fig. 3B)
trials. All the plots in Fig. 3 show the initial eye speeds on single
uncrossed trials in which the target was moving in one direction (,
the best direction of the neuron being studied), the eye movement on
single uncrossed trials when the target was moving in the opposite
direction (
), and the movement on single crossed trials in which one
stimulus moved in the best direction and the other moved in the
opposite direction (+; or at ±90° from that direction.). The two
different models make very different predictions for the crossed
trials. The vector-average model predicts that the points should
cluster near 0,0, whereas the winner-take-all model predicts that the
points should cluster at the same regions as in the uncrossed trials,
i.e., near (1,1) and (
1,
1). For short-duration trials (Fig.
3A), the pursuit-initiation speeds for single target motion
(
and
) clustered into the two areas in the top right
and bottom left quadrants appropriate for pursuing the two targets moving along the best and opposite directions. When two targets were present (+), the initial pursuit speeds were
clearly different and had much lower initial speeds in both the
horizontal and vertical directions. In contrast, for the long-duration trials (Fig. 3B), the eye movements tended to match the
speed of the target stimulus in the crossed trials as well as the
uncrossed trials regardless of the direction of the target motion (+ cluster near both
and
). Although there was clear clustering of
the responses into particular regions, it is also clear from Fig. 3
that some eye movements fell in locations well away from the three
predicted locations. This was due to the fact that this plot
illustrates a randomly selected subset of trials, including those in
which the monkey made a large error and did not receive a reward. This
shows that across the behavioral sessions, the monkey's initial
pursuit speed on long-duration crossed trials was similar to that on
uncrossed trials, whereas on short-duration crossed trials, the speed
was reduced on nearly every trial.
|
We plotted the initial pursuit direction as well as the initial saccade amplitude and direction on plots similar to those in Fig. 3. Pursuit direction also changed on short-duration but not long-duration trials as can be inferred from the plots in Fig. 3 because any such change in direction is indicated by a change in the ratio of horizontal and vertical eye speed. Saccade direction and amplitude showed the same distributions, with crossed trials clustered between the two distributions shown for the uncrossed short-duration trials and separated into two distinct clusters on crossed long-duration trials. Therefore changes in the eye movement metrics were consistent across these sample trials taken from all of the sessions in spite of the varying stimulus conditions necessitated by the requirements of the individual neurons under study.
We extended this analysis to all of the trials across all sessions by determining the error for each eye metric which was taken as the difference between the eye metric in the crossed trials and the same metric on uncrossed trials. Figure 4 shows the distribution of these errors for pursuit speed for the best direction on short-duration uncrossed trials (heavy solid line), short-duration uncrossed trials in the null direction (heavy dashed line), and crossed trials where the target moved in the best direction and the distracter moved in the null direction (thin line). The abscissa in Fig. 4 indicates the radial error for the combined differences in horizontal and vertical speeds that are normalized as in Fig. 3. If the eye speed matched the speed of the target moving in the best direction, the error should be near 0 as it is for pursuit of that target in the uncrossed trials (heavy solid line in Fig. 4). If the speed matched that of the target moving in the null direction as is the case when the monkey pursues the target moving in the null direction on the other uncrossed trials (heavy dashed line), the value should be 2.64 (right arrow in Fig. 4; see legend). The thin solid curve in Fig. 4 shows the errors observed when the monkey pursued the target moving in the best direction on the crossed trials, and these errors are centered near the midpoint as expected (1.41; left arrow). Each distribution is clearly unimodal and largely nonoverlapping, indicating that the eye movements to these different stimuli were consistent across trials. We found the same pattern of results for the initial pursuit direction, saccade amplitude, and saccade direction (not shown).
|
This unimodal characteristic of the pursuit speed error shown in Fig. 4
and the unimodality of the curves for the other metrics shows that the
eye movements were consistent across trials for a given stimulus
condition. This unimodal curve rules out the possibility that the
monkey made saccades on crossed stimulus trials to the target on some
trials and the distracter on others because if that was the case, the
distribution of errors in the crossed stimulus trials would be bimodal.
Because the eye movement distribution in crossed trials (such as for
speed in Fig. 4) was clearly unimodal, we can be confident that
averaging across trial types for each session is justified, consistent
with previous reports of pursuit eye movement metrics with two stimuli
present (Ferrera and Lisberger 1995, 1997
;
Lisberger and Ferrera 1997
).
Finally, it is clear from the eye movement traces in Fig. 2 that there
was generally an increase in the saccade latency on the crossed trials
compared with the uncrossed trials, and previous studies also indicated
that a distracter stimulus can increase the latency by as much as 50 ms
(Ferrera and Lisberger 1995). To determine if a similar
effect occurred here, we compared the initial saccade latency between
the uncrossed and crossed trial types when the target was moving in the
same direction across sessions for both monkeys. The initial latency
was defined as the time period between the fixation point offset and
the initiation of the saccadic eye movement used to acquire the target.
There was a modest but statistically significant increase in saccade latency between the two trial types for the short-duration stimuli (unpaired t-test; P < 0.01) of ~25 ms for
monkey N and 42 ms for monkey P. For the
long-duration stimuli, however, there was no statistically significant
difference in the one monkey (N) tested.
It had been noted previously, however, that the increase in latency was
largely confined to conditions in which the distracter moved in the
opposite direction to the target, whereas a decreased latency occurred
when the distracter moved in the same direction as the target
(Ferrera and Lisberger 1995, 1997
). To test this possibility, we restricted the analysis to trial conditions in which
the distracter moved either at 180° from the target or at ±45° of
the target, which was as close as our stimuli came to both moving in
the same direction. This analysis did show that the trials with the
longest latencies tended to be those with the stimuli moving at 180°
of each other and that the shortest latencies tended to be those with
the stimuli moving within 45° of each other, but this difference
between groups did not reach statistical significance (1-tailed,
unpaired t-test; P = 0.073).
Eye-movement metrics and model predictions
One of our major goals was to determine if the eye metrics were
predicted more closely by the vector-average or the winner-take-all model of the underlying neuronal activity. Both models make specific predictions, and Fig. 5 shows how these
predictions differ. The top panel shows the stimulus
conditions for the uncrossed (left) and the crossed
(right) trials when the two stimuli move at 90° to each
other. The possible resulting saccadic (- - -) and pursuit () eye
movements are shown below each stimulus configuration. For both models,
the eye movement on the uncrossed trials should accurately follow the
target stimulus (middle left) but not the distracter
stimulus (bottom left). For the crossed trials, the winner-take-all model predicts that the eye movements will be identical
between the crossed and the uncrossed trials when the target is pursued
correctly or quite different if the monkey pursues the distracter (Fig.
5, right). The vector-average model predicts that the
initial eye movement will be in a different direction and at a lower
speed on the crossed trials compared with the uncrossed trials (Fig. 5,
bottom right).
|
To compare these predictions to the behavioral observations, we first
found the mean of each of the eye movement metrics (initial pursuit
speed and direction and initial saccade amplitude and direction) for
each trial type in all sessions. We then performed regression analysis
between the eye movement metrics measured on the crossed trials and the
winner-take-all and vector-average predictions. Figure
6 shows this type of qualitative
comparison across all sessions for monkey N using the
initial direction of pursuit as an example. Figure 6A shows
the clear relation between pursuit and target direction on uncrossed
trials for both short-duration (left) and long-duration
(right) trials and serves as a reference for the subsequent
comparisons. Figure 6B compares the initial pursuit
direction observed on the crossed trials with that predicted by the
winner-take-all model when the monkey pursued the target moving in the
same direction as in 6A. For this analysis, it was necessary
to transform upward eye movement directions to prevent large apparent
differences due to the circularity of this measure (see figure legend
and METHODS), which is why the y-axis ranges from 90 to 450°.
|
The results of this analysis indicate that the winner-take-all model was a better predictor of pursuit behavior on the long- than the short-duration trials. This relation was not due to the monkey making eye movements to the distracter stimulus, however, because the relationship between the observed eye movement and the direction of the distracter stimulus showed even greater scatter (Fig. 6C). When the observed pursuit direction was compared with the prediction of the vector-average model (Fig. 6D), the vector-average model better predicted the eye movement on the short-duration trials than on the long-duration trials.
These differences between the observed pursuit and the two model
predictions were quantified by directly comparing the errors between
the observed and predicted eye movement metrics of the two models
against each other for both monkeys (Fig.
7). In these plots, points falling below
the indicate a smaller error by the vector-average model than by the
winner-take-all model and points above the
indicate a smaller error
by the winner-take-all model than by the vector-average model. Errors
for the initial pursuit direction for both monkeys (Fig. 7A)
showed that the majority of the points fell below this line for the
short-duration trials (left), indicating that the
vector-average model more closely predicted the observed data. In
contrast, on the long-duration trials the majority of points fell above
the line, indicating that the winner-take-all model more closely
predicted the observed data. These differences were statistically
significant for both monkeys on the short-duration trials and for the
long-duration trials in monkey N, the only monkey tested
(P < 0.01, paired t-test; see Table
1). Three other measures of the initial
eye movement showed the same result. The initial pursuit speed (Fig.
7B), the amplitude of the initial saccade (Fig.
7C), and the direction of the initial saccade (Fig.
7D) all showed smaller errors for the vector-average model
on the short-duration trials and a smaller error for the
winner-take-all model on the long-duration trials. These differences
were also significant (Table 1).
|
|
This correspondence between the experimental data on the short-duration crossed trials and the vector-averaging model also raises an issue related to the strategy the monkey may have been using in these experiments. If the monkey adopted the strategy of making the initial saccade to a location near the intersection of the two stimulus objects, it then could use near foveal vision to make the appropriate shape discrimination and then make subsequent saccadic and pursuit eye movements to follow the target. The most compelling evidence against this possibility is that there was a significant pursuit speed immediately after the initial saccade whether the motion of the two stimuli was different by 45, 90, or 135°. This is shown by the smaller, but nonzero eye speeds of Fig. 3 and the small errors of the vector-average prediction in Fig. 7B. If the monkeys were adopting the preceding strategy, they would not generate a pursuit movement to a nonexistent stimulus that was consistent with the vector-average motion of the two stimuli. We therefore conclude that the vector average is more likely reflecting the representation of the visual stimuli than a change in the monkey's oculomotor strategy in the crossed trials.
We also compared the initial saccade metrics directly to the initial
pursuit metrics. For the short-duration crossed trials in particular,
it may be that there was a difference between these two components of
the eye movement. For example, it may be that the initial saccade was
made appropriately, but the initial pursuit eye movement followed the
vector average of the two stimuli. Inspection of Fig. 7 suggests that
this is unlikely, as both the initial saccade direction and amplitude
more closely followed the vector-average model on the short-duration
trials and the winner-take-all model on the long-duration trials. To
verify this statistically, we measured the error between the eye metric
(initial saccade direction and amplitude, pursuit direction and speed)
for all correct trials in both monkeys and subjected these data to
factor analysis. If the initial saccade metrics and pursuit metrics had
different errors, this should result in these data being composed of at least two factors. We found that the data were best represented by one
factor for each of the four trial types, indicating that across trials
the metrics of the initial saccade and pursuit eye movements covaried
with each other. From this analysis, we conclude that both the saccadic
and pursuit eye movements were affected equivalently. This similarity
of smooth pursuit and saccades is also consistent with the deficits in
both pursuit and saccades made to moving targets following localized
lesions of MT (Newsome et al. 1985; Schiller and
Lee 1994
); both the amplitude of the initial saccade and the
pursuit speed to a moving target were incorrect following the lesion.
Finally, we considered the possibility that the stimulus velocity could alter the distribution of errors. We used three different velocities, tailored to the best response of the neuron under study (see METHODS). The majority of our stimuli were at 2 pixels/frame to have the stimuli traverse as much of the RF of the neuron as possible, as most neurons had RFs between 10 and 20° in radius (72/145 experiments), the second most commonly used velocity was 1.5 pixels/frame (52/145 experiments) with 1 pixel/frame (21 experiments) being the least common. An ANOVA test showed no statistically significant effect of velocity on any metric for either the short- or long-duration trials or in either monkey (all P values > 0.05), indicating that stimulus speeds within the range of 8-25°/s resulted in similar effects on the eye movement metrics.
In summary, the analysis of the errors indicated that the eye movements were best predicted by a vector-averaging model when the two stimuli intersected on short-duration trials but that a winner-take-all model best predicted the eye movements on long-duration trials. Thus by only slightly increasing the spatial distance and separation timing of the stimulus motion, it was possible to convert the eye metrics to identical stimuli from being more closely predicted by a vector-average model to a winner-take-all model.
Neuronal responses in MT and MST
The activity of single neurons in cortical areas MT and MST were recorded at the same time as the measurements of the initial eye movements, and Fig. 8 shows an example of the responses of an MT neuron recorded during the same trials illustrated in Fig. 2. On the uncrossed trials (left), this MT neuron gave a robust response while the stimulus was moving in the best direction within its RF. The neuron had a particularly large onset response on the short-duration trials (Fig. 8, A and B), as the stimulus appeared in motion at the edge of the RF. In contrast to this robust activity to single stimuli, there was a significant reduction in the response on the short-duration trials when a distracter moved through the RF in the null direction at the same time the target moved through the RF in the best direction (Fig. 8B). This reduction in the response was not nearly as apparent for the long-duration crossed stimulus condition (Fig. 8D), although there was still a reduced response for crossed trials compared with the uncrossed trials (Fig. 8, D compared with C).
|
For each neuron, we calculated the ratio of the response during the crossed and uncrossed trials (crossed trial response/uncrossed trial response). The example neuron shown in Fig. 8 had a ratio of 0.61 for the short-duration trials and 0.89 for the long-duration trials. Across the sample of neurons in MT, the mean and standard deviation of the ratio for the short-duration trials was 0.63 ± 0.32 and for long-duration trials the population average was 0.96 ± 0.32. For MST neurons, these values were 0.66 ± 0.31 for short-duration trials and 0.89 ± 0.32 for long-duration trials. This indicates that the difference in responses between the short- and long-duration trials under these two stimulus conditions was fairly uniform across the population of neurons in each area and that there was little difference between areas MT and MST (unpaired 1-tailed t-test; P > 0.05 for both short- and long-duration trials).
To compare the neuronal activity to the previous measures of the
initial eye-movement metrics, we first determined whether the responses
of the MT and MST neurons also were better predicted by the
vector-average model or the winner-take-all model depending on the
stimulus conditions. As we did for the eye movements (Fig. 6), we
compared the observed neuronal responses with those predicted by the
winner-take-all and vector-averaging models. These comparisons are
shown in Fig. 9 for
all neurons recorded in area MT in monkey N. This figure
shows the responses when the target for the eye movement was in the
best direction (Fig. 9, A and B, ) and the responses when the target for the eye movement was in one of the seven
non-best directions (Fig. 9, C and D). In each
case, the regression coefficients indicated a better fit for the
vector-average prediction on the short-duration trials than the
long-duration trials, whereas the analysis showed a better fit for the
winner-take-all prediction on the long-duration trials than the
short-duration trials. This was also apparent from the distribution of
points when the responses were compared against the winner-take-all
prediction on the short-duration trials (Fig. 9, A and
C, left). When the target was moving in the best direction,
the responses were smaller than expected, presumably because the
response on crossed trials was the average of the response on the
uncrossed trials where the stimulus moved in the best direction and a
non-best direction (which by definition had a smaller response).
Similarly the responses were greater than predicted when the target was
moving in a non-best direction for the same reason. This difference
necessarily would be smallest for neurons with either low firing rates
and on trials where the two stimuli moved in directions near each
other. For stimuli moving along the best-null axis, this difference was
greatest, as indicated by the greater scatter of points for the higher
firing rates. The result for neurons located in cortical area MST was similar (Fig. 10).
|
|
We made a direct comparison between the predictions of the two models that was similar to that already described for the eye movement metrics (Fig. 7). Figures 11 and 12 show the results of this analysis on the MT and MST neurons, respectively. Like the results described for the eye movements, the responses on the short-duration trials were better predicted by the vector-average model and the responses on the long-duration trials were better predicted by the winner-take-all model. These differences were statistically significant (see Table 2) when the best direction and the seven other directions were tested (Figs. 11 and 12, left) and when only trials in which the target and distracter moved along the best-null axis were considered (Figs. 11 and 12, right). Again, analysis of the neuronal responses as a function of stimulus speed showed no difference in the effects on the single neuron activity as was the case for the eye movement metrics (ANOVA, all P values > 0.05).
|
|
|
The results thus far indicate that the same cortical neurons can alter their activity between that better described by a vector-average model and that better described by a winner-take-all model. The change in processing depends on the stimulus conditions, but which difference in the stimulus conditions is important remains unclear. One change is the length of time that the monkey had to make the shape discrimination, select the correct stimulus as the target, and plan the pursuit eye movement. If this length of time altered the monkey's behavior and/or neuronal responses of MT and MST neurons, this should be evident by comparing the saccade latency between trials where the winner-take-all model was a better predictor with the saccade latency where the vector-average model was a better predictor. We therefore compared the saccade latency between these two trial types (those points falling above vs. below the line of perfect correlation shown in Figs. 11 and 12). In most cases, there was a trend for the trials that were better predicted by the winner-take-all model to have longer latencies than for trials that were better predicted by the vector-average model (Table 3). In some cases, this difference reached statistical significance, but these differences were generally quite small (rarely as much as 10 ms; see Table 3). Thus although there was a tendency for the trials best described by the winner-take-all model to have a longer latency, this effect was small, and we conclude that it is unlikely to be an important indicator of the shift from a vector-average to a winner-take-all mechanism.
|
In summary, these results showed that the responses of single neurons in MT and MST can be predicted by either a vector-average model or a winner-take-all model depending on the stimulus conditions. These conditions were identical to those that showed a similar difference in the accuracy of the two models in predicting the metrics of the initial eye movements. Thus in the same monkey, and even on interleaved trials, both the eye movements and the responses of these single neurons could be reliably, and substantially, altered by providing an extended period of time and spatial distance of stimulus motion in which to plan the eye movement.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We used the intersecting motion of two stimuli to determine how
populations of neurons process visual motion for the initiation of
pursuit eye movements. We made two salient observations. First, by
increasing the spatial and temporal separation of the two stimuli in
the visual field, we found that we could shift the model of the initial
saccadic and pursuit eye movements that better predicted the behavior
from a vector average of the two stimuli to a winner-take-all model of
the target stimulus. Second, recording from single neurons in areas MT
and MST showed that the model best predicting the activity of these
neurons also shifted from vector average to winner take all under the
same stimulus conditions that caused the shift in the eye movements.
Thus the activity of the MT and MST neuronal populations paralleled
changes in oculomotor behavior. Although it is possible that other
models could better predict these results, we have concentrated on
contrasting the winner-take-all and vector-averaging models for two
main reasons. First, other models tested by us and others have shown
very poor predictive power for either eye movements or neuronal
activity, including vector summation and probability summation
(Ferrera and Lisberger 1997; Groh et al.
1997
; Lisberger and Ferrera 1997
;
Recanzone et al. 1997
) Second, several other studies
have found either winner-take-all or vector-averaging predictions under
similar stimulus conditions. Although nonlinear or other types of
models may show better predictive power for these measures, it is clear
from this study as well as previous studies that these linear models
capture the essence of both the neuronal activity and initial pursuit
movements. We first will discuss the factors that influence this shift
in behavior from vector-averaging to winner-take-all models in the
context of this and previous experiments and then consider the relation of the change in neuronal activity to the hypothesized population coding.
Model predictions of pursuit behavior
Under our experimental conditions, when the two stimuli were
moving relatively close together in space and there was little time
before the monkey was required to initiate pursuit, the initial eye
movement metrics were better predicted by the vector average of the eye
movements when the two component stimuli were presented alone in that
visual field. Close together in our experiments meant that the two
stimuli crossed within 150 ms after target motion began and traveled
over a course of 1° to ~4°, depending on the stimulus velocity
and direction. In addition, because the monkey had the information
about which stimulus was to be the pursuit target only after the
appearance of the moving stimuli, it had only 150 ms to make the
appropriate shape discrimination before pursuit initiation was allowed.
We think these results are consistent with those of several recent
experiments that had generally similar stimulus conditions.
Lisberger and Ferrera (1997) used a paradigm in which
the two stimuli moved 3° and intersected at the fixation point 150 ms
after motion onset, but it was only at that time that the correct
pursuit target was indicated (by the other stimulus disappearing) and
so the pursuit was triggered as soon as the target was identified. They
found that a vector average best predicted the subsequent eye
movements. When combining the motion of a visual target with electrical
stimulation of MT, Groh et al., (1997)
also found the
vector average to be the best predictor of the observed pursuit
movements. In their experiment, the visual target was positioned to
move through the center of the RFs of the neurons electrically
stimulated so the activity related to the two "stimuli" overlapped.
The trigger to initiate pursuit of the visual target usually occurred
at the same time that the motion of the visual target and the
electrical stimulation began, so again there was little time before
pursuit initiation was required. Thus although there are substantial
differences between these experiments, they all had overlapping stimuli
that moved over a relatively short distance for a short time, and they gave the monkey only a short time to plan the pursuit initiation.
When we used stimuli with increased separation of the two stimuli in
space, which also allowed the monkey a longer time between target
identification and the trigger to initiate pursuit, the eye movement
metrics were better predicted by a winner-take-all model rather than a
vector-average model. This increased separation in our experiments
extended the duration of the target motion from 150 to 450 ms before
the monkey was allowed to initiate pursuit and extended the distance
traveled by these stimuli from 3.5 to 11°. Ferrera and
Lisberger (1995, 1997
) also used a paradigm in which a
winner-take-all model best predicted the initial pursuit, but the
stimulus conditions were substantially different. The two stimuli did
not overlap (Ferrera and Lisberger 1997
) and could be in
different hemifields and so did not have overlapping stimuli comparable
with those in our experiment. In their experiments, the fixation period
before the trigger for pursuit was 300 ms, closer to our 450-ms period,
but they commented that they obtained the same pursuit initiation for
periods between 100 and 800 ms. The similarity of their experiment and
ours is the lack of stimulus proximity during at least the early phases
of pursuit initiation, although in all of the results shown there was a
300-ms period before the pursuit-initiation trigger. Thus although both
their experiments and ours provide nonoverlapping stimuli for a longer period during the trial and a longer time for the monkey to plan the
pursuit initiation and the pursuit initiation in both was best
described by a winner-take-all model, the differences between the
paradigms make it difficult to conclude whether it is the same
variables that lead to the winner-take-all prediction.
Another set of experiments also has shown a shift from an average to a
winner-take-all prediction using motion stimuli (Zohary et al.
1996). In these experiments, the direction of motion of two
overlapping directions of predominant motion of randomly moving dots
was perceived as an average when the directions of the two motions
varied by little and as a winner-take-all when one stimulus direction
became stronger than the other. Some subjects also could switch back
and forth between the two modes depending on the task instructions,
indicating that the underlying computation of the population activity
is not dependent just on the stimulus conditions. Thus in both these
psychophysical experiments and our pursuit-initiation experiments, the
description could be shifted from an average to a winner-take-all by
modifying the stimulus conditions and/or modifying the conditions under
which the target selection was made: an increase in fixation time in
pursuit initiation or task instructions in the case of psychophysical judgments.
Unfortunately our experiments on pursuit initiation do not resolve which of several confounded variables underlies the shift in the computational mechanism. The stimulus features that varied together were the distance and duration of the motion stimulus. By initiating the target motion at a greater distance from the intersection point, given that the stimuli were moving at the same speed, there was also a longer period of time that the motion was present before the intersection of the two stimuli. It is therefore not possible to determine from our experiments the degree to which these two differences influenced the change in pursuit initiation. A second major factor is the differences in time allowed before pursuit initiation was triggered that allowed more or less time for the monkey to shift its attention to the target stimulus and/or prepare for the movement. We cannot determine whether it is the shift in stimulus conditions or the time allowed before pursuit initiation that is most important, although from even the brief summary of previous experiments, it is clear that both are involved.
The observations of a shift from vector average to winner take all for
motion stimuli is reminiscent of a similar well-established shift for
saccades when two stationary targets are presented to human subjects
(Coren and Hoenig 1972; Findley 1982
).
When the two targets are close together, saccades are made to a point
between them, the average of the two locations, whereas when the
targets are further apart, the saccades are made to one target or the other, in a winner take all fashion. In so far as the population activity underlying this saccadic averaging or separation is similar to
that for the initiation of pursuit to a moving stimulus, it would
strongly suggest that the key factor is the separation of the two
pursuit targets. Such averaging or winner-take-all shifts also have
been demonstrated in monkeys (Ottes et al. 1984
).
However, even in these studies of averaging saccades, subsequent
experiments have demonstrated that factors other than the presence of
the two stimuli can influence the behavior and by inference the
underlying computation. He and Kowler (1989)
showed in
human experiments, for example, that the average end location of
saccades shifts based on the prior history of target locations with no
change in the current stimuli. Furthermore recent experiments in the monkey (Edelman and Keller 1998
) showed that averaging
saccades were more likely when saccade latency was short.
The underlying computations for saccades in the presence of two targets also shows sensitivity both to the configuration of the targets and to other factors including the time before the movement is made so that this control of saccades has striking similarities to the control of pursuit.
For both the pursuit and saccadic systems, the shift in type of movement made is hardly surprising and in fact makes biological sense. When the stimuli are close together in time and space and little time is available before the saccade is made, the pursuit or saccade is made to an average of the two targets. This would appear to be a reasonable strategy because an average approximation would bring both targets closer to the fovea quickly and a subsequent saccade or pursuit movement could be made if necessary. On the other hand, when the stimuli are farther apart and there is more time before the movement, the pursuit or saccade can be made accurately to one target or the other. This strategy brings the target on the fovea with only one saccade or pursuit movement.
Neuronal activity and model predictions
In both MT and MST, the neuronal behavior shifted from a
vector-average to a winner-take-all prediction as the spatial-temporal motion of the stimulus and the time available before pursuit initiation changed, just as was the case with the pursuit and saccadic behavior. For the short-duration trials, the fit of the observations with the
vector averaging was similar to that found in our previous experiments
in which the two stimuli moved across the RFs of MT and MST neurons
while the monkey continued to fixate throughout the trial
(Recanzone et al. 1997), which is not surprising given that the stimulus conditions were almost identical. This similarity in
results with pursuit initiation and without it does suggest, however,
that on the short-duration trials, the stimulus conditions were the
critical variables. This argues for a relatively greater importance of
the stimulus configuration than time available before the movement
initiation on the short-duration trials. The averaging result is also
consistent with models of MT using summation with divisive
normalization (Carandini and Heeger 1994
; Heeger
et al. 1996
; Simoncelli and Heeger 1998
).
The relation of the long-duration trials to a winner-take-all
prediction is consistent with the pursuit and saccade behavior, but it
does not have an obvious precedent at the neuronal level. What is
surprising is that the neurons in both MT and MST behaved so similarly.
One might have expected to see some indication of a shift in processing
from one area to the next, particularly given the substantial change in
the RF sizes between these two areas (Desimone and Ungerleider
1986; Komatsu and Wurtz 1988
).
A limitation of our analysis of the neuronal responses is the method we
used to generate the vector-average and the winner-take-all model
predictions. For the eye-movement predictions, we compared the eye
movement metrics using the behavior on uncrossed trials to predict the
behavior on crossed trials. This is appropriate because the movement to
each of the component stimuli on the crossed trials predicts the true
vector average of the eye movement, and the errors between observed and
predicted were relatively small for at least one of the two
predictions. However, for the neuronal responses we used a scalar
average, not a vector average, of the responses to the two-component
stimuli on the crossed trials. This scalar average does not predict the
same response if MT and MST neurons also are tuned for small variations
of stimulus speed. For example, the scalar average of two stimuli
moving at a 90° angle predicts a response for the intermediate
direction and the same speed, whereas a vector average predicts the
response to a stimulus moving in an intermediate direction at a speed
1.41 times greater. Stimulus speed does influence the activity of MT neurons (Allman et al. 1985; Lagae et al.
1993
; Maunsell and Van Essen 1983
; Mikami
et al. 1986
), although the speed-tuning functions for such
small differences are not easily predicted from those data. Because we
could not directly measure the responses of these neurons to single
stimuli moving at each of the possible vector-average directions and
speeds, this may in part account for the errors observed in the
short-duration crossed trials. Across the population of neurons, the
vector-average response would in some cases be greater than the
prediction from the scalar average, and in other cases the
vector-average response would be smaller than the scalar average. Thus
the difference would be expected to increase the variance in the error
across the population of neurons but likely not the mean. Taking the
stimulus speed selectivity into account may well produce smaller
overall errors on the short-duration crossed trials.
If we consider only the stimulus conditions in our experiments and take
the view that MT and MST essentially consists of velocity maps of
visual space, we can visualize the changes that might occur when
comparing the short- and long-duration trials (Fig. 13A). In this view, the
motion of the two stimuli at any given time can be represented on such
a velocity map by two independent regions or hills of activity. When
the stimuli are separated, as in the long-duration trials and the
uncrossed trials, the two hills of activity are clearly separated (Fig.
13A, left), and pursuit can be to either target represented
by either of the separate hills. In contrast, when the stimuli cross in
close temporal and spatial proximity, the hills of activity on the
velocity map can be regarded as overlapping (Fig. 13A,
right), and the pursuit is to the average of the two hills. In
this case, neurons show a reduced response to the best stimulus
direction and increased activity to the non-best directions so that on
this velocity map, the distribution of activity would be the average of
the two component stimuli. The changing interaction between these two
regions as stimuli approach each other must depend on both excitatory
and inhibitory interactions as has been proposed in models of the interaction (Ferrera and Lisberger 1995, 1997
).
|
When viewed in this way, the hypothetical interactions between
populations of neurons in MT or MST during pursuit initiation are
similar in several respects to the neuronal activity in the monkey
superior colliculus thought to be associated with the initiation of
saccades made in the presence of two stationary targets. It generally
is accepted that the activity related to an impending saccade can be
represented by a hill of activity among the saccade related burst
neurons on the collicular map and that saccade generation results from
the vector summation of this activity (van Gisbergen et al.
1987; van Opstal and van Gisbergen 1989
; and see
review by van Gisbergen and van Opstal 1989
) although
the experimental evidence is sparse (Lee et al. 1988
).
Several studies have shown that the neurons in the colliculus are
active during averaging saccades (Edelman and Keller
1998
; Glimcher and Sparks 1993
; van Opstal and van Gisbergen 1990
). Edelman and Keller
(1998)
also found that when the averaging saccades occurred,
the movement fields of the collicular neurons were broader than when
saccades were made to single targets. The conceptions of the shifts in the activity of the neurons that might underlie these saccades (Edelman and Keller 1998
; Krommenhoek and
Wiegerinck 1998
; van Opstal and van Gisbergen
1990
) are similar to those depicted for the pursuit eye
movements. The difference is that while the map for activity preceding
pursuit is in velocity space, the map for activity preceding the
saccade would be in retinotopic space. Thus while the space on which
the activity is plotted is different, there are substantial
similarities between the conception of the underlying maps and how the
shift from an average to a winner-take-all might come about at the
level of neuronal populations.
One possible interaction within the velocity map that is consistent
with several of our observations is an interconnection of neurons that
have the same velocity vector. The increased spatial separation of
stimuli in the long-duration trials would allow cortical neurons with
the same velocity preference that were activated early in the stimulus
path to influence the activity of the neuron later in the path
(centered in our experiments on where the stimuli crossed). Figure
13B shows a schematic drawing of how such a mechanism based
on the direction preference of columns of neurons in area MT or MST
could implement such modulation. Figure 13B, left, shows the
RFs and best directions of neurons that, if interconnected in an
excitatory fashion, would be activated sequentially and positively
influence the activity of the next column of neurons as a stimulus
traversed the visual field. Such sequential activation commonly would
occur in nature either during self-motion or while viewing the linear
motion of other objects. These interactions are consistent with the
influences of stimuli moving in the best direction of the recorded
neuron up to two or more RF diameters away on MT neurons
(Britten and Heuer 1999) and with the anatomic intracortical connections described between neurons with similar orientation selectivity in striate cortex (see Gilbert et al. 1996
), but whether these connections exit between neurons with similar direction preference in MT has not been determined. A second
type of cortico-cortical interaction is shown in Fig. 13B, right, where neurons with RFs in the same region of visual space, but with different best directions, provide inhibitory input to each
other. There is also good evidence that such interactions occur in MT
and MST neurons, for example, the demonstrations of divisive
normalization when stimuli move in different directions (e.g.,
Recanzone et al. 1997
; Snowden et al.
1991
).
Our results indicate that the influence of neurons with RF locations and directional tuning consistent with the target trajectory is greater than the influence of neurons with different best directions and the same RF location. In this case, divisive normalization would result in responses closer to a winner-take-all model in the long-duration trials because of the excitatory input from other cortical neurons responding to the target as it moved toward the RF. This long-range excitatory input would provide a stronger input to the recorded neuron than would the columns of neurons responsive to the direction of the distracter. This would result in activity that was closer (but still lower) to that measured in trials where only one stimulus was in the RF of the neuron and would effectively shift the response from being consistent with a vector-average model to a winner-take-all model. On the short-duration trials, all stimuli initiated their motion near the edges of the RF of the neuron under study, and thus interactions from neurons one or more RF diameters away would be minimal. A direct test of this hypothesis would be to present stimuli that frequently change in direction, thereby not activating these hypothetical long-range connections. We predict that in this case, both the neuronal activity and the initial smooth pursuit eye movement would be better predicted by a vector-average model, in spite of the fact that the monkey would have the same amount of time to identify the target stimulus under both conditions.
![]() |
ACKNOWLEDGMENTS |
---|
The authors thank their colleagues at the Center for Neuroscience (particularly K. H. Britten, H. Heuer, and S. Elfar) and at the Laboratory of Sensorimotor Research for suggestions on previous versions of the manuscript, U. Schwarz for support during the collection of data for these experiments, and the National Institutes of Health Laboratory of Diagnostic Radiology Research for providing the MRIs of the monkeys.
Funding was provided by the National Eye Institute (R. H. Wurtz and G. H. Recanzone) and the National Research Council, National Institute on Deafness and Other Communication Disorders Grant DC-02371, The Klingenstein Fund, and the Sloan Foundation (G. H. Recanzone).
![]() |
FOOTNOTES |
---|
Address for reprint requests: R. H. Wurtz, Laboratory of Sensorimotor Research, Building 49, Room 2A50, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD 20892-4435.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
Received 24 March 1999; accepted in final form 1 June 1999.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|