Shift in Smooth Pursuit Initiation and MT and MST Neuronal Activity Under Different Stimulus Conditions

Gregg H. Recanzone1 and Robert H. Wurtz2

 1Center for Neuroscience and Section of Neurobiology, Physiology and Behavior, University of California, Davis, California 95616; and  2Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland 20892


    ABSTRACT
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Recanzone, Gregg H. and Robert H. Wurtz. Shift in Smooth Pursuit Initiation and MT and MST Neuronal Activity Under Different Stimulus Conditions. J. Neurophysiol. 82: 1710-1727, 1999. The activity of neurons in extrastriate middle temporal (MT) and medial superior temporal (MST) areas were studied during the initiation of pursuit eye movements in macaque monkeys. The intersecting motion of two stimuli was used to test hypotheses about how these direction- and speed-sensitive neurons contribute to the generation of pursuit. The amplitude and direction of the initial saccade to the target and the initial speed and direction of pursuit were best predicted by a vector-average model of the underlying neuronal activity with relatively short time and spatial separation before a visual pursuit target and a distracter stimulus crossed in the visual field. The resulting eye movements were best described by a winner-take-all model when the time and spatial separation between the two stimuli was increased before the stimuli crossed. Neurons in MT and MST also shifted their activity from that best described by a vector average to a winner-take-all model under the same stimulus conditions. The changes in activity of neurons in both areas were generally similar to each other during these changes in pursuit initiation. Thus a slight alteration in the target motion produced a concurrent shift in both the neuronal processing and the movement execution. We propose that the differences in the oculomotor behavior can be accounted for by shifts in the overlap of active neuronal populations within MT and MST.


    INTRODUCTION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

A central question in neuroscience is how neurons in the brain represent the sensory-motor events that determine the metrics of movements. This representation in the primate brain has been found to lie in the activity of a population of neurons that are active before a range of movements. Such population coding has been observed in the primate motor cortex during hand-reaching movements (Georgopoulos 1996; Georgopoulos et al. 1986) and in the superior colliculus during saccadic eye movements (Lee et al. 1988; McIlwain 1991; Munoz and Wurtz 1995; Sparks and Mays 1990). In both cases, the model that best represented the translation of neuronal activity into movement was one that took the vector sum or average of the activity of each neuron across the population (Georgopoulos et al. 1986; Lee et al. 1988).

Such a population of active neurons, each tuned to a range of stimulus parameters, also has been found in the sensory processing on which movement frequently depends. Nowhere is this population activity clearer than in the direction- and speed-selective neurons in the extrastriate cortex of the primate, particularly the middle temporal (MT) and medial superior temporal (MST) areas (Allman et al. 1985; Desimone and Ungerleider 1986; Maunsell and Van Essen 1983; Tanaka et al. 1989; Van Essen et al. 1981). The use of this visual information to guide smooth pursuit eye movements is also well established and is based on the change in activity of single neurons during pursuit, the pursuit deficits following ablation, and the alteration of pursuit by electrical microstimulation (Dursteler et al. 1987; Groh et al. 1997; Komatsu and Wurtz 1988, 1989; Newsome et al. 1985; Schiller and Lee 1994; Yamasaki and Wurtz 1990). Although several different classes of models could be used to predict the neuronal activity of these cortical areas and pursuit behavior, the two models that have been most successful have been the vector-averaging and winner-take-all models. The vector-average models assume that all the vectors representing each active neuron are averaged, whereas the winner-take-all models assume that only the neurons with the highest activity contribute to the movement generation (Ferrera and Lisberger 1995, 1997; Groh et al. 1997; Lisberger and Ferrera 1997).

In testing which of these models best predict the observed pursuit behavior, recent experiments have compared the pursuit generated when one stimulus moved across the visual field with that produced when two stimuli moved across the visual field. The introduction of the second stimulus tests how the underlying neuronal populations could interact to produce the metrics of the initial eye movement: do they combine (vector averaging) to produce an averaged eye movement or does one population predominate (winner take all) to produce an eye movement similar to that when only one stimulus is presented. Experiments using two visual stimuli (Lisberger and Ferrera 1997) or one stimulus coupled with electrical stimulation of MT (Groh et al. 1997) found that a vector-averaging model best predicted the pursuit behavior. Similarly the responses of MT and MST neurons were best predicted by the vector-average model of two stimuli moving through the receptive field in fixating monkeys (Recanzone et al. 1997). However, in another experiment using a different two-stimulus task, pursuit was better predicted by a winner-take-all model (Ferrera and Lisberger 1995, 1997).

In the present experiments, we directly investigated the activity of neurons in MT and MST that form the basis for these model predictions. We first verified the alterations in pursuit behavior in the presence of two moving stimuli. By increasing the time and spatial separation of two moving stimuli before the onset of pursuit, we found that the best prediction of the pursuit behavior shifted from a vector average to a winner-take-all model. We also found that the activity of neurons in MT or MST also shifted from a vector-average to a winner-take-all model as the stimulus conditions changed. We propose that the behavioral changes are consistent with shifts in the overlap of active neuronal populations on a conceptual velocity map of MT and MST.

A brief report of this study has appeared previously (Recanzone et al. 1993).


    METHODS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

The same two monkeys used in these experiments also were studied in a previous experiment (Recanzone et al. 1997), and the experimental procedures were the same. All procedures were approved by the Institute Animal Care and Use Committee and complied with Public Health Service policy on the humane care and use of laboratory animals.

Behavioral tasks

Two adult male rhesus monkeys (Macaca mulatta) were used in this study (monkeys N and P). Each monkey was trained to perform several versions of a pursuit eye movement task, and the two tasks used in most experiments are illustrated in Fig. 1. Each trial was initiated when the monkey looked at a central fixation stimulus (fs) that was one of two different shapes (Fig. 1A). The shape of the fixation stimulus cued the monkey as to which of two moving stimuli it was required to acquire with a saccade and track with a smooth pursuit eye movement to obtain a liquid reward. The monkey maintained fixation for a random interval between 100 and 500 ms (Fig. 1B) to within ±2° of the center of the fixation stimulus. On all trials, two visual stimuli were then presented in motion in one of two possible stimulus configurations. Because all of the behavioral results described in this report were collected while recording from a single neuron in either cortical area MT or MST, the stimulus conditions were set so that at least one of these two stimuli always moved through the receptive field (RF) of the neuron under study (Fig. 1, - - -). Trials consisted of either one stimulus moving through the RF of the neuron and the other stimulus moving near the same horizontal eccentricity in the other visual hemifield (referred to as uncrossed trials; Fig. 1, left) or the trials consisted of two stimuli moving through the RF of the neuron in which one stimulus moved in the best direction of the neuron and the other stimulus moved in one of seven different, non-best directions (referred to as crossed trials; Fig. 1, right).



View larger version (22K):
[in this window]
[in a new window]
 
Fig. 1. Schematic diagram of the behavioral paradigms. Representative stimulus configurations at 5 different time points are illustrated for an uncrossed trial (left) and a crossed trial (right). A: before the onset of the trial, the shape of the fixation point stimulus (fs) served as the cue for which of the 2 subsequently moving stimuli was the target. B: after fixation there was a randomized 100- to 500-ms pause. C: at trial start, 2 objects appeared in motion. On uncrossed trials, each object was in a different hemifield. On crossed trials, both objects moved toward the center of the receptive field (RF) of the neuron under study ( - - -). D: 150 ms (or 450 ms for long-duration trials) after motion onset, 1 or both stimuli were located at the center of the RF of the neuron under study and the fixation stimulus was extinguished. E: moving stimuli continued their trajectory and the monkey was required to pursue the target (square in this example) for 300-900 ms. left-arrow , direction of motion of each of the 2 shapes.

The monkey had to maintain fixation until the fixation point was extinguished 150 ms after the onset of the motion of the two stimuli (Fig. 1D). The monkey was rewarded for making a saccade to the visual stimulus that was the same shape as the fixation stimulus and then accurately pursuing that target stimulus for 300-900 ms (Fig. 1E). The initial saccade had to be made within 450 ms of the fixation stimulus offset (Fig. 1D) and had to be within 10 deg of the target stimulus. The eye position also had to be within ±5° of the target during the pursuit phase of the trial starting 300 ms after the initial saccade. These windows were necessarily large due to the size of the stimulus objects and the accuracy of the initial saccade during the crossed trials as described in RESULTS. On uncrossed trials, one stimulus was located at the center of the RF at the time that the fixation stimulus was extinguished, and the stimulus in the opposite visual field was located at the mirror image location. On crossed trials, both stimuli were at the center of the RF at the time that the fixation point was extinguished and were superimposed with the same luminance as a single stimulus. This gave the perception to human observers of two superimposed opaque objects moving across the visual field and crossing through each other at the location corresponding to the center of the RF. These two trial types allowed us to determine the effects of a distracter stimulus by comparing the eye movements and single-neuron activity in trials where the target was the only stimulus moving through the RF with trials where the distracter stimulus was also moving through the RF. In both cases, the motion of the target stimulus was the same, and the monkey had to make the same shape discrimination.

A third and fourth set of trial types, termed the long-duration trials, were identical to the crossed and uncrossed trial types described in the preceding text except that both visual stimuli initially were displaced in space such that 450 ms elapsed before the fixation point was extinguished after stimulus motion onset, and at that point the stimulus objects were at the same location as in the first two tasks (e.g., at the center of the RF; Fig. 1D). These trials provided a greater separation in space between the two stimuli and allowed the monkey an additional 300 ms to perform the shape discrimination before the stimuli reached the RF of the neuron under study. Again, the starting location of the stimuli was dependent on the response properties and the RF size of the neuron under study on that session. The stimulus velocity on the long-duration trials was the same as for the short-duration trials, resulting in different distances in the starting locations of the stimuli between sessions, but the stimuli always reached the center of the RF 450 ms after stimulus motion onset.

Stimuli were moved in eight different directions (0, 45, 90, 135, 180, 225, 270, and 315°). For uncrossed trials, the stimulus that was the target could move in any of the eight directions in any trial as could the stimulus in the other visual hemifield. For the crossed trials, the target stimulus also could move in any of the eight directions and the distracter moved in one of the other seven directions. All of these stimulus conditions (crossed trials, uncrossed trials, different target directions) were presented on randomly interleaved trials, and when long-duration trials were used, they also were introduced with the other trial types in randomly interleaved order. A complete data set consisted of at least eight correct trials for each stimulus type, although most of the data of this report are from 10 to 15 correct trials for each stimulus type.

Several factors that were checked in an initial analysis of the behavior were found not to be statistically significant and will not be considered further. We found no significant differences, or apparent trends, between either the eye metrics or the neuronal responses on the short-duration trials when they were interleaved with long-duration trials and when they were not. The behavioral performance for the short-duration trial types were not significantly different between the two monkeys nor was there any significant difference in the percent of correct trials (trials in which the target was accurately tracked for 300 ms after the initial saccade) depending on which stimulus was the target for the eye movement (repeated measures ANOVA, all P values > 0.1). The best performance was for both the short- and long-duration uncrossed trials (86% correct), and the lowest performance was for crossed trials (77%). These behavioral observations are in good agreement with previous studies using similar one- and two-stimulus configurations while the monkey performed a speed discrimination task (Treue and Maunsell 1996).

Monkey N completed 80 sessions and monkey P completed 65 sessions of the short-duration task in which both crossed and uncrossed trials were presented (Fig. 1). For the long-duration task, monkey N completed 60 sessions, but monkey P frequently was unable to reliably suppress the initial saccade for a sufficient period to complete at least eight trials under all conditions for the long-duration trials (only 7 sessions with only 3-7 successful trials per stimulus per session). Although these limited data were consistent with those obtained in monkey N, this dataset is not sufficiently extensive to report.

Visual stimuli

Visual stimuli were back-projected onto a tangent screen placed 57.4 cm from the monkey using a video projector (Electrochrome, SVGA, 1024 × 768 pixel resolution). Each pixel subtended a visual angle of 0.13° horizontally and 0.12° vertically. Images were created by a PC and were presented at a frame rate of 72 Hz. Stimulus objects were brighter (1.8 cd/m2) than the background (0.2 cd/m2). Two of five different objects (circle, square, diamond, plus sign, and triangle) were used in each behavioral session, with each object of equal luminance and size (same number of pixels/object) subtending a maximum visual angle of 1.8°. Objects were moved across the screen by displacing each illuminated pixel by 1 or 2 pixels each frame in either the horizontal, vertical, or both directions. Three stimulus speeds were produced by pixel shifts of 1, 1.5, or 2 pixels/frame corresponding to ~9, 13.5, or 18°/s along the horizontal and vertical directions and 12, 18.5, or 25°/s along the obliques. For speeds using 1.5 pixels/frame, all stimulus pixels were displaced by 1 and 2 pixels on alternate frames. These stimulus motions were perceived by human observers as continuous, smooth motion.

Data collection procedures

Each monkey underwent magnetic resonance imaging (MRI) scans in the saggital and frontal planes to aid in aligning the recording cylinders and in reconstructing the electrode penetrations. Under general anesthesia, a scleral search coil was implanted in each eye and a restraining head post was implanted on the skull using dental acrylic following procedures described in Duffy and Wurtz (1995). After behavioral training was completed, a recording cylinder was implanted over the parietal cortex directly over cortical areas MT and MST in the stereotaxic vertical plane. Monkey P had one cylinder and monkey N had a second cylinder implanted over the opposite hemisphere after recordings from the first hemisphere were completed.

Horizontal and vertical eye position was recorded using the magnetic search coil technique (Fuchs and Robinson 1966; Judge et al. 1980). Eye position was digitized at 500 Hz with a resolution of 0.1°. Data acquisition and control of the video display computer was controlled by a real-time experimental system (REX) (Hays et al. 1982) and run on a PDP11/73. Both target and eye position were monitored on-line as well as stored for off-line analysis.

Tungsten microelectrodes were advanced into the region of the superior temporal sulcus through guide tubes positioned in a grid within the recording cylinder (Crist et al. 1988). Neuronal signals were amplified, filtered, and displayed on an oscilloscope and audio monitor using conventional methods. Single neurons were isolated using a time and amplitude window discriminator (Bak). The occurrence of each neuronal spike was digitized and time stamped at 1 kHz and stored for off-line analysis.

Neurons recorded in this study met the same three criteria described previously (Recanzone et al. 1997): the activity of the neuron was altered by the presence of visual stimuli (but not necessarily moving stimuli), isolation was sufficient to be confident that only a single neuron was being recorded, and the center of the RF was between 5 and 25° of eccentricity. This last requirement was necessary as the monkeys were unable to maintain fixation if a target or distracter stimulus moved through the fixation point before the fixation point offset and the monkeys were unable to reliably make the visual shape discrimination at eccentricities >25°.

RFs were defined using hand held bars, spots, and patterns of light. RF edges were defined using both visual and audio criteria as locations where the neuron no longer responded to moving, flashed, or stationary visual stimuli. We categorized each neuron as being located within cortical area MT or MST based on the location and depth of the electrode within the recording cylinder relative to the MRI images and the characteristics of the visual stimuli required to maximally activate the neuron (Allman et al. 1985; Baker et al. 1981; Lagae et al. 1993). MT neurons responded best to small moving stimuli and had smaller RFs than MST neurons. MST neurons usually responded better to larger moving stimuli such as patterns than to moving spots. We only rarely encountered neurons with very large RFs that responded best to large patterned stimulation similar to those described in the dorsal region of MST (MSTd) (Komatsu and Wurtz 1988; Tanaka and Saito 1989; Tanaka et al. 1986, 1989, 1993), and the vast majority of our sample of MST neurons were characteristic of those located in the lateral region (MSTl). However, it is possible that some of the neurons in this study were located within MSTd.

The stimulus speed and shape used in a particular behavioral session were based on the best responses recorded during preliminary characterization of the cell's tuning and RF properties and were adjusted so that the stimulus would traverse as much of the RF as possible before reaching the center at 150 ms after motion onset on the short-duration trials. RF sizes ranged from diameters of 5-24° for MT neurons and 5-35° for MST neurons and increased in diameter with increasing eccentricity as described previously (e.g., Desimone and Ungerleider 1986; Komatsu and Wurtz 1988). We noted no differences in the response properties of our sampled neurons when the stimulus onset was slightly within, at, or slightly outside the edge of the RF, and thus all data were pooled.

At the end of the experiments, histological sections through the superior temporal sulcus were obtained as described previously (Recanzone et al. 1997). MT was identified on the posterior bank by its dense myelination. Orientation for the general region of MST was provided by the region of dense myelination on the anterior bank. Drawings of the sections showed that the guide tubes were directed toward the superior temporal sulcus and that the electrode tracks passed through or below the densely myelinated area on the anterior bank (MST) and the posterior bank (MT). The categorization of each cell to MT or MST was based on the physiological criteria described above and was consistent with the cytoarchitectonic definition of these two cortical areas.

Data analysis and model predictions

Smooth pursuit metrics were measured for 40 ms starting 20 ms after the completion of the initial saccade. This period was chosen as it does not contain corrective saccades (Lisberger and Westbrook 1985) and follows the same analysis period reported recently by Groh et al. (1997). The onset and offset of the saccades were defined based on the velocity, acceleration, and duration of the eye movement using a saccade-detection algorithm. All accepted individual eye movements then were inspected visually to ensure that only eye movements that were initiated between 100 and 450 ms after the fixation point offset and that contained only a single saccade before the pursuit measurements were included in this analysis.

The saccade amplitude was defined as the length of a vector (in deg) from the fixation point to the eye position 20 ms after the initial saccade. The saccade direction was defined as the direction of this vector. The initial pursuit direction was defined as the direction of a vector drawn from the eye position starting at 20 and ending at 60 ms after the saccade, and the initial pursuit speed (°/s) was defined by the length of this vector divided by 0.040 s.

The eye metrics (initial saccade amplitude and direction and initial pursuit speed and direction) were compared with the predictions of winner-take-all and vector-average models of the behavior. For these two predictions, the eye metric was measured for each stimulus direction on the uncrossed trials, and each was described as a vector with both magnitude and direction. The winner-take-all model predicts that the eye movement should follow one vector or the other when the two stimuli were presented inside the RF, and we compared the eye movement on each of the two uncrossed trials with the eye movement on crossed trials when the same target stimulus was presented. To predict the vector-average result, the vectors for the two stimuli presented on the uncrossed trials were averaged, taking into account both the direction and velocity of each eye movement; the average therefore represents a true vector average and is not the linear average of the two directions or speeds. We also compared the results with those predicted by a vector sum model by summing the two component vectors. The vector sum model was never a better predictor than either of the other two models (as was also the case in previous experiments of Ferrera and Lisberger 1995; Groh et al. 1997; Lisberger and Ferrera 1997), and we will not consider the predictions of this model any further.

To determine qualitatively which model made the best prediction, the initial pursuit speed and direction and the initial saccade amplitude and direction were averaged across all correct trials for a given stimulus condition. Each of these points across all sessions for a given monkey then were subjected to regression analysis with two dependent variables (Mosteller et al. 1983; see also Recanzone et al. 1997). For analysis of the direction data, we removed any artificially large differences in target directions as, for example, between 358 and 3°, which could be taken as either a difference of 363 or 5°. When we compared the difference between the measured and predicted directions, we always used the difference that was smallest, that is, 5 rather than 363° in this example. To quantitatively measure the difference in the predictive power of the two models, the absolute value of the error between the measured eye metric and that predicted by each of the three different models was measured. These errors then were subjected to a paired t-test to determine if one model consistently better predicted the eye movement metric across stimulus conditions and behavioral sessions.

For the responses of individual MT and MST neurons, the firing rate was measured in the period from the onset of the moving visual stimuli to the onset of the initial saccade (Fig. 1B). The firing rate before the onset of the moving stimulus also was calculated (spontaneous activity) and subtracted from the driven activity. For each neuron, the best direction was defined as the stimulus direction that gave the largest response during the uncrossed trials. The null direction was defined as the direction 180° from the best direction. As with the eye metric measurements, the winner-take-all prediction was defined as a response equivalent to that of the target stimulus during the uncrossed trials, where a single stimulus was moving through the RF of the neuron. The vector-average model was predicted by the averaged response for the two stimulus directions taken during the separate uncrossed trials and thus is a scalar average of the responses (see Recanzone et al. 1997). The errors between the predicted and observed responses were calculated as a contrast ratio, |(predicted - observed)|/(predicted + observed) because the activity of individual neurons varied widely so that there were large differences in the total activity (e.g., 10 spikes from neurons with a peak activity of 12 spikes) and small differences (e.g., 10 spikes where the peak activity was 80). The absolute value was used to prevent positive and negative differences from summing near zero because deviations of the model predictions of either sign were equally important.


    RESULTS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Our major observation was that the presence of a distracter stimulus that crossed the trajectory of the target stimulus affected both the metrics of the initial pursuit eye movement and the responses of single neurons in areas MT and MST. The eye movement metrics and the neuronal activity were consistent with either a vector-averaging or winner-take-all models of the activity of the MT and MST neuronal population, but which model was most appropriate changed between the short- and long-duration trials. We first will describe the changes in the initial pursuit eye movements, then compare these behavioral observations with the predictions of the vector-average and winner-take-all model, and finally describe the changes in single-neuron activity recorded as the pursuit behavior changed in comparison with the activity predicted by the same models.

Eye-movement metrics

In each experiment, we compared the pursuit initiation with only one stimulus in each visual hemifield with that with two stimuli in the visual hemifield and varied the time and distance over which the stimuli traveled in each condition. Figure 2 shows the eye movements from one experiment recorded during these four stimulus conditions with a target that was in the left hemifield and moved leftward. The two columns in Fig. 2 show the eye movements to the same target stimulus on the uncrossed trials (left -when the 2 stimuli were in opposite visual fields) and crossed (right -when the 2 stimuli crossed in the same visual field; see METHODS and Fig. 1). The rows in Fig. 2 show the effect of changing the duration and distance of stimulus motion from a relatively short-duration and stimulus path (top) to a relatively long duration and length (bottom). For brevity, we will refer to these spatial-temporal changes in the stimulus motion across trials as short- and long-duration trials. The time at which the stimuli crossed the center of the field are aligned vertically (Fig. 2, A and C and B and D) so that the target stimulus trajectories during the last 700 ms are aligned on the time axis.



View larger version (33K):
[in this window]
[in a new window]
 
Fig. 2. Example of differences between pursuit eye movements under 4 different stimulus conditions. All eye movements from 1 session where the same target moved leftward on all trials. A: horizontal eye position as a function of time during the short-duration uncrossed trials. Target trajectory (T) is shown as the solid gray line. B: eye position measured on crossed trials with the same target trajectory as in A, and with a distracter (D) moving in the rightward direction (dashed gray line). C: eye position as a function of time during the long-duration uncrossed trials. D: eye position as a function of time during the long-duration crossed trials. C and D are horizontally aligned in time such that the location of the visual stimuli are identical with those of the short-duration trials shown in A and B. Insets: starting stimulus condition for each trial type. On the short-duration crossed trials, the initial saccade and pursuit eye movement did not follow the target, whereas the eye movements did follow the target on long-duration trials.

On the short-duration trials (Fig. 2, A and B), the eye movements on the uncrossed trials (Fig. 2A) were characterized by an initial saccade that brought the eye close to the target and an initial pursuit movement where the eye immediately moved at nearly the same speed and direction as the target with occasional corrective or catch-up saccades. On crossed short-duration trials (Fig. 2B), the initial saccade was not to the target (solid gray line) and the initial direction and speed was not matched to the target direction and speed as indicated by the relatively flat portions of the eye traces immediately after the first saccade. Normal pursuit was not generally resumed until after the first corrective saccade, and by that time, the two stimulus objects had moved away from each other. Thus the uncrossed case shows the usual pattern of saccades and pursuit for a single target, whereas the crossed case shows a saccade directed between the target and the distracter coupled with a reduced initial pursuit speed.

On the long-duration trials (Fig. 2, C and D), where the period of time before the release from fixation was extended 300 ms, the eye movements on the uncrossed trials (Fig. 2C) were similar to the short-duration uncrossed trials (Fig. 2A). For the crossed long-duration trials (Fig. 2D), the initial pursuit eye movements were much more like those on the uncrossed trials than the short-duration crossed trials. The eye movements were now closer to the target and generally had a direction and speed similar to the target. Thus there was a significant alteration in the initial saccade and speed of pursuit for the short-duration but not the long-duration crossed trials compared with the uncrossed trials.

To see whether these changes in pursuit and saccades were consistent on a trial-by-trial basis, we analyzed the eye movements across all 145 behavioral sessions for both monkeys. We first normalized the eye metrics in each session to allow for the different target directions, speeds, and eccentricities that resulted from tailoring the stimuli to the RF properties of the neuron under study. To do this normalization for the trials in each session, we first took the average of all correct eye movements for uncrossed trials where the target stimulus moved in the best direction of the neuron under study and set the value of the horizontal and vertical components of the averaged eye metrics (initial pursuit speed and direction or initial saccade amplitude and direction) to +1.0. We then did the same for the uncrossed trials where the stimulus moved in the direction opposite to the best direction and assigned these horizontal and vertical components a value of -1.0. These eye movements to the single target were generally very accurate so that these averages gave good estimates of the monkey's ability to follow the target moving in a given direction and speed in that session. We then measured the horizontal and vertical values of these same eye movement metrics in the crossed stimulus trials and plotted these values on the axes based on the normalized values.

Figure 3 shows a plot of initial horizontal and vertical pursuit speed for sample trials from each of the 145 sessions and compares this pursuit on short-duration (Fig. 3A) with those on long-duration (Fig. 3B) trials. All the plots in Fig. 3 show the initial eye speeds on single uncrossed trials in which the target was moving in one direction (, the best direction of the neuron being studied), the eye movement on single uncrossed trials when the target was moving in the opposite direction (), and the movement on single crossed trials in which one stimulus moved in the best direction and the other moved in the opposite direction (+; or at ±90° from that direction.). The two different models make very different predictions for the crossed trials. The vector-average model predicts that the points should cluster near 0,0, whereas the winner-take-all model predicts that the points should cluster at the same regions as in the uncrossed trials, i.e., near (1,1) and (-1,-1). For short-duration trials (Fig. 3A), the pursuit-initiation speeds for single target motion ( and ) clustered into the two areas in the top right and bottom left quadrants appropriate for pursuing the two targets moving along the best and opposite directions. When two targets were present (+), the initial pursuit speeds were clearly different and had much lower initial speeds in both the horizontal and vertical directions. In contrast, for the long-duration trials (Fig. 3B), the eye movements tended to match the speed of the target stimulus in the crossed trials as well as the uncrossed trials regardless of the direction of the target motion (+ cluster near both  and ). Although there was clear clustering of the responses into particular regions, it is also clear from Fig. 3 that some eye movements fell in locations well away from the three predicted locations. This was due to the fact that this plot illustrates a randomly selected subset of trials, including those in which the monkey made a large error and did not receive a reward. This shows that across the behavioral sessions, the monkey's initial pursuit speed on long-duration crossed trials was similar to that on uncrossed trials, whereas on short-duration crossed trials, the speed was reduced on nearly every trial.



View larger version (22K):
[in this window]
[in a new window]
 
Fig. 3. Single trial initial pursuit speeds in the short-duration (A) and long-duration (B) trials sampled across all behavioral sessions for both monkeys. Trials were selected as follows. For each of the 145 sessions, 1 trial was randomly selected in which a single stimulus moved in 1 direction, 1 in which a single stimulus moved in the opposite direction (these were in fact the best and null directions for the neuron being simultaneously recorded), and 3 from the crossed trial condition in which 1 stimulus moved in the "best" direction and the other moved at +90, +180, and -90° from that direction, where either stimulus was the target. Each session for each of the 2 monkeys therefore contributes 8 different data points in each panel. Eye movements were normalized (see RESULTS) to account for different stimulus directions, speeds, and eccentricities between sessions. Note that on the initial pursuit speed in the short-duration crossed trials fell between those for the 2 uncrossed trial directions, but in the long-duration trials the initial pursuit speeds were clustered near those for the uncrossed trials.

We plotted the initial pursuit direction as well as the initial saccade amplitude and direction on plots similar to those in Fig. 3. Pursuit direction also changed on short-duration but not long-duration trials as can be inferred from the plots in Fig. 3 because any such change in direction is indicated by a change in the ratio of horizontal and vertical eye speed. Saccade direction and amplitude showed the same distributions, with crossed trials clustered between the two distributions shown for the uncrossed short-duration trials and separated into two distinct clusters on crossed long-duration trials. Therefore changes in the eye movement metrics were consistent across these sample trials taken from all of the sessions in spite of the varying stimulus conditions necessitated by the requirements of the individual neurons under study.

We extended this analysis to all of the trials across all sessions by determining the error for each eye metric which was taken as the difference between the eye metric in the crossed trials and the same metric on uncrossed trials. Figure 4 shows the distribution of these errors for pursuit speed for the best direction on short-duration uncrossed trials (heavy solid line), short-duration uncrossed trials in the null direction (heavy dashed line), and crossed trials where the target moved in the best direction and the distracter moved in the null direction (thin line). The abscissa in Fig. 4 indicates the radial error for the combined differences in horizontal and vertical speeds that are normalized as in Fig. 3. If the eye speed matched the speed of the target moving in the best direction, the error should be near 0 as it is for pursuit of that target in the uncrossed trials (heavy solid line in Fig. 4). If the speed matched that of the target moving in the null direction as is the case when the monkey pursues the target moving in the null direction on the other uncrossed trials (heavy dashed line), the value should be 2.64 (right arrow in Fig. 4; see legend). The thin solid curve in Fig. 4 shows the errors observed when the monkey pursued the target moving in the best direction on the crossed trials, and these errors are centered near the midpoint as expected (1.41; left arrow). Each distribution is clearly unimodal and largely nonoverlapping, indicating that the eye movements to these different stimuli were consistent across trials. We found the same pattern of results for the initial pursuit direction, saccade amplitude, and saccade direction (not shown).



View larger version (28K):
[in this window]
[in a new window]
 
Fig. 4. Distribution of initial pursuit speed errors across all trials in all sessions. Abscissa shows the difference between the normalized eye speed on uncrossed trials (+1 for best direction; -1 for null direction as in Fig. 3). That difference is expressed as a radial error to incorporate both the horizontal and vertical speed differences: radial speed error = radical  [(hc - hu)2 + (vc- vu)2] where c and u represent crossed and uncrossed trials in the horizontal (hc and hu) and vertical (vc and vu) directions, respectively. Perfect pursuit that matches the speed of the target in the best direction is 0: radical  [(1 - 1)2 + (1 - 1)2); for pursuit of the uncrossed stimulus moving in the null direction, 2.64: radical [(-- 1)2 + (-- 1)2] (right arrow); for pursuit of the intermediate location, 1.41: radical [(0 - 1)2 + (0 - 1)2] (left arrow). Data plotted are the radial speed errors for pursuit in uncrossed trials for targets moving in the best direction (heavy solid line), the opposite, or null direction (heavy dashed line) and in crossed trials for targets moving in the best direction and the distracter moving in the null direction (thin line). Means and standard deviations of these distributions are given below the plot. Ordinate was normalized to the peak of the distribution for the uncrossed targets moving in the best direction (each plot is based on 1,643 trials). Each distribution is clearly unimodal and largely nonoverlapping, indicating that the eye movements to these different stimuli were consistent across trials.

This unimodal characteristic of the pursuit speed error shown in Fig. 4 and the unimodality of the curves for the other metrics shows that the eye movements were consistent across trials for a given stimulus condition. This unimodal curve rules out the possibility that the monkey made saccades on crossed stimulus trials to the target on some trials and the distracter on others because if that was the case, the distribution of errors in the crossed stimulus trials would be bimodal. Because the eye movement distribution in crossed trials (such as for speed in Fig. 4) was clearly unimodal, we can be confident that averaging across trial types for each session is justified, consistent with previous reports of pursuit eye movement metrics with two stimuli present (Ferrera and Lisberger 1995, 1997; Lisberger and Ferrera 1997).

Finally, it is clear from the eye movement traces in Fig. 2 that there was generally an increase in the saccade latency on the crossed trials compared with the uncrossed trials, and previous studies also indicated that a distracter stimulus can increase the latency by as much as 50 ms (Ferrera and Lisberger 1995). To determine if a similar effect occurred here, we compared the initial saccade latency between the uncrossed and crossed trial types when the target was moving in the same direction across sessions for both monkeys. The initial latency was defined as the time period between the fixation point offset and the initiation of the saccadic eye movement used to acquire the target. There was a modest but statistically significant increase in saccade latency between the two trial types for the short-duration stimuli (unpaired t-test; P < 0.01) of ~25 ms for monkey N and 42 ms for monkey P. For the long-duration stimuli, however, there was no statistically significant difference in the one monkey (N) tested.

It had been noted previously, however, that the increase in latency was largely confined to conditions in which the distracter moved in the opposite direction to the target, whereas a decreased latency occurred when the distracter moved in the same direction as the target (Ferrera and Lisberger 1995, 1997). To test this possibility, we restricted the analysis to trial conditions in which the distracter moved either at 180° from the target or at ±45° of the target, which was as close as our stimuli came to both moving in the same direction. This analysis did show that the trials with the longest latencies tended to be those with the stimuli moving at 180° of each other and that the shortest latencies tended to be those with the stimuli moving within 45° of each other, but this difference between groups did not reach statistical significance (1-tailed, unpaired t-test; P = 0.073).

Eye-movement metrics and model predictions

One of our major goals was to determine if the eye metrics were predicted more closely by the vector-average or the winner-take-all model of the underlying neuronal activity. Both models make specific predictions, and Fig. 5 shows how these predictions differ. The top panel shows the stimulus conditions for the uncrossed (left) and the crossed (right) trials when the two stimuli move at 90° to each other. The possible resulting saccadic (- - -) and pursuit (left-arrow ) eye movements are shown below each stimulus configuration. For both models, the eye movement on the uncrossed trials should accurately follow the target stimulus (middle left) but not the distracter stimulus (bottom left). For the crossed trials, the winner-take-all model predicts that the eye movements will be identical between the crossed and the uncrossed trials when the target is pursued correctly or quite different if the monkey pursues the distracter (Fig. 5, right). The vector-average model predicts that the initial eye movement will be in a different direction and at a lower speed on the crossed trials compared with the uncrossed trials (Fig. 5, bottom right).



View larger version (22K):
[in this window]
[in a new window]
 
Fig. 5. Predicted results from the winner-take-all and vector-average models. Stimulus configuration is shown for the uncrossed trials (top left) and the crossed trials (top right) at the time point corresponding to the time the stimuli started to move (Fig. 1C). Predicted eye movements on single trials are shown below each stimulus configuration as - - - representing the trajectory of the eye (saccade) and right-arrow at the end of the line representing the initial pursuit direction and speed (length of arrow). For both models, the eye movement should be very accurate in the uncrossed trials and follow the trajectory of the target stimulus ("correct trial"). Alternatively, the monkey could make an incorrect eye movement and follow the trajectory of the distracter stimulus ("incorrect trial"). For the winner-take-all model, the eye movement in the uncrossed and the crossed trials should be identical on correct trials ("WTA Target") or follow the distracter accurately on incorrect trials ("WTA Distracter"). For the vector-average model, the eye movement in the crossed trials should go in the average direction of the 2 stimuli at a lower speed ("Vector Average"). fs: fixation stimulus; dashed box: hypothetical RF.

To compare these predictions to the behavioral observations, we first found the mean of each of the eye movement metrics (initial pursuit speed and direction and initial saccade amplitude and direction) for each trial type in all sessions. We then performed regression analysis between the eye movement metrics measured on the crossed trials and the winner-take-all and vector-average predictions. Figure 6 shows this type of qualitative comparison across all sessions for monkey N using the initial direction of pursuit as an example. Figure 6A shows the clear relation between pursuit and target direction on uncrossed trials for both short-duration (left) and long-duration (right) trials and serves as a reference for the subsequent comparisons. Figure 6B compares the initial pursuit direction observed on the crossed trials with that predicted by the winner-take-all model when the monkey pursued the target moving in the same direction as in 6A. For this analysis, it was necessary to transform upward eye movement directions to prevent large apparent differences due to the circularity of this measure (see figure legend and METHODS), which is why the y-axis ranges from -90 to 450°.



View larger version (33K):
[in this window]
[in a new window]
 
Fig. 6. Initial pursuit direction was predicted by the winner-take-all or vector-average models depending on the stimulus conditions. Regression analysis of the initial eye direction for monkey N during the short-duration trials (left) and long-duration trials (right). Each data point in each plot represents the average initial pursuit direction for each stimulus condition across sessions for monkey N (80 short-duration sessions and 60 long-duration sessions). There were 14 crossed trial types in each session (7 with the distracter moving in the non-best direction and 7 with the target moving in the non-best direction) so that each plots shows 1,120 and 840 data points for the short-duration and long-duration trials, respectively. Overlap of points is therefore substantial. ---: perfect correlation. A: regression analysis between the initial eye direction (y axis) and the target direction (x axis) in the uncrossed trials (short-duration: r = 0.992; P < 0.001; long-duration: r = 0.993, P < 0.001). B: regression analysis between the initial eye direction in the crossed trials (y axis) and the target direction. Winner-take-all computation predicts perfect agreement between these 2 measures (short-duration: r = 0.881 P < 0.001; long-duration: r = 0.918; P < 0.001). y axis is from -90 to 450° due to the transformation of values to reduce differences to <180 deg. For example, target directions of 45° and measured directions of 350° were transformed to 45 and -10°, and target directions of 315° and measured directions of 15° were transformed to 315 and 375°. C: regression analysis between the initial eye direction on the crossed trials and the direction of motion of the distracter stimulus (short-duration: r = 0.568, P < 0.05; long-duration: r = 0.701, P < 0.05). D: regression analysis between the initial eye direction in the crossed trials and the direction predicted by the vector-average model (short-duration: r = 0.942, P < 0.001; long-duration: r = 0.601, P < 0.05).

The results of this analysis indicate that the winner-take-all model was a better predictor of pursuit behavior on the long- than the short-duration trials. This relation was not due to the monkey making eye movements to the distracter stimulus, however, because the relationship between the observed eye movement and the direction of the distracter stimulus showed even greater scatter (Fig. 6C). When the observed pursuit direction was compared with the prediction of the vector-average model (Fig. 6D), the vector-average model better predicted the eye movement on the short-duration trials than on the long-duration trials.

These differences between the observed pursuit and the two model predictions were quantified by directly comparing the errors between the observed and predicted eye movement metrics of the two models against each other for both monkeys (Fig. 7). In these plots, points falling below the --- indicate a smaller error by the vector-average model than by the winner-take-all model and points above the --- indicate a smaller error by the winner-take-all model than by the vector-average model. Errors for the initial pursuit direction for both monkeys (Fig. 7A) showed that the majority of the points fell below this line for the short-duration trials (left), indicating that the vector-average model more closely predicted the observed data. In contrast, on the long-duration trials the majority of points fell above the line, indicating that the winner-take-all model more closely predicted the observed data. These differences were statistically significant for both monkeys on the short-duration trials and for the long-duration trials in monkey N, the only monkey tested (P < 0.01, paired t-test; see Table 1). Three other measures of the initial eye movement showed the same result. The initial pursuit speed (Fig. 7B), the amplitude of the initial saccade (Fig. 7C), and the direction of the initial saccade (Fig. 7D) all showed smaller errors for the vector-average model on the short-duration trials and a smaller error for the winner-take-all model on the long-duration trials. These differences were also significant (Table 1).



View larger version (44K):
[in this window]
[in a new window]
 
Fig. 7. Comparison of the errors from the winner-take-all and vector-average predictions. Error plots during the short-duration (left) and long-duration (right) trials for monkeys N (open circle ) and P (+). Error between the measured parameter and that predicted by the winner-take-all model are plotted on the x axis against the error from the same measured data and the prediction from the vector-average model. ---, equal error between the 2 models. Points falling above --- indicate a greater error for the vector-average model; points falling below --- indicate a greater error for the winner-take-all model. On each graph, each data point is the error between the measured eye metric and the prediction for a single stimulus direction, as shown in Fig. 6, B and D. A: initial pursuit direction. B: initial pursuit speed. C: initial saccade amplitude. D: initial saccade direction. Data are plotted on logarithmic axes as the differences in the 2 predictions were small on trials where the 2 stimuli moved at 45° from each other and large when the 2 stimuli moved at 180° from each other.


                              
View this table:
[in this window]
[in a new window]
 
Table 1. Eye metric errors

This correspondence between the experimental data on the short-duration crossed trials and the vector-averaging model also raises an issue related to the strategy the monkey may have been using in these experiments. If the monkey adopted the strategy of making the initial saccade to a location near the intersection of the two stimulus objects, it then could use near foveal vision to make the appropriate shape discrimination and then make subsequent saccadic and pursuit eye movements to follow the target. The most compelling evidence against this possibility is that there was a significant pursuit speed immediately after the initial saccade whether the motion of the two stimuli was different by 45, 90, or 135°. This is shown by the smaller, but nonzero eye speeds of Fig. 3 and the small errors of the vector-average prediction in Fig. 7B. If the monkeys were adopting the preceding strategy, they would not generate a pursuit movement to a nonexistent stimulus that was consistent with the vector-average motion of the two stimuli. We therefore conclude that the vector average is more likely reflecting the representation of the visual stimuli than a change in the monkey's oculomotor strategy in the crossed trials.

We also compared the initial saccade metrics directly to the initial pursuit metrics. For the short-duration crossed trials in particular, it may be that there was a difference between these two components of the eye movement. For example, it may be that the initial saccade was made appropriately, but the initial pursuit eye movement followed the vector average of the two stimuli. Inspection of Fig. 7 suggests that this is unlikely, as both the initial saccade direction and amplitude more closely followed the vector-average model on the short-duration trials and the winner-take-all model on the long-duration trials. To verify this statistically, we measured the error between the eye metric (initial saccade direction and amplitude, pursuit direction and speed) for all correct trials in both monkeys and subjected these data to factor analysis. If the initial saccade metrics and pursuit metrics had different errors, this should result in these data being composed of at least two factors. We found that the data were best represented by one factor for each of the four trial types, indicating that across trials the metrics of the initial saccade and pursuit eye movements covaried with each other. From this analysis, we conclude that both the saccadic and pursuit eye movements were affected equivalently. This similarity of smooth pursuit and saccades is also consistent with the deficits in both pursuit and saccades made to moving targets following localized lesions of MT (Newsome et al. 1985; Schiller and Lee 1994); both the amplitude of the initial saccade and the pursuit speed to a moving target were incorrect following the lesion.

Finally, we considered the possibility that the stimulus velocity could alter the distribution of errors. We used three different velocities, tailored to the best response of the neuron under study (see METHODS). The majority of our stimuli were at 2 pixels/frame to have the stimuli traverse as much of the RF of the neuron as possible, as most neurons had RFs between 10 and 20° in radius (72/145 experiments), the second most commonly used velocity was 1.5 pixels/frame (52/145 experiments) with 1 pixel/frame (21 experiments) being the least common. An ANOVA test showed no statistically significant effect of velocity on any metric for either the short- or long-duration trials or in either monkey (all P values > 0.05), indicating that stimulus speeds within the range of 8-25°/s resulted in similar effects on the eye movement metrics.

In summary, the analysis of the errors indicated that the eye movements were best predicted by a vector-averaging model when the two stimuli intersected on short-duration trials but that a winner-take-all model best predicted the eye movements on long-duration trials. Thus by only slightly increasing the spatial distance and separation timing of the stimulus motion, it was possible to convert the eye metrics to identical stimuli from being more closely predicted by a vector-average model to a winner-take-all model.

Neuronal responses in MT and MST

The activity of single neurons in cortical areas MT and MST were recorded at the same time as the measurements of the initial eye movements, and Fig. 8 shows an example of the responses of an MT neuron recorded during the same trials illustrated in Fig. 2. On the uncrossed trials (left), this MT neuron gave a robust response while the stimulus was moving in the best direction within its RF. The neuron had a particularly large onset response on the short-duration trials (Fig. 8, A and B), as the stimulus appeared in motion at the edge of the RF. In contrast to this robust activity to single stimuli, there was a significant reduction in the response on the short-duration trials when a distracter moved through the RF in the null direction at the same time the target moved through the RF in the best direction (Fig. 8B). This reduction in the response was not nearly as apparent for the long-duration crossed stimulus condition (Fig. 8D), although there was still a reduced response for crossed trials compared with the uncrossed trials (Fig. 8, D compared with C).



View larger version (43K):
[in this window]
[in a new window]
 
Fig. 8. Representative middle temporal area (MT) neuron showing differential responses depending on the stimulus conditions. Single-unit responses during uncrossed (left, A and C) and crossed (right, B and D) trials from the neuron recorded during the same trials illustrated in Fig. 2. Inset: stimulus configuration in the horizontal dimension including the location of the RF (dashed box). Each tic in the raster represents a single action potential, each row represents a single trial. Poststimulus time histograms (PSTHs) were constructed from these rasters using 3-ms time bins. Horizontal dashed lines show the horizontal extent of the RF. Location of the RF is illustrated by the vertical black line at the time point that the stimuli are intersecting in the crossed trials. Panels show the response of the neuron during the short-duration uncrossed trials (A), short-duration crossed trials (B), long-duration uncrossed trials (C), and long-duration crossed trials (D). These plots are offset in the horizontal axis to align all plots to the same stimulus configuration, as in Fig. 2.

For each neuron, we calculated the ratio of the response during the crossed and uncrossed trials (crossed trial response/uncrossed trial response). The example neuron shown in Fig. 8 had a ratio of 0.61 for the short-duration trials and 0.89 for the long-duration trials. Across the sample of neurons in MT, the mean and standard deviation of the ratio for the short-duration trials was 0.63 ± 0.32 and for long-duration trials the population average was 0.96 ± 0.32. For MST neurons, these values were 0.66 ± 0.31 for short-duration trials and 0.89 ± 0.32 for long-duration trials. This indicates that the difference in responses between the short- and long-duration trials under these two stimulus conditions was fairly uniform across the population of neurons in each area and that there was little difference between areas MT and MST (unpaired 1-tailed t-test; P > 0.05 for both short- and long-duration trials).

To compare the neuronal activity to the previous measures of the initial eye-movement metrics, we first determined whether the responses of the MT and MST neurons also were better predicted by the vector-average model or the winner-take-all model depending on the stimulus conditions. As we did for the eye movements (Fig. 6), we compared the observed neuronal responses with those predicted by the winner-take-all and vector-averaging models. These comparisons are shown in Fig. 9 for all neurons recorded in area MT in monkey N. This figure shows the responses when the target for the eye movement was in the best direction (Fig. 9, A and B, ) and the responses when the target for the eye movement was in one of the seven non-best directions (Fig. 9, C and D). In each case, the regression coefficients indicated a better fit for the vector-average prediction on the short-duration trials than the long-duration trials, whereas the analysis showed a better fit for the winner-take-all prediction on the long-duration trials than the short-duration trials. This was also apparent from the distribution of points when the responses were compared against the winner-take-all prediction on the short-duration trials (Fig. 9, A and C, left). When the target was moving in the best direction, the responses were smaller than expected, presumably because the response on crossed trials was the average of the response on the uncrossed trials where the stimulus moved in the best direction and a non-best direction (which by definition had a smaller response). Similarly the responses were greater than predicted when the target was moving in a non-best direction for the same reason. This difference necessarily would be smallest for neurons with either low firing rates and on trials where the two stimuli moved in directions near each other. For stimuli moving along the best-null axis, this difference was greatest, as indicated by the greater scatter of points for the higher firing rates. The result for neurons located in cortical area MST was similar (Fig. 10).



View larger version (33K):
[in this window]
[in a new window]
 
Fig. 9. Activity of MT neurons followed either the vector-average or winner-take-all models depending on the stimulus conditions. Regression analysis from all neurons recorded in cortical area MT in monkey N on short-duration (left) or long-duration (right) trials. Average response for each of the 8 directions of motion in each session is represented by a single point, corresponding to (8 × 45 sessions = 360) and (8 × 27 sessions = 216) data points for the plots of the short-duration and long-duration trials, respectively. A: winner-take-all prediction. Response during the uncrossed trials is plotted with the response in the crossed trials when the target was moving in the best direction of the neuron under study. Winner-take-all model predicts perfect correlation in this case. Vertical alignment of responses in A is due to the same firing rate used in the uncrossed trials (target moving in the best direction) and compared with each of the crossed trial conditions where the target moved in the best direction and the distracter moved in each of the 7 non-best directions (short-duration: r = 0.919, P < 0.001; long-duration: r = 0.944, P < 0.001). B: vector-average prediction is based on the average response from the uncrossed trials in each of the 2 directions, and is plotted against the response measured in the crossed trials when the target was moving in the best direction (short-duration: r = 0.902, P < 0.001; long-duration: r = 0.812, P < 0.001). C and D: same analysis as in A and B when the target was moving in the non-best direction and the distracter in the crossed trials was moving in the best direction (C, short-duration: r = 0.776, P < 0.001; long-duration: r = 0.906, P < 0.001; D, short-duration: r = 0.917, P < 0.001; long-duration: r = 0.881, P < 0.001). Diagonal line: perfect correlation.



View larger version (32K):
[in this window]
[in a new window]
 
Fig. 10. Single medial superior temporal area (MST) neurons followed either the vector-average or winner-take-all models depending on the stimulus conditions. Same analysis and conventions as in Fig. 9. All regression analysis had P values < 0.001. A: short-duration: r = 0.896, long-duration: r = 0.923. B: short-duration: r = 0.915; long-duration: r = 0.872. C: short-duration: r = 0.651; long-duration: r = 0.938. D: short-duration: r = 0.878; long-duration: r = 0.836.

We made a direct comparison between the predictions of the two models that was similar to that already described for the eye movement metrics (Fig. 7). Figures 11 and 12 show the results of this analysis on the MT and MST neurons, respectively. Like the results described for the eye movements, the responses on the short-duration trials were better predicted by the vector-average model and the responses on the long-duration trials were better predicted by the winner-take-all model. These differences were statistically significant (see Table 2) when the best direction and the seven other directions were tested (Figs. 11 and 12, left) and when only trials in which the target and distracter moved along the best-null axis were considered (Figs. 11 and 12, right). Again, analysis of the neuronal responses as a function of stimulus speed showed no difference in the effects on the single neuron activity as was the case for the eye movement metrics (ANOVA, all P values > 0.05).



View larger version (50K):
[in this window]
[in a new window]
 
Fig. 11. Winner-take-all and vector-average models better predicted single MT neuron activity depending on the stimulus conditions. Error between the measured response on the crossed trials and the winner-take-all model (x axis) and the vector-average model (y axis) are plotted for all directions of motion (left) or when the stimuli moved only along the best-null axis (right). A: results from the short-duration trials. B: results from the long-duration trials. Each data point represents a single stimulus direction and plots the same data as shown in Fig. 9. Conventions as in Fig. 7.



View larger version (52K):
[in this window]
[in a new window]
 
Fig. 12. Winner-take-all and vector-average models better predict single MST neuron activity depending on the stimulus conditions. Same analysis and conventions as shown in Fig. 11.


                              
View this table:
[in this window]
[in a new window]
 
Table 2. Errors for MT and MST neurons

The results thus far indicate that the same cortical neurons can alter their activity between that better described by a vector-average model and that better described by a winner-take-all model. The change in processing depends on the stimulus conditions, but which difference in the stimulus conditions is important remains unclear. One change is the length of time that the monkey had to make the shape discrimination, select the correct stimulus as the target, and plan the pursuit eye movement. If this length of time altered the monkey's behavior and/or neuronal responses of MT and MST neurons, this should be evident by comparing the saccade latency between trials where the winner-take-all model was a better predictor with the saccade latency where the vector-average model was a better predictor. We therefore compared the saccade latency between these two trial types (those points falling above vs. below the line of perfect correlation shown in Figs. 11 and 12). In most cases, there was a trend for the trials that were better predicted by the winner-take-all model to have longer latencies than for trials that were better predicted by the vector-average model (Table 3). In some cases, this difference reached statistical significance, but these differences were generally quite small (rarely as much as 10 ms; see Table 3). Thus although there was a tendency for the trials best described by the winner-take-all model to have a longer latency, this effect was small, and we conclude that it is unlikely to be an important indicator of the shift from a vector-average to a winner-take-all mechanism.


                              
View this table:
[in this window]
[in a new window]
 
Table 3. Latency differences between WTA and vector-averaging predictions in MT and MST neurons

In summary, these results showed that the responses of single neurons in MT and MST can be predicted by either a vector-average model or a winner-take-all model depending on the stimulus conditions. These conditions were identical to those that showed a similar difference in the accuracy of the two models in predicting the metrics of the initial eye movements. Thus in the same monkey, and even on interleaved trials, both the eye movements and the responses of these single neurons could be reliably, and substantially, altered by providing an extended period of time and spatial distance of stimulus motion in which to plan the eye movement.


    DISCUSSION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

We used the intersecting motion of two stimuli to determine how populations of neurons process visual motion for the initiation of pursuit eye movements. We made two salient observations. First, by increasing the spatial and temporal separation of the two stimuli in the visual field, we found that we could shift the model of the initial saccadic and pursuit eye movements that better predicted the behavior from a vector average of the two stimuli to a winner-take-all model of the target stimulus. Second, recording from single neurons in areas MT and MST showed that the model best predicting the activity of these neurons also shifted from vector average to winner take all under the same stimulus conditions that caused the shift in the eye movements. Thus the activity of the MT and MST neuronal populations paralleled changes in oculomotor behavior. Although it is possible that other models could better predict these results, we have concentrated on contrasting the winner-take-all and vector-averaging models for two main reasons. First, other models tested by us and others have shown very poor predictive power for either eye movements or neuronal activity, including vector summation and probability summation (Ferrera and Lisberger 1997; Groh et al. 1997; Lisberger and Ferrera 1997; Recanzone et al. 1997) Second, several other studies have found either winner-take-all or vector-averaging predictions under similar stimulus conditions. Although nonlinear or other types of models may show better predictive power for these measures, it is clear from this study as well as previous studies that these linear models capture the essence of both the neuronal activity and initial pursuit movements. We first will discuss the factors that influence this shift in behavior from vector-averaging to winner-take-all models in the context of this and previous experiments and then consider the relation of the change in neuronal activity to the hypothesized population coding.

Model predictions of pursuit behavior

Under our experimental conditions, when the two stimuli were moving relatively close together in space and there was little time before the monkey was required to initiate pursuit, the initial eye movement metrics were better predicted by the vector average of the eye movements when the two component stimuli were presented alone in that visual field. Close together in our experiments meant that the two stimuli crossed within 150 ms after target motion began and traveled over a course of 1° to ~4°, depending on the stimulus velocity and direction. In addition, because the monkey had the information about which stimulus was to be the pursuit target only after the appearance of the moving stimuli, it had only 150 ms to make the appropriate shape discrimination before pursuit initiation was allowed. We think these results are consistent with those of several recent experiments that had generally similar stimulus conditions. Lisberger and Ferrera (1997) used a paradigm in which the two stimuli moved 3° and intersected at the fixation point 150 ms after motion onset, but it was only at that time that the correct pursuit target was indicated (by the other stimulus disappearing) and so the pursuit was triggered as soon as the target was identified. They found that a vector average best predicted the subsequent eye movements. When combining the motion of a visual target with electrical stimulation of MT, Groh et al., (1997) also found the vector average to be the best predictor of the observed pursuit movements. In their experiment, the visual target was positioned to move through the center of the RFs of the neurons electrically stimulated so the activity related to the two "stimuli" overlapped. The trigger to initiate pursuit of the visual target usually occurred at the same time that the motion of the visual target and the electrical stimulation began, so again there was little time before pursuit initiation was required. Thus although there are substantial differences between these experiments, they all had overlapping stimuli that moved over a relatively short distance for a short time, and they gave the monkey only a short time to plan the pursuit initiation.

When we used stimuli with increased separation of the two stimuli in space, which also allowed the monkey a longer time between target identification and the trigger to initiate pursuit, the eye movement metrics were better predicted by a winner-take-all model rather than a vector-average model. This increased separation in our experiments extended the duration of the target motion from 150 to 450 ms before the monkey was allowed to initiate pursuit and extended the distance traveled by these stimuli from 3.5 to 11°. Ferrera and Lisberger (1995, 1997) also used a paradigm in which a winner-take-all model best predicted the initial pursuit, but the stimulus conditions were substantially different. The two stimuli did not overlap (Ferrera and Lisberger 1997) and could be in different hemifields and so did not have overlapping stimuli comparable with those in our experiment. In their experiments, the fixation period before the trigger for pursuit was 300 ms, closer to our 450-ms period, but they commented that they obtained the same pursuit initiation for periods between 100 and 800 ms. The similarity of their experiment and ours is the lack of stimulus proximity during at least the early phases of pursuit initiation, although in all of the results shown there was a 300-ms period before the pursuit-initiation trigger. Thus although both their experiments and ours provide nonoverlapping stimuli for a longer period during the trial and a longer time for the monkey to plan the pursuit initiation and the pursuit initiation in both was best described by a winner-take-all model, the differences between the paradigms make it difficult to conclude whether it is the same variables that lead to the winner-take-all prediction.

Another set of experiments also has shown a shift from an average to a winner-take-all prediction using motion stimuli (Zohary et al. 1996). In these experiments, the direction of motion of two overlapping directions of predominant motion of randomly moving dots was perceived as an average when the directions of the two motions varied by little and as a winner-take-all when one stimulus direction became stronger than the other. Some subjects also could switch back and forth between the two modes depending on the task instructions, indicating that the underlying computation of the population activity is not dependent just on the stimulus conditions. Thus in both these psychophysical experiments and our pursuit-initiation experiments, the description could be shifted from an average to a winner-take-all by modifying the stimulus conditions and/or modifying the conditions under which the target selection was made: an increase in fixation time in pursuit initiation or task instructions in the case of psychophysical judgments.

Unfortunately our experiments on pursuit initiation do not resolve which of several confounded variables underlies the shift in the computational mechanism. The stimulus features that varied together were the distance and duration of the motion stimulus. By initiating the target motion at a greater distance from the intersection point, given that the stimuli were moving at the same speed, there was also a longer period of time that the motion was present before the intersection of the two stimuli. It is therefore not possible to determine from our experiments the degree to which these two differences influenced the change in pursuit initiation. A second major factor is the differences in time allowed before pursuit initiation was triggered that allowed more or less time for the monkey to shift its attention to the target stimulus and/or prepare for the movement. We cannot determine whether it is the shift in stimulus conditions or the time allowed before pursuit initiation that is most important, although from even the brief summary of previous experiments, it is clear that both are involved.

The observations of a shift from vector average to winner take all for motion stimuli is reminiscent of a similar well-established shift for saccades when two stationary targets are presented to human subjects (Coren and Hoenig 1972; Findley 1982). When the two targets are close together, saccades are made to a point between them, the average of the two locations, whereas when the targets are further apart, the saccades are made to one target or the other, in a winner take all fashion. In so far as the population activity underlying this saccadic averaging or separation is similar to that for the initiation of pursuit to a moving stimulus, it would strongly suggest that the key factor is the separation of the two pursuit targets. Such averaging or winner-take-all shifts also have been demonstrated in monkeys (Ottes et al. 1984). However, even in these studies of averaging saccades, subsequent experiments have demonstrated that factors other than the presence of the two stimuli can influence the behavior and by inference the underlying computation. He and Kowler (1989) showed in human experiments, for example, that the average end location of saccades shifts based on the prior history of target locations with no change in the current stimuli. Furthermore recent experiments in the monkey (Edelman and Keller 1998) showed that averaging saccades were more likely when saccade latency was short.

The underlying computations for saccades in the presence of two targets also shows sensitivity both to the configuration of the targets and to other factors including the time before the movement is made so that this control of saccades has striking similarities to the control of pursuit.

For both the pursuit and saccadic systems, the shift in type of movement made is hardly surprising and in fact makes biological sense. When the stimuli are close together in time and space and little time is available before the saccade is made, the pursuit or saccade is made to an average of the two targets. This would appear to be a reasonable strategy because an average approximation would bring both targets closer to the fovea quickly and a subsequent saccade or pursuit movement could be made if necessary. On the other hand, when the stimuli are farther apart and there is more time before the movement, the pursuit or saccade can be made accurately to one target or the other. This strategy brings the target on the fovea with only one saccade or pursuit movement.

Neuronal activity and model predictions

In both MT and MST, the neuronal behavior shifted from a vector-average to a winner-take-all prediction as the spatial-temporal motion of the stimulus and the time available before pursuit initiation changed, just as was the case with the pursuit and saccadic behavior. For the short-duration trials, the fit of the observations with the vector averaging was similar to that found in our previous experiments in which the two stimuli moved across the RFs of MT and MST neurons while the monkey continued to fixate throughout the trial (Recanzone et al. 1997), which is not surprising given that the stimulus conditions were almost identical. This similarity in results with pursuit initiation and without it does suggest, however, that on the short-duration trials, the stimulus conditions were the critical variables. This argues for a relatively greater importance of the stimulus configuration than time available before the movement initiation on the short-duration trials. The averaging result is also consistent with models of MT using summation with divisive normalization (Carandini and Heeger 1994; Heeger et al. 1996; Simoncelli and Heeger 1998).

The relation of the long-duration trials to a winner-take-all prediction is consistent with the pursuit and saccade behavior, but it does not have an obvious precedent at the neuronal level. What is surprising is that the neurons in both MT and MST behaved so similarly. One might have expected to see some indication of a shift in processing from one area to the next, particularly given the substantial change in the RF sizes between these two areas (Desimone and Ungerleider 1986; Komatsu and Wurtz 1988).

A limitation of our analysis of the neuronal responses is the method we used to generate the vector-average and the winner-take-all model predictions. For the eye-movement predictions, we compared the eye movement metrics using the behavior on uncrossed trials to predict the behavior on crossed trials. This is appropriate because the movement to each of the component stimuli on the crossed trials predicts the true vector average of the eye movement, and the errors between observed and predicted were relatively small for at least one of the two predictions. However, for the neuronal responses we used a scalar average, not a vector average, of the responses to the two-component stimuli on the crossed trials. This scalar average does not predict the same response if MT and MST neurons also are tuned for small variations of stimulus speed. For example, the scalar average of two stimuli moving at a 90° angle predicts a response for the intermediate direction and the same speed, whereas a vector average predicts the response to a stimulus moving in an intermediate direction at a speed 1.41 times greater. Stimulus speed does influence the activity of MT neurons (Allman et al. 1985; Lagae et al. 1993; Maunsell and Van Essen 1983; Mikami et al. 1986), although the speed-tuning functions for such small differences are not easily predicted from those data. Because we could not directly measure the responses of these neurons to single stimuli moving at each of the possible vector-average directions and speeds, this may in part account for the errors observed in the short-duration crossed trials. Across the population of neurons, the vector-average response would in some cases be greater than the prediction from the scalar average, and in other cases the vector-average response would be smaller than the scalar average. Thus the difference would be expected to increase the variance in the error across the population of neurons but likely not the mean. Taking the stimulus speed selectivity into account may well produce smaller overall errors on the short-duration crossed trials.

If we consider only the stimulus conditions in our experiments and take the view that MT and MST essentially consists of velocity maps of visual space, we can visualize the changes that might occur when comparing the short- and long-duration trials (Fig. 13A). In this view, the motion of the two stimuli at any given time can be represented on such a velocity map by two independent regions or hills of activity. When the stimuli are separated, as in the long-duration trials and the uncrossed trials, the two hills of activity are clearly separated (Fig. 13A, left), and pursuit can be to either target represented by either of the separate hills. In contrast, when the stimuli cross in close temporal and spatial proximity, the hills of activity on the velocity map can be regarded as overlapping (Fig. 13A, right), and the pursuit is to the average of the two hills. In this case, neurons show a reduced response to the best stimulus direction and increased activity to the non-best directions so that on this velocity map, the distribution of activity would be the average of the two component stimuli. The changing interaction between these two regions as stimuli approach each other must depend on both excitatory and inhibitory interactions as has been proposed in models of the interaction (Ferrera and Lisberger 1995, 1997).



View larger version (50K):
[in this window]
[in a new window]
 
Fig. 13. Hypothetical population activity in MT and MST. A: activity of the population of neurons plotted in a velocity space with axes of horizontal and vertical velocity (x and y axes) with neuronal activity on the z axis. Peaks of activity are represented as discrete hills for individual stimuli, which on long-duration trials were well separated (left) but were overlapping on short-duration trials (right). Note that these plots are for a velocity space and are not intended to represent a spatial map of either MT or MST. B: 2 different types of possible cortico-cortical connections. Left: neurons with RFs (- - -) and best directions (right-arrow) that are aligned with the trajectory of a moving stimulus and would provide excitatory input to the neuron under study, whose RF is denoted by the RF. Right: RFs of many neurons with the same RF location in visual space but different best directions, which would result in divisive normalization, and therefore apparent inhibition, of the neuron under study. In long-duration trials, the target stimulus would activate neurons with RFs with the same best directions that are interconnected with the neuron under study. This would result in an increase in activity of the neuron under study, approaching the winner-take-all prediction. In the short-duration trials, such a cortico-cortical interaction does not occur, and thus the response of the neuron is only influenced by neurons with similar RF locations with different best directions. Thickness of the - - - marking the RF indicate the influence on the responses of the recorded neuron (heavy lines: excitatory, thin lines: inhibitory).

When viewed in this way, the hypothetical interactions between populations of neurons in MT or MST during pursuit initiation are similar in several respects to the neuronal activity in the monkey superior colliculus thought to be associated with the initiation of saccades made in the presence of two stationary targets. It generally is accepted that the activity related to an impending saccade can be represented by a hill of activity among the saccade related burst neurons on the collicular map and that saccade generation results from the vector summation of this activity (van Gisbergen et al. 1987; van Opstal and van Gisbergen 1989; and see review by van Gisbergen and van Opstal 1989) although the experimental evidence is sparse (Lee et al. 1988). Several studies have shown that the neurons in the colliculus are active during averaging saccades (Edelman and Keller 1998; Glimcher and Sparks 1993; van Opstal and van Gisbergen 1990). Edelman and Keller (1998) also found that when the averaging saccades occurred, the movement fields of the collicular neurons were broader than when saccades were made to single targets. The conceptions of the shifts in the activity of the neurons that might underlie these saccades (Edelman and Keller 1998; Krommenhoek and Wiegerinck 1998; van Opstal and van Gisbergen 1990) are similar to those depicted for the pursuit eye movements. The difference is that while the map for activity preceding pursuit is in velocity space, the map for activity preceding the saccade would be in retinotopic space. Thus while the space on which the activity is plotted is different, there are substantial similarities between the conception of the underlying maps and how the shift from an average to a winner-take-all might come about at the level of neuronal populations.

One possible interaction within the velocity map that is consistent with several of our observations is an interconnection of neurons that have the same velocity vector. The increased spatial separation of stimuli in the long-duration trials would allow cortical neurons with the same velocity preference that were activated early in the stimulus path to influence the activity of the neuron later in the path (centered in our experiments on where the stimuli crossed). Figure 13B shows a schematic drawing of how such a mechanism based on the direction preference of columns of neurons in area MT or MST could implement such modulation. Figure 13B, left, shows the RFs and best directions of neurons that, if interconnected in an excitatory fashion, would be activated sequentially and positively influence the activity of the next column of neurons as a stimulus traversed the visual field. Such sequential activation commonly would occur in nature either during self-motion or while viewing the linear motion of other objects. These interactions are consistent with the influences of stimuli moving in the best direction of the recorded neuron up to two or more RF diameters away on MT neurons (Britten and Heuer 1999) and with the anatomic intracortical connections described between neurons with similar orientation selectivity in striate cortex (see Gilbert et al. 1996), but whether these connections exit between neurons with similar direction preference in MT has not been determined. A second type of cortico-cortical interaction is shown in Fig. 13B, right, where neurons with RFs in the same region of visual space, but with different best directions, provide inhibitory input to each other. There is also good evidence that such interactions occur in MT and MST neurons, for example, the demonstrations of divisive normalization when stimuli move in different directions (e.g., Recanzone et al. 1997; Snowden et al. 1991).

Our results indicate that the influence of neurons with RF locations and directional tuning consistent with the target trajectory is greater than the influence of neurons with different best directions and the same RF location. In this case, divisive normalization would result in responses closer to a winner-take-all model in the long-duration trials because of the excitatory input from other cortical neurons responding to the target as it moved toward the RF. This long-range excitatory input would provide a stronger input to the recorded neuron than would the columns of neurons responsive to the direction of the distracter. This would result in activity that was closer (but still lower) to that measured in trials where only one stimulus was in the RF of the neuron and would effectively shift the response from being consistent with a vector-average model to a winner-take-all model. On the short-duration trials, all stimuli initiated their motion near the edges of the RF of the neuron under study, and thus interactions from neurons one or more RF diameters away would be minimal. A direct test of this hypothesis would be to present stimuli that frequently change in direction, thereby not activating these hypothetical long-range connections. We predict that in this case, both the neuronal activity and the initial smooth pursuit eye movement would be better predicted by a vector-average model, in spite of the fact that the monkey would have the same amount of time to identify the target stimulus under both conditions.


    ACKNOWLEDGMENTS

The authors thank their colleagues at the Center for Neuroscience (particularly K. H. Britten, H. Heuer, and S. Elfar) and at the Laboratory of Sensorimotor Research for suggestions on previous versions of the manuscript, U. Schwarz for support during the collection of data for these experiments, and the National Institutes of Health Laboratory of Diagnostic Radiology Research for providing the MRIs of the monkeys.

Funding was provided by the National Eye Institute (R. H. Wurtz and G. H. Recanzone) and the National Research Council, National Institute on Deafness and Other Communication Disorders Grant DC-02371, The Klingenstein Fund, and the Sloan Foundation (G. H. Recanzone).


    FOOTNOTES

Address for reprint requests: R. H. Wurtz, Laboratory of Sensorimotor Research, Building 49, Room 2A50, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD 20892-4435.

The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Received 24 March 1999; accepted in final form 1 June 1999.


    REFERENCES
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

0022-3077/99 $5.00 Copyright © 1999 The American Physiological Society