Responses of MT and MST Neurons to One and Two Moving Objects in the Receptive Field

G. H. Recanzone1, R. H. Wurtz2, and U. Schwarz2

1 Center for Neuroscience, Section of Neurobiology, Physiology, and Behavior, University of California, Davis, California 95616; and 2 Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland 20892

    ABSTRACT
Abstract
Introduction
Methods
Results
Discussion
References

Recanzone, G. H., R. H. Wurtz, and U. Schwarz. Responses of MT and MST neurons to one and two moving objects in the receptive field. J. Neurophysiol. 78: 2904-2915, 1997. To test the effects of complex visual motion stimuli on the responses of single neurons in the middle temporal visual area (MT) and the medial superior temporal area (MST) of the macaque monkey, we compared the response elicited by one object in motion through the receptive field with the response of two simultaneously presented objects moving in different directions through the receptive field. There was an increased response to a stimulus moving in a direction other than the best direction when it was paired with a stimulus moving in the best direction. This increase was significant for all directions of motion of the non-best stimulus and the magnitude of the difference increased as the difference in the directions of the two stimuli increased. Similarly, there was a decreased response to a stimulus moving in a non-null direction when it was paired with a stimulus moving in the null direction. This decreased response in MT did not reach significance unless the second stimulus added to the null direction moved in the best direction, whereas in MST the decrease was significant when the second stimulus direction differed from the null by 90° or more. Further analysis showed that the two-object responses were better predicted by taking the averaged response of the neuron to the two single-object stimuli than by summation, multiplication, or vector addition of the responses to each of the two single-object stimuli. Neurons in MST showed larger modulations than did neurons in MT with stimuli moving in both the best direction and in the null direction and the average better predicted the two-object response in area MST than in area MT. This indicates that areas MT and MST probably use a similar integrative mechanisms to create their responses to complex moving visual stimuli, but that this mechanism is further refined in MST. These experiments show that neurons in both MT and MST integrate the motion of all directions in their responses to complex moving stimuli. These results with the motion of objects were in sound agreement with those previously reported with the use of random dot patterns for the study of transparent motion in MT and suggest that these neurons use similar computational mechanisms in the processing of object and global motion stimuli.

    INTRODUCTION
Abstract
Introduction
Methods
Results
Discussion
References

A subset of neurons in the middle temporal visual area (MT) of monkey extrastriate cortex provides important motion information about large stimuli, such as random dot patterns (Newsome and Pare 1988; Qian and Andersen 1994; Snowden et al. 1991, 1992) and sinewave or squarewave gratings (Movshon et al. 1985; Rodman and Albright 1989; Stoner and Albright 1992, 1996). Recent studies have concentrated on the interaction of two such stimuli moving in the receptive field of a neuron by using either grating stimuli (Movshon et al. 1985; Rodman and Albright 1989) or random dot patterns (Qian and Andersen 1994; Snowden et al. 1991, 1992) and have shown that the introduction of a second stimulus can affect the response of most cells in MT. Qian and Andersen (1994) suggest that MT neurons suppress the inputs from local regions within the receptive field of the neuron that contain stimuli moving in opposite directions and Movshon et al. (1985) and Rodman and Albright (1989) have suggested that MT is the first cortical area in which different directions of motion of the individual components of a complex stimulus are combined to form a representation of the global motion of the stimulus.

In addition to processing information related to the perception of such global motion, MT provides the oculomotor system with the appropriate motion information to code smooth pursuit and saccadic eye movements to moving targets (Dursteler and Wurtz 1988; Dursteler et al. 1987; Erickson and Dow 1989; Komatsu and Wurtz 1988; Newsome et al. 1985; Yamasaki and Wurtz 1991). In contrast with the large pattern stimuli used in the experiments on global motion, these experiments on MT and its neighbor, MST, used isolated moving spots as targets. Neurons in these areas discharge in relation to pursuit eye movements and damage to these areas alters the control of pursuit. All of these oculomotor experiments used single pursuit targets and, in light of the striking interactions demonstrated between multiple global stimuli, we investigated the responses of neurons when two discrete spots of light moved through the receptive field. Because both MT and MST contribute to pursuit, we also determined whether or not these responses to spot stimuli changed between MT and MST. Comparison of the interactions between such object stimuli with the interactions between multiple larger field global stimuli should indicate whether or not processing is similar for the two types of stimuli. In short, can the same neuronal mechanisms account for the processing of two distinct types of motion information?

The results of this study indicate that most neurons in both MT and MST are strongly influenced by the presence of a second stimulus moving in a direction different from the first. The responses to the two-object stimuli were greatest when one of the two simultaneously presented stimuli moved in the best direction and smallest when one of the two stimuli moved in the null direction, yet were always greater than when a single stimulus moved in the non-null direction. The magnitude of the response was directly related to the relative strength of single stimuli moving through the receptive field and could be accurately predicted by taking the average of the two single-object stimulus responses. The overall effects and the strength of the averaging prediction were stronger in MST than in MT. Thus single neurons in MT and MST weigh inputs across all directions of motion in generating their response to complex moving stimuli. We think these results are consistent with previous reports of modulation of MT neurons to either moving gratings (Movshon et al. 1985; Rodman and Albright 1989) or random dot patterns (Qian and Andersen 1994; Snowden et al. 1991), indicating that similar computational mechanisms are used in these cortical areas for the different stimuli.

A brief report of these results has appeared previously (Recanzone and Wurtz 1994).

    METHODS
Abstract
Introduction
Methods
Results
Discussion
References

Behavioral tasks

We recorded single neurons from three hemispheres of two adult male rhesus monkeys (Macaca mulatta). The monkey sat in a primate chair facing a tangent screen placed 57.4 cm from the animal. Visual stimuli were back-projected onto the 100 × 100° screen using a video projector (Electrochrome, SVGA, 1024 × 768 pixel resolution). Individual pixels subtended a visual angle of 0.12° vertically and 0.13° horizontally. Images were created on an 80486-based PC with software specifically designed to generate moving stimuli and were presented at a rate of 72 Hz. Stimuli were brighter (1.8 cd/m2) than the background (0.2 cd/m2). Five different objects were used (circle, square, diamond, plus sign, and triangle) but only one type of object was used in recording from each neuron. Each object subtended a maximum visual angle of 1.8° and all were equal in luminance and size (equal numbers of pixels/stimulus). Objects were moved by displacing the illuminated pixels by one or two pixels between each frame in either the horizontal, vertical, or both directions. Stimuli moved at 1, 1.5, or 2 pixels/frame corresponding to ~9, ~13.5, or ~18°/s along the horizontal and vertical directions and 12, 18.5, or 25°/s along the obliques, respectively. For velocities at 1.5 pixels/frame the image would alternate between displacement of 1 and 2 pixels per frame. At these frame rates the perception by human observers was that these stimuli moved at a constant velocity. The particular object and the stimulus velocity chosen were those that elicited the most vigorous response during preliminary characterization of the cell's receptive field and direction tuning properties.

Each monkey was trained to look at a projected fixation stimulus (0.5° square) for a variable period of time for a fluid reward. Eye position was recorded by using the magnetic search coil technique (Fuchs and Robinson 1966; Judge et al. 1980). The monkeys were rewarded for keeping their eyes within ±0.75° of the fixation stimulus. After acquiring the fixation target, there was a variable delay of 300-500 ms before one or two visual objects were presented in motion in a direction toward the center of the receptive field of the neuron under study (hereafter referred to as one-object and two-object stimuli). We attempted to locate the stimuli within the receptive field of the neuron at stimulus onset to avoid presenting the stimuli in the potentially inhibitory surround of the excitatory receptive field center (Allman et al. 1985; Tanaka et al. 1986). The stimulus objects were located within the receptive field at stimulus onset for all MST neurons and 29/48 MT neurons. The remaining MT neurons had the smallest receptive fields and the stimulus was initially presented outside of the excitatory receptive field boundary. The response properties of these neurons were analyzed separately and showed no statistically significant differences or apparent trends when compared with the remaining MT neurons on any aspect of this report, so all MT neurons were pooled.

The stimuli were located at the center of the receptive field 150 ms after stimulus onset. In the case of the two-object stimuli, the two objects merged to become completely overlapping, making a single object with the same dimensions and luminance as each individual stimulus for one video frame (Fig. 1) and then continued their trajectories through the receptive field to once again become two distinct objects. These stimuli gave the perception of two superimposed opaque objects moving across the visual field.


View larger version (16K):
[in this window]
[in a new window]
 
FIG. 1. Stimulus configuration. Each column shows stimulus position at 3 successive times for a single-object trial (left) and a 2-object trial (right). Filled circle, stimulus; plus sign, fixation point (FP); dashed line, receptive field (RF) of a hypothetical neuron; arrow, direction of motion of each stimulus object. At start of each trial (time 0) a stimulus appeared near the edge of receptive field in motion toward center of receptive field. At time 150 ms, all stimuli were present at center of receptive field. On 2-object trials, these 2 objects exactly superimposed to create a single object identical to other single objects for 1 frame and then moved away from each other as they continued on their trajectory. Neuronal activity was recorded and all data were analyzed from 0-300 ms, when all stimuli were again near the edge of receptive field.

Stimuli consisted of a single stimulus in each of eight directions of motion (0, 45, 90, 135, 180, 225, 270, and 315°) and all possible combinations of two different directions (36 stimuli total). All stimuli were presented on randomly interleaved trials. A complete data set had a minimum of 8 completed trials for each of the 36 stimuli, but most commonly there were 10 correct trials for each stimulus, yielding a total of 288-360 total stimuli.

Physiological procedures

Before the behavioral training was initiated, magnetic resonance imaging (MRI) scans in the sagittal and frontal planes were taken in each monkey to aid in aligning the recording cylinder and in electrode reconstruction. Under general anesthesia a scleral search coil was implanted in each eye and a head holder was attached to the skull for restraining the head, following the procedures detailed in Duffy and Wurtz (1995). After the animal was trained on a visual discrimination task that was the focus of a different study, recording cylinders were implanted over the parietal cortex directly over MT and MST in the stereotaxic vertical plane. The head holder, cylinders, and plugs for the eye coils were embedded in a dental acrylic cap that covered the top of the skull. All materials except the eye coil wire and eye coil connector were made of plastic or titanium to allow the MRI scans. All protocols were approved by the Institute Animal Care and Use Committee and complied with Public Health Service policy on the humane care and use of laboratory animals.

Horizontal and vertical eye position information was digitized at 500 Hz with a resolution of 0.1°. The experiments were controlled by a real-time experimental system (REX) (Hays et al. 1982), which was run on a PDP 11/73. Both target and eye position could be observed on-line and the digitized eye position and target position were stored for subsequent off-line analysis.

Tungsten microelectrodes were advanced into the brain toward MT and MST by using guide tubes positioned in a grid within the recording cylinder (Crist et al. 1988). Neuronal signals were amplified, filtered, and displayed on an oscilloscope and audio monitor by using conventional methods. Single neurons were isolated with a time-amplitude window discriminator (Bak). The occurrence of each action potential for each trial was time stamped at 1 kHz and stored for off-line analysis. Neurons selected for study met three criteria: 1) the activity of the neuron was altered by the presence of visual stimuli (but not necessarily moving stimuli) within the receptive field, 2) isolation was sufficient to be confident that only a single neuron was being recorded, and 3) the center of the receptive field was between 5 and 25° in eccentricity. This last criterion was necessary as these monkeys were also trained to perform a smooth pursuit eye movement and visual discrimination tasks as part of another study and were unable to do so for stimuli at eccentricities outside this range. Receptive fields were defined by using hand-manipulated spots, bars, and random dot patterns of light. Receptive field edges were defined as the locations in the visual field in which the neuron no longer responded to either stationary, flashed, or moving visual stimuli. The visual stimulation parameters were set such that the two-object stimuli intersected at the approximate geometric center of the receptive field estimated from these receptive field borders.

We identified MST and MT by their location and by the characteristics of the visual stimuli required to maximally alter their activity. MT cells responded best to small moving spots of light and had smaller receptive fields than MST cells, whereas MST cells usually responded slightly better to moving random dot patterns than to moving spots. Neurons with very large receptive fields that responded best to large random dot patterns similar to those described in the dorsal region of MST (MSTd) (Komatsu and Wurtz 1988; Tanaka and Saito 1989; Tanaka et al. 1989, 1993) were rarely encountered and the vast majority of our sample consisted of neurons characteristic of the lateral portion of MST (MSTl). However, we cannot exclude the possibility that some neurons located in the dorsal regions of MST are included in this sample.

At the end of the entire experimental series, the monkeys were deeply anesthetized with pentobarbital sodium and perfused with saline and 10% Formalin. Parasagittal sections through the region of the superior temporal sulcus were stained for cells with thionin or for fibers with a modified silver stain (Gallyas 1979). MT was identified on the posterior bank by its dense myelination. Orientation for the general region of MST was provided by the region of dense myelination on the anterior bank. Drawings of the sections showed that the guide tubes were directed toward the superior temporal sulcus and that the electrode tracks passed through or below the densely myelinated area on the anterior bank (MST) and the myelinated area on the posterior bank (MT). The categorization of each cell as falling into MT or MST however, was based on the physiological criteria described above and was consistent with the cytoarchitectonic definition of these two cortical areas.

Single neuron analysis

Peristimulus time histograms were constructed with 3-ms time bins. All data presented here represent the first 300 ms after stimulus onset for those trials in which the monkey maintained fixation throughout the stimulus presentation sequence. The spontaneous activity was defined as the activity during the 300 ms preceding the onset of the moving visual stimuli while the monkey was fixating. This activity was subtracted from the subsequent driven activity unless otherwise noted in the text. The time window used for the visual response extended from stimulus onset for 300 ms. Stimulus onset was used as time 0 so that all neurons in our sample could be compared over the same time period.

The best direction for each neuron was defined as the direction of motion of the one-object stimulus that resulted in the largest number of action potentials. The null direction was defined as the direction of motion 180° from the best direction.

The suppression index (Isupp) was computed by using the formula provided by Snowden et al. (1991): Isupp = 1 - [response to the 2-object stimulus (best and null) / response to the 1-object stimulus (best)].

The direction index (DI) was computed from the one-object stimuli by using the method of Baker et al. (1981): DI = 1 - (null direction response / best direction response).

    RESULTS
Abstract
Introduction
Methods
Results
Discussion
References

Interactions of two objects in the receptive field

A full dataset was obtained from 48 neurons in area MT and 67 neurons in area MST, in which at least eight trials of each of the 36 possible stimuli were presented while the monkey maintained fixation for the entire trial. The principal finding of this study is that two stimuli within the receptive field of a single neuron in MT and MST interact to produce a predictable increase or decrease in the response of the neuron. A typical example from an MT neuron is shown in Fig. 2. Each post stimulus time histogram (PSTH) shows the response of the neuron to stimuli moving in the direction indicated by the arrows. The PSTHs along the diagonal edge on the right represent the responses to the one-object stimuli. Inspection of these histograms shows that the addition of a second stimulus (PSTHs not on the diagonal) virtually always modulated the response of the neuron, as can be seen by comparing the responses along a single column or across a single row to the PSTH at the top of a column or the far right of a row.


View larger version (23K):
[in this window]
[in a new window]
 
FIG. 2. Poststimulus time histograms (PSTH) for all stimuli from a representative middle temporal visual area (MT) neuron. First 300 ms of response to each of 36 stimuli is shown. Columns show direction of motion of one stimulus and rows show direction of motion of the other stimulus. Panels where both arrows are in same direction (top of each column and rightmost PSTH of each row) had only 1 stimulus in receptive field. Comparison of response, either down a column or across a row, shows effect of 2nd stimulus. Vertical axis: instantaneous firing rate (each tic mark at 3 spikes/s). Horizontal axis: time (each long tic mark is 100 ms).

Figure 3A shows polar plots of the same data to allow a comparison of the responses of the neuron to different sets of stimuli. The center polar plot shows the response of the neuron to each of the eight single stimuli (direction tuning curve). This neuron had its largest response at 45°. The other eight polar plots show the response of this neuron to two stimuli in motion, with one always in the direction indicated by the arrowhead along one radial arm of the plot, and the direction of the other stimulus indicated along the other seven arms of the polar plot. These plots show that the addition of a second stimulus had two major effects. First, when one of the two objects moved in the best direction (45° in Fig. 3A, top right), the response was consistently greater than the response to motion of single objects moving in the non-best direction. As the difference between the best direction and the non-best direction increased, the difference in the response between the one- and two-object stimuli increased. Second, when one of the two stimuli moved in the null direction (225° in Fig. 3A, bottom left), the response to the two-object stimuli was consistently smaller than the one-object stimuli in these non-null directions. The same interactions occurred for the second neuron shown in Fig. 3B, but compared with the neuron in Fig. 3A the neuron in Fig. 3B showed a greater difference in the response when the reference stimulus moved in the best direction (270°, middle left) and was also somewhat greater when the reference stimulus moved in the null direction (90°, middle right) when compared with the responses to stimuli off the best-null axis. Most of the neurons showed interactions somewhere between these two examples where the response to the two-object stimulus was intermediate to the response to either of the two directions of motion presented singly.


View larger version (22K):
[in this window]
[in a new window]
 
FIG. 3. Polar plots of 2 representative neurons. Comparison of responses to 1-object and 2-object stimuli for same neuron illustrated in Fig. 2 (A) and a 2nd MT neuron (B). Polar plots A and B (middle) show response to single stimuli moving in receptive field of neuron. Each subsequent plot shows response to 2-object stimuli, one moving in direction indicated (right-arrow) on one arm of plot and one moving in direction indicated on other 7 arms of polar plot. Spontaneous activity averaged across trials was 0.4 and 1.3 spikes/300 ms for A and B, respectively. Each arm of plot has a length of 100 spikes.

Comparison of MT and MST

We saw these response modulations to two simultaneously presented stimuli throughout our sample of MT and MST neurons. Figure 4 shows examples of two single neurons from each area. In these polar plots, the dashed line shows the response to the one-object stimuli, the thin dashed circle indicates the response in the best direction, the heavy line shows the response for two-object stimuli when one object moved in the best direction, and the thin line shows the response for two-object stimuli when one object moved in the null direction. In the examples from both MT and MST, the presence of a stimulus moving in the best direction consistently resulted in a greater response compared with the response to a non-best stimulus alone, indicating that motion in the best direction increased the neuronal response relative to the object moving in a non-best direction. Similarly, the response when a second stimulus was moving in a non-best direction was nearly always less than that for the best direction alone. A similar case could usually be made for the decreased response for the two stimuli with one moving in the null direction. Both sets of plots also show that there was some variability across neurons, with some showing obvious modulation by the second stimulus, whereas others showed little or no obvious modulation (e.g., Fig. 4D).


View larger version (31K):
[in this window]
[in a new window]
 
FIG. 4. Representative polar plots of 2 neurons from MT (A and B) and 2 from medial superior temporal area (MST) (C and D). Dashed line: response to a single object moving in receptive field of neuron. Thin line: response to 2 objects in receptive field when 1 object is moving in null direction. Heavy line: response to 2 objects in receptive field when 1 object is moving in best direction. Thin dashed circle: magnitude of response to single stimulus moving in best direction (183, 177, 140, and 83 spikes/s for A-D, respectively). These 4 examples are representative of range of responses observed across our sample of MT and MST neurons. Spontaneous activity of these neurons was 1.3, 26.3, 2.3, and 9.4 spikes/s for A-D, respectively.

These results indicate that there is an effect on the responses of both MT and MST neurons when more than one stimulus was presented simultaneously in the receptive field. To determine how consistently the neuronal responses were affected by the addition of a second stimulus, we created polar plots derived from the responses pooled across all neurons. The response of each neuron was plotted so that the best direction was positioned at 0° and all responses were normalized to the response to the one-object stimulus moving in the best direction. Figure 5 plots the averaged response across the sample of recorded neurons. The dashed line shows the response when only one stimulus was presented and the dark line shows the response to two-object stimuli when one object was moving in the best direction (A and C) or the null direction (B and D). The thin dashed circle has a radius equal to the response to either the best direction alone (A and C) or the null direction alone (B and D).


View larger version (29K):
[in this window]
[in a new window]
 
FIG. 5. Polar plots representing response for 300 ms period beginning at stimulus onset averaged across all neurons recorded in area MT (A and B) and area MST (C and D). Dashed line: response to single-object stimuli. Dashed circle: maximal response to best direction alone (A and C) or null direction alone (B and D). Dark line: response to 2-object stimuli. Best direction of all neurons was normalized to 0° on plots and amplitude of responses is averaged activity for each neuron normalized by activity in best direction measured in 1-object condition. P-value from 1-tailed t-test: *P < 0.05; **P < 0.01; ***P < 0.001.

Three points are clear for two-object stimuli when one object moved in the best direction (Fig. 5, A and C): 1) there was an increased response to the two-object stimuli compared with the one-object stimulus moving in the non-best direction in both MT and MST and the effect was significant for all directions (P < 0.001, paired t-test); 2) the absolute magnitude of the difference between the one-object and two-object stimuli increased as the difference in direction between the two stimuli increased; and 3) the increased responses to the two-object stimuli were statistically significantly larger in MST than in MT (P < 0.01).

A similar pattern was seen for neurons recorded while one of the two objects in the two-object stimuli moved in the null direction (Fig. 5, B and D) but the effects were not as robust. The response to the two-object stimulus was smaller than for the one-object stimulus moving in the non-null direction. In area MT, these decreases did not reach statistical significance unless the second stimulus added to the null direction moved in the best direction. In area MST, the magnitude of the reduction in the response reached statistical significance for all two-object stimuli in which one object was moving at >= 90° from the null direction. This reduction of the response was greater in MST than in MT for each direction and across all directions pooled [analysis of variance (ANOVA), P < 0.001].

In summary, there was a clear interaction between different objects moving simultaneously through the receptive field of both MT and MST neurons compared with the one-object stimuli---a greater response when one object moved in the best direction and a smaller response when one of the two objects moved in the null direction, compared with the one-object stimuli presented off the best-null axis. The differences were greater in MST than in MT. This indicates that stimuli moving within the receptive field boundaries of these neurons can enhance or inhibit the activity to other moving stimuli, depending on the direction of motion of the two stimuli.

Comparison of response magnitudes

The magnitude of the responses to the combined stimulus motions may be related to the strength of the response of a given neuron to the best direction or to the null direction of stimulus motion. To test for this possibility for motion in the best direction, we compared the magnitude of the response to the combined stimuli with the responses to the non-best direction relative to the best direction presented alone. The scatter plots of this analysis are shown in Fig. 6. To obtain a measure of the relative strength of the best response, the response to single stimulus motion in the best direction was divided by the response to motion in each of the non-best directions (Fig. 6, abscissa). The measure of the magnitude of the increased response for the two-object stimuli was the response to two-object motion divided by the response to one-object motion in the non-best direction (Fig. 6, ordinate). If the relative magnitude of the response in the best direction is important, larger responses should be correlated with larger relative responses to the two stimuli and the points on the graphs in Fig. 6, A and B should fall near the dashed line. For both MT and MST there was a statistically significant correlation between these two measures (r = 0.958 and 0.944 for MT and MST, respectively). The slope of these regression lines were 0.723 and 0.642 for MT and MST, respectively, and both crossed the y-axis near the origin.


View larger version (30K):
[in this window]
[in a new window]
 
FIG. 6. Regression analysis of response ratio between 2-object stimulus and 1-object stimulus. A (area MT) and B (area MST) show ratio of response to a stimulus moving in best direction divided by response of stimulus moving in other, non-best direction. These plots show effectiveness of best direction to modulate response of neuron for a stimulus moving in a non-best direction, as a function of difference in response of neuron to single stimuli moving in best and non-best directions. Inset: equation for best fit regression line, r-value, and P-value.

These results indicate that the relative strength of the stimulus moving in the best direction is linearly related to the magnitude of the response when the best direction stimulus is paired with one moving in a non-best direction. This further indicates that individual MT and MST neurons weigh inputs for stimuli moving in all directions and are not selectively responding to individual components of complex moving stimuli. If the latter were true, the effect of a second object moving in the receptive field of these neurons would have a minimal effect on the response of the cell, particularly when the second object is moving at or near the null direction.

Mechanisms of two stimulus interactions

To determine what kinds of computations give rise to the responses observed in the two-object stimulus condition on the basis of the responses to each of the two stimuli presented alone, we considered four potential algorithms: summation, probability multiplication, vector addition, and averaging.

The first algorithm was to simply compare the sum of the responses to each of the two single objects in motion to the response when both objects were presented simultaneously. From a cursory inspection of the data it was clear that this algorithm would overestimate the response. For example, a simple summation would predict all responses in Fig. 5, A and C to be greater than the thin dashed circle (the response in the best direction) and all responses in Fig. 5, B and D to be greater than the thicker dashed line. Linear regression analysis was performed between two dependent variables (Mosteller et al. 1983) and the response summed for the two stimuli presented alone against the measured two-object stimulus. This analysis showed an overestimation of the response by a factor of 2.4 and 2.0 for area MT and MST, respectively (data not shown), although there was a statistically significant correlation between the predicted and measured response (r = 0.940 and r = 0.952 for MT and MST, respectively; P < 0.001 for both).

The second algorithm was a multiplicative one in which the probability of a neuronal response was calculated for each single object in 1-ms time bins and these probabilities were multiplied to predict the response when both stimuli were presented simultaneously. For this analysis the spontaneous activity was not subtracted in the calculation of the probability of the response, as the probability of a spontaneous response in 1-ms time bins was negligible even for the neurons with the highest overall spontaneous activity. We chose this method as we reasoned that if the response of a neuron was near saturation during a particular time period for both single-object stimuli, it would not be possible for the neuron to fire two action potentials within 1 ms as the summation algorithm produces. Thus, the prediction would be somewhat reduced from a simple summation algorithm and would be closer to the physiological membrane properties of these cortical neurons. For both MT and MST, there was a statistically significant, robust correlation between the predicted and observed responses. (Fig. 7, A and B; r = 0.959; P < 0.001 for both MT and MST). Although the correlation is quite robust, it is clear that this method overestimates the actual response (slope of the regression line is 1.52 and 1.41 for MT and MST, respectively).


View larger version (36K):
[in this window]
[in a new window]
 
FIG. 7. Regression analysis for spike rate predicted by using 3 different methods. All possible tests for 28 2-object stimuli (all of combinations with 2 separate moving stimuli, see Fig. 2) for all recorded neurons in a given area are shown in each plot. In each plot measured response is plotted on x-axis and predicted response is plotted on y-axis. Inset: equation for regression line, r-value, and P-value. A and B: regression analysis of measured response and predicted response on the basis of probability multiplication. C and D: regression analysis of measured response and predicted response on the basis of vector addition. E and F: regression analysis of measured response and predicted response based on averaging.

A third algorithm that seemed likely to replicate the responses of these neurons was vector addition. In this method, the response to each one-object stimulus was described as a vector with the length equal to the response of the neuron and direction equal to the direction of the stimulus. These vectors were then added and the length of the resultant vector was compared with the response measured when both stimuli were presented simultaneously. Figure 7, C and D shows the correlation between these two values for MT and MST. Again, although there was a robust correlation that was statistically significant (P < 0.001), this method consistently overestimated the response of the neuron when the two stimuli were presented together by a factor greater than two (slope = 2.54 and 2.92 for MT and MST, respectively).

The final potential algorithm we tested is that the neurons average the response of the two independent stimuli to produce the response when both are presented simultaneously. For this analysis, we summed the response from the two one-object stimuli and divided by two. We then compared this predicted value to the measured value when those two stimuli were presented simultaneously. Figure 7, E and F, shows the regression plots for all stimuli and all recorded neurons. The averaging method did accurately predict the responses that were observed in both MT and MST. The slope of the regression line for both MT and MST was very near 1.0, in contrast with the higher slopes seen with the multiplicative and vector addition regression analysis. This algorithm better predicted the responses of MT neurons than MST neurons (slope of the regression lines, 1.20 and 0.99; r = 0.940 and 0.952; P < 0.001 for MT and MST, respectively).

We further analyzed these data to determine whether the averaging method better predicted the neuronal response for neurons located in area MST as compared with area MT. Many of the responses in our sample had very low firing rates because we used all possible directions of motion, so we used an index that takes into account the overall firing of the cell: (predicted - measured) / (predicted + measured).

The results are shown in Fig. 8, where the mean and standard errors for all MT and MST neurons are shown for each two-object separation, as well as all two-object stimuli combined (Fig. 8, ALL). Statistical analysis between the two populations of neurons showed that the averaging method was consistently more accurate at predicting the response across the population of MST cells than for MT cells, although even in area MT the averaging method predicted the neuronal response very accurately.


View larger version (34K):
[in this window]
[in a new window]
 
FIG. 8. Accuracy of averaging prediction. Accuracy index for all 2-object responses of all neurons recorded in area MT (black-square) and MST (square ) was calculated and mean and standard deviation are plotted. For this index, a value of 0.01 is approximately within 1% and a value of 0.05 corresponds to ~10%. P-values for a 2-tailed t-test: *P < 0.05; **P < 0.01; ***P < 0.001. A 2nd analysis was done with the use of absolute value of accuracy (making all values positive) with same overall result.

Therefore, although all four algorithms predict responses that are well correlated with the measured response, only the averaging method accurately predicted the magnitude of the response. This prediction was more accurate across the population of neurons tested in cortical area MST than the population of neurons recorded in area MT. These data further indicate that MT and MST neurons are influenced by the direction of all motion within their receptive fields.

Comparison to measures of random dot interactions

Previous studies of MT and MST using two random dot patterns moving in the best and null directions used a suppression index to describe these interactions (Qian and Andersen 1994; Snowden et al. 1991). To make a direct comparison between these two different classes of stimuli, we computed this suppression index for the response of all neurons in the present study. This suppression index, like the commonly used direction index (Baker et al. 1981), subtracts from one the ratio of the response to the two stimuli to the response to the single stimulus (see METHODS). Thus the larger the number, the greater the suppression by the second stimulus and a negative suppression index indicates a larger response (enhancement) to the two-object stimulus. To compute both the suppression and direction indexes, stimuli moving in only the best and null directions were used. Figure 9 shows the frequency distribution of the suppression index computed for all neurons in area MT (A) and MST (B). The dashed line separates neurons having a negative suppression index (left) and a positive suppression index (right). For both MT and MST, the distribution was significantly different from a distribution with a mean of 0 (2-tailed t-test). The overall distributions showed a greater suppression index in area MST than in area MT (P < 0.05, 2-tailed t-test).


View larger version (35K):
[in this window]
[in a new window]
 
FIG. 9. Suppression index for all studied neurons. Top: frequency distribution of neurons with different values of suppression index for MT (A) and MST (B) when 2 stimuli are moving in best and null directions. Dashed vertical line: division between negative (left) and positive (right) values for suppression index. Inset: means and standard deviations. P-values taken from t-test against a population with a mean of 0 (no suppression). C and D: regression analysis between suppression index and direction index for each recorded neuron. Dashed line: difference between negative suppression index (below) and positive suppression index (above). Solid line: best fit linear regression; equation given in inset, along with r and P values.

To determine the relationship between the magnitude of the suppression index and the directional tuning, we performed a regression analysis between these two measures by using the procedures outlined in Snowden et al. (1991). Figure 9, C and D shows the scatter plots for all neurons recorded in MT and MST. In agreement with the results using random dot patterns, MT neurons did show a statistically significant (P < 0.05) positive correlation between the direction and suppression indexes, indicating that the more sharply tuned the neuron, the greater the suppression by a stimulus moving in the null direction. A similar statistically significant (P < 0.05) positive correlation was also observed in area MST neurons, which also had a greater slope of the regression line (0.295 and 0.705 for MT and MST, respectively).

These data indicate that the presence of a stimulus moving in the null direction does significantly suppress the response to a stimulus moving in the best direction across the sample of neurons tested in both MT and MST. This result is consistent with the above analysis that showed that the magnitude of the effect was related to the difference in the responses of the neuron to the two stimuli presented in isolation (Fig. 6). These results are also consistent with the results reported for random dot pattern stimuli (Qian and Andersen 1994; Snowden et al. 1991), indicating that similar stimulus interactions to these different classes of complex stimuli are present in these two cortical areas.

    DISCUSSION
Abstract
Introduction
Methods
Results
Discussion
References

The presentation of two stimulus objects moving in different directions in the receptive field of MT and MST neurons modulated the response relative to the response to either object presented alone in the vast majority of neurons studied. There was a greater response to a stimulus moving in a direction other than the best direction when it was paired with a stimulus moving in the best direction. The increase in the response was significant for all directions of motion of the non-best stimulus and the magnitude increased as the difference in the directions of the two stimuli increased. The magnitude of the increased response was directly related to the relative responses of the neurons to the best and non-best stimuli presented in isolation. There was a decreased response to a stimulus moving in the non-null direction, when it was paired with a stimulus moving in the null direction. In MT, the decreased response did not reach significance unless the second stimulus added to the null direction moved in the best direction; whereas in MST the decrease was significant once the second stimulus direction differed from the null by 90°. These experiments show that neurons in both MT and MST integrate the motion of all directions in their response to complex moving stimuli. We will discuss the differences in these responses between MT and MST, the use of the average response as the best predictor for the responses, the comparison of these results with those for global motion, and their relevance for the oculomotor system.

Comparison of responses in MT and MST

We used the same experimental paradigms and the same stimulus conditions to investigate the responses in cortical areas MT and MST, which enabled us to see several key differences in the responses in these two cortical areas. The main difference between the samples of MT and MST neurons was that MST neurons showed greater and more consistent responses. For MT neurons, only stimuli moving in the best direction were statistically significantly affected by a stimulus also moving in the null direction. In contrast, neurons in area MST were affected by all directions except for those at <90° from the null direction. Thus although there is a very small or no overt response to a stimulus moving in the null direction in area MST neurons, this stimulus can nonetheless exert a very powerful effect on the responsiveness of many cells to stimuli moving in other directions. This suggests that a difference between the response of neurons in MT and MST might be a stronger inhibitory response in MST. This strong inhibitory effect of stimuli moving in the null direction is consistent with the strong inhibitory responses recently suggested to play a role in the response of MSTd neurons to optic flow stimuli (Duffy and Wurtz 1997). The general similarity of the responses in MT and MST suggest that both MT and MST may well use a similar integrative mechanisms to create their responses to complex moving visual stimuli, but that inhibition may play a larger role in MST than in MT.

Another key difference between the MT and MST neurons was that the averaging algorithm was much more accurate in predicting the response of MST neurons than for MT neurons. This indicates that MST neurons are more strongly influenced by the motion of multiple components of a complex stimulus moving through the receptive field than are MT neurons.

Finally, the suppression index and the correlation between the suppression and direction indexes was greater for the sampled neurons in area MST than the sampled neurons in area MT as we will consider below.

Several factors in our experiments could potentially limit these results and their effects should be considered. The first question is whether the stimulus motion was centered on the receptive field of the neurons. At the start of the experiment, we determined the receptive field center and response preferences of the neuron to optimize the stimuli. Although we attempted to place the stimuli so that they were located at the geometric center of the receptive field at 150 ms after motion onset, the precise placement was likely to be slightly off-center in some neurons. Also the response profiles of MT neurons are not necessarily uniform throughout the excitatory receptive field (Raiguel et al. 1995). Neither of these factors should be critical for our results because comparisons were consistently made for the same stimuli presented either singly or in pairs. Thus if in a particular direction of motion the stimulus did not move through the peak active zone of the receptive field, that same stimulus was still the one paired with the second moving stimulus. Given the high percentage of neurons that showed similar effects when two stimuli were presented, it is likely that these responses are not affected by the precise location of the stimuli within the receptive field.

We also attempted to restrict the analysis to time periods in which the stimulus was within the receptive field of the neuron. This meant that the stimuli were presented within the receptive field at stimulus onset in the majority of neurons. However, 19 neurons in MT were too small for this to be accomplished and still had the stimulus at the center of the receptive field at 150 ms after stimulus onset. Separate analysis of these MT neurons showed no statistically significant differences in any aspect of the present report when compared with the remaining MT neurons. Given that evidence, the inclusion or exclusion of a potential onset response does not likely contribute to the overall result.

A final consideration is that the velocities of the stimuli were likely to be nonoptimal for many neurons. The activity of MT and MST neurons is known to be modulated by the velocity of the stimulus (Lagae et al. 1993, 1994; Maunsell and van Essen 1983; Tanaka et al. 1986). We only chose three different velocities based on the video display frame rate, and therefore were most likely presenting the stimuli at suboptimal velocities. Again, given that the comparisons were made between the one stimulus and the paired stimuli, it is unlikely that these differences in velocity would make a significant difference in the results.

Prediction of two-object stimulus responses

Four different algorithms were tested to determine the relationship of the two-object stimulus responses on the basis of the single-object stimulus responses: a summation, multiplication, vector addition, and the average of the responses to each of the two single-object stimuli. The averaging method was the most accurate at predicting the response to the two-object stimuli of the four tested. Simple summation, multiplication of the response probabilities at the two different directions, and a vector addition of the two single-object stimuli all overestimated the two-object stimulus response. These procedures assume that the responses to the two stimuli are independent processes and the overestimation of the actual response by this method indicates that the inputs coding different directions of stimulus motion to individual MT and MST neurons are dependent on each other. Similar types of studies, that use either random dot patterns or two stimuli moving in the best direction at different locations within the receptive field of MT neurons, demonstrated a similar averaging output (Britten and Newsome 1990; Britten 1995). A recent study of complex cells in cat primary visual cortex has shown that the responses of single neurons to two random dot patterns moving in different directions are closely predicted by the average of each individual random dot pattern moved in isolation (van Wezel et al. 1996). Similarly, the average of the preferred and anti-preferred responses of neurons recorded was in sound agreement with the averaged response to transparent motion stimuli for both the population of MT neurons studied (Qian and Anderson 1994; Fig. 11A) and the most directionally selective neurons in the primary visual cortex (V1) (Qian and Andersen 1994; Fig. 17). It is possible that at least a subset of the inputs to MT, and therefore MST, are already coded into this averaging response from the primary visual cortex. Regardless of where the averaging takes place, a test of the global nature of this algorithm would be to see how well the averaging method predicts the responses to three- or four-object stimuli and similarly how well such a method predicts the responses of complex stimuli in other cortical areas.

How exactly the cerebral cortex would produce such an averaging response is not presently clear. Several models have been proposed that incorporate some form of simple division by the neurons (e.g., Carandini and Heeger 1994; Heeger 1993; Heeger et al. 1996), which is biologically plausible based on biophysical properties of the cell membrane (Carandini and Heeger 1994). Similar kinds of averaging responses are produced for random dot patterns and sinewave grating stimuli moving in two directions (Heeger et al. 1996), although this model slightly overestimates the response for low firing rates and underestimates the response for higher firing rates relative to the averaging prediction. The difference between the predicted responses by the model of Heeger et al. and the present study may be accounted for by the difference in stimuli (single vs. multiple objects) or it may be the result of slight differences in the weighing functions for the inputs from other MT and MST neurons in the model relative to the neurons studied in this report.

Relation to global and transparent motion

The responses of MT and MST neurons to two salient, distinct objects within the receptive field were qualitatively similar to the responses observed when large field global motion stimuli are presented (Qian and Andersen 1994; Snowden et al. 1991). A direct comparison between these two classes of data showed generally sound agreement in the suppression index and the relationship between the suppression and direction indexes (Fig. 9). Our sample did, however, have a slightly lower suppression index than described previously for random dot patterns (Qian and Andersen 1994; Snowden et al. 1991). This may well be because of the larger number of broadly tuned and weakly driven neurons in our sample.

The similarity between the results with random dot patterns and the two-object stimuli of this study and previous studies in cat and monkey V1 neurons (Qian and Andersen 1994, 1995; Snowden et al. 1991; van Wezel et al. 1996) indicates that the modulation of responses for stimuli moving in non-best directions progressively increases as one moves up the cortical hierarchy from V1 to MT to MST. It is also apparent from these studies that the suppression index is correlated with the direction index, which is predicted by an averaging algorithm. Neurons with high directional selectivity would necessarily show a greater suppression of the response when the null direction stimulus is presented in addition to the best stimulus, as the responses to these two stimuli in isolation are very different. It is probably not the case, however, that the differences in the suppression by non-best direction stimuli can be entirely accounted for by an increase in direction selectivity between cortical areas. If this were the case, the correlation between directional selectivity and the suppression index would be constant across cortical areas. This was not directly observed between cortical areas MT and MST in this study (Fig. 9) and the correlation between these two measures appears even weaker (Qian and Andersen 1994) or nonexistent (Snowden et al. 1991) in cortical area VI. It is more likely that areas MT and MST are progressively refining the processing of their inputs with many different best directions and direction selectivities, through a process closely predicted by an averaging algorithm, to create the responses described in this report. As cortical area MST is largely believed to process more complex moving stimuli, particularly global motion stimuli related to self-motion (e.g., see Duffy and Wurtz 1995; Tanaka and Saito 1989), it will be interesting to test this hypothesis by determining if the averaging algorithm accurately predicts the responses of MST neurons to different classes of global stimuli.

Previous studies recording the responses of cortical neurons to moving random dot patterns have been interpreted to indicate that area MT is providing important information allowing the segregation of different objects in the perception known as transparent motion and also that area MT is the first cortical region in which suppression of local motion signals is achieved to reduce the noise of a motion stimulus (Qian and Andersen 1994). The data of this report suggest that similar mechanisms are being employed for discrete objects as well as for larger (background) objects that are moving independently.

The responses of area MT neurons to two moving gratings are also consistent with an averaging algorithm (Movshon et al. 1985; Stoner and Albright 1992). Our single-object stimuli give rise to a strong perception of two distinct objects moving in different directions, which is most similar to the noncoherent gratings that give rise to the component type of response in area MT (Stoner and Albright 1992). The averaging algorithm using single objects moving at 135° from each other would produce the bilobed tuning function, similar to that described for component neurons. As the averaging prediction was more robust for neurons in area MST, it will be interesting to determine if there is a difference between neurons in these two cortical areas when the perception of the stimuli is one of a single moving large object, for example a shift from the component type to the pattern type of responses observed to noncoherently and coherently moving grating stimuli (Stoner and Albright 1992).

Effect on oculomotor control

The results of this study, along with those using larger global motion stimuli, suggest that the responses of MT and MST neurons can best be predicted by an average of the response to the individually perceived components. Although such averaging is a potentially useful mechanism for the perception of complex moving and overlapping stimuli, the averaging would seem to provide a less reliable signal to the oculomotor system for the control of pursuit and saccadic eye movements to moving targets. If MT and MST neurons did not integrate the responses for objects moving in different directions through the receptive field, i.e., if they were purely "direction X" feature detectors, the signal to the oculomotor system would be very reliable regardless of the number of potential targets. In this case, the ability to segment larger stimulus features and to integrate appropriate local signals that exist as large objects move across each other would clearly suffer. The use of a single neuronal process for integrating these different moving stimuli should result in some sacrifice by the oculomotor system to perform an important and difficult task, the perception of relative motion of larger objects.

An alternative possibility is that the oculomotor system determines the direction and velocity of moving objects by a population response. In the case of the two-object stimulus, those neurons that have receptive field centers near the intersection points would be most affected and the response would be near the average of the response to the two individual components presented singly. However, other cortical neurons would have receptive field centers away from the intersection of the objects and would therefore have only one stimulus in the receptive field and in some cases moving in the best direction. Thus although the usually most robustly responding neurons would have an altered responses, the surrounding, usually less robustly responding neurons would be providing less ambiguous information. If a winner-take-all method is used to determine the direction of motion of the two objects, as has been suggested to account for behavioral choice between two different directions of motion of random dot stimuli (Salzman and Newsome 1994), the oculomotor processing may not suffer as much as may be predicted.

The limited evidence of the effects of two or more objects moving on the oculomotor response suggests that multiple objects do impact the latency, accuracy, and velocity of saccadic and smooth pursuit eye movements. Measurements of eye movements to two objects moving in a nonoverlapping manner when only one object is the target for a pursuit eye movement do show a significant increase in saccade latency (Ferrera and Lisberger 1995). Preliminary data from our laboratory indicates that there is a significant impact on the latency, accuracy, and initial velocity of pursuit and saccadic eye movements that use the same type of intersecting objects moving in different directions through the receptive field (Recanzone and Wurtz, unpublished observations). Such pursuit might be referred to as averaging pursuit in analogy to the averaging saccades that are made to a location between two adjacent stationary targets (Findlay 1982). A recent report also showed that microstimulation in area MT affects smooth pursuit in the macaque consistent with a vector averaging of the neuronal population response (Groh et al. 1997). These authors suggest that an averaging across the population of responding neurons drives motor behavior, such as smooth pursuit and saccadic eye movements, whereas a winner-take-all mechanism would be used for a categorical perception. The data of this report support that hypothesis.

    ACKNOWLEDGEMENTS

  The authors thank K. Powell, and D. Arends for expert animal husbandry, T. Ruffner, A. Nichols, and A. Hays for technical assistance, the NIH Laboratory of Diagnostic Radiology for MRI scans, M. Smith for histology, and K. Britten, M. Sutter, B. Olshausen, R. van Wezel, S. Elfan, H. Heuer, and K. Nace for comments on previous versions of this manuscript.

  Funding was provided by the National Eye Institute and the National Research Council, DC-02371-01A2, the Klingenstein Fund, and the Sloan Foundation (to G. H. Recanzone).

    FOOTNOTES

   Present address of W. Schwarz: Dept. of Neurology, Frauenkliniikstr. 26, University Hospital, CH-8091, Zurich, Switzerland.

  Address for reprint requests: R. H. Wurtz, Laboratory of Sensorimotor Research, Building 49, Room 2A50, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD 20892.

  Received 14 April 1997; accepted in final form 22 August 1997.

    REFERENCES
Abstract
Introduction
Methods
Results
Discussion
References

0022-3077/97 $5.00 Copyright ©1997 The American Physiological Society