Allgemeine Zoologie und Neurobiologie, Ruhr University Bochum, 44780 Bochum, Germany
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Paolini, Monica, Claudia Distler, Frank Bremmer, Markus Lappe, and Klaus-Peter Hoffmann. Responses to Continuously Changing Optic Flow in Area MST. J. Neurophysiol. 84: 730-743, 2000. We studied the temporal behavior and tuning properties of medial superior temporal (MST) neurons in response to constant flow-field stimulation and continuously changing flow-field stimulation (transitions), which were obtained by morphing one flow field into another. During transitions, the flow fields resembled the motion pattern seen by an observer during changing ego-motion. Our aim was to explore the behavior of MST cells in response to changes in the flow-field pattern and to establish whether the responses of MST cells are temporally independent or if they are affected by contextual information from preceding stimulation. We first tested whether the responses obtained during transitions were linear with respect to the two stimuli defining the transition. In over half of the transitions, the cell response was nonlinear: the response during the transition could not be predicted by the linear interpolation between the stimulus before and after the transition. Nonlinearities in the responses could arise from a dependence on temporal context or from nonlinearities in the tuning to flow-field patterns. To distinguish between these two hypotheses, we fit the responses during transitions and during continuous stimuli to the predictions of a temporally independent model (temporal-independence test) and we compared the responses during transitions to the responses elicited by inverse transitions (temporal-symmetry test). The effect of temporal context was significant in only 7.2% and 5.5% of cells in the temporal-independence test and in the temporal-symmetry test, respectively. Most of the nonlinearities in the cell responses could be accounted for by nonlinearities in the tuning to flow-field stimuli (i.e., the responses to a restricted set of flow fields did not predict the responses to other flow fields). Tuning nonlinearities indicate that a complete characterization of the tuning properties of MST neurons cannot be obtained by testing only a small number of flow fields. Because the cells' responses do not depend on temporal context, continuously changing stimulation can be used to characterize the receptive field properties of cells more efficiently than constant stimulation. Temporal independence in the responses to transitions indicates that MST cells do not code for second-order temporal properties of flow-field stimuli, i.e., for changes in the flow field through time that can be construed as paths through the environment. Information about ego-motion three-dimensional paths through the environment may either be processed at the population level in MST or in other cortical areas.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The vigorous response to
flow-field patterns has long been considered evidence that area medial
superior temporal (MST) is involved in processing the complex motion
patterns that are generated during self-motion in the environment
(Britten and van Wezel 1998; Duffy and Wurtz
1991a
,b
; Lagae et al. 1994
; Lappe et al.
1996
; Saito et al. 1986
; Tanaka and Saito
1989
; Tanaka et al. 1986
, 1989
). Because such
motion patterns often change through time as the direction and speed of
self-motion vary, the ability to quickly and smoothly adapt to change
is essential to obtain a stable self-motion percept and to generate
appropriate motor commands. Changes in the pattern of optic flow occur,
for instance, because of eye movements (Erikson and Thier
1994
; Kawano et al. 1994
; Lappe et al.
1998
; Shenoy et al. 1999
; Thier and
Erikson 1992
). We studied the temporal properties of MST cells
during both constant and continuously changing flow-field stimulation
to see how MST cells react to changes in the flow-field pattern.
Previous work in area MST and other visual areas responsive to
flow-field stimuli focused on the response of cells to a constant flow-field pattern that was preceded and followed by a dark screen, a
stationary random dot pattern, or by a set of random dots moving noncoherently (Duffy and Wurtz 1991a,b
; Saito et
al. 1986
; Tanaka and Saito 1989
; Tanaka
et al. 1986
, 1989
). Experiments following this paradigm have
been fundamental in establishing the tuning properties of MST cells,
such as the width of the tuning curve or the preference of some cells
for complex patterns, such as spirals (Graziano et al.
1994
) and combinations of flow-field components (Lagae
et al. 1994
; Orban et al. 1992
). The temporal profile of the response of MST cells during the presentation of constant flow fields was analyzed by Duffy and Wurtz (1991a
,
1997
), who found that the response throughout the stimulus
presentation was uniform, even for long periods of time (up to 15 s), with the exception of an initial peak, which was often
nonselective. Over longer time periods, the cell activity could be
affected by temporal context information. For example, they showed that the response to the same stimulus could be enhanced or reduced if the
frequency of presentation of the stimulus within a stimulus set was changed.
We devoted our attention to the changes in the response patterns caused by continuous changes in the stimulus or, more generally, to the dependence of cell responses on temporal information over a shorter time scale. We analyzed temporal changes in cell responses during constant stimulation and continuously changing flow fields (transitions). In constant flow fields, the same flow-field pattern was shown for the entire stimulus duration. In transitions, we changed the flow-field pattern continuously by morphing one flow field into another. The flow-field parameters of transitions changed in every frame; the percept of a smooth, gradually changing motion was preserved, however. By comparing the flow-field tuning and the temporal structure of the responses during constant stimulation and transitions, we were able to establish how MST cells respond to changes in optic flow parameters and, more specifically, to determine whether cell responses are affected by the temporal context provided by preceding stimulation.
We tested the responses of cells for dependence on temporal context and linearity. A temporally independent response would suggest that a cell does not directly encode temporal changes: the cell simply responds to the stimulus being presented without being affected by contextual information. The presence of strong temporal effects in the cell responses would raise the possibility that MST cells code for second-order properties of the stimulus, which in the case of flow-field patterns could give information about specific paths in the three-dimensional (3D) environment. We defined a response during a transition as linear if it could be predicted by the responses of the two stimuli that defined the transition. A linear response implies temporal independence and linear tuning. A nonlinear response may be caused by nonlinearities in the tuning curve (with respect to the flow fields that we used to define the coordinates of the stimulus space), by dependence on temporal context, or both.
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We recorded from area MST of two awake and behaving adult male monkeys (Macaca mulatta) engaged in a central fixation task. All procedures were in accordance with published guidelines on the use of animals in research (European Community Council Directive 86/609/ECC).
Animal preparation
To obtain detailed information about the location of cortical
areas of interest, we initially used a 1.5 Tesla Brucker scanner to
obtain magnetic resonance (MR) scans of the heads of the
animals. In the first animal, we obtained horizontal slices 1.1 mm
thick with a voxel size of 0.78 mm. In the second animal, slices of 1 mm thickness were initially collected sagitally, with voxels of the
same size. We used Scion Image (Beta 2.0, Scion Corporation) and Image
(National Institutes of Health) to reslice the MR raw images in
the other planes (coronal, sagittal, and axial) to obtain a complete
brain atlas for each animal. Prior to training and recording, a
head-holding device was secured to the skull with screws. Two scleral
search coils (Judge et al. 1980) connected to a plug on
the skull were implanted to monitor eye position. A recording chamber
with a diameter of 22 mm was placed over the occipital cortex in a
parasagittal stereotaxic plane tilted back 60° from the vertical. To
further increase the stability of the implant, a dental acrylic skull
cap was placed around the head-holding device, scleral search-coils
plug, and chamber. During surgical procedures, animals were initially
treated with atropine and sedated with ketamine hydrochloride (10 mg/kg) and were later maintained under general anesthesia with
pentobarbital sodium (Nembutal; initially 10 mg/kg i.v., as needed
later). Heart rate, blood pressure, and O2
concentration in the blood (SPO2) were monitored
throughout surgery. Training sessions started no sooner than one week
after surgery.
Training and recording sessions
During training and recording sessions, a monkey, while performing fixation tasks for liquid reward (water), sat in a primate chair with its head fixed at a distance of 48 cm from a screen. Water intake between training or recording sessions was controlled, but the animals had unrestricted water access at least one day per week. The training phase lasted until the animal could reliably fixate a 0.8° red light-emitting diode (LED) fixation dot for 3 s within a 2.5° window. Training and recording sessions lasted 3-6 h and included frequent breaks. Tungsten-in-glass electrodes with an impedance of 1-2 MW at 1 kHz were advanced transdurally through guide tubes with a hydraulic microdrive (Narishige) that was mounted on the recording chamber. Electrode penetrations were aligned with the chamber orientation, i.e., parallel to the 60° parasagittal plane. Electrode depth and the position and extent of receptive fields were noted to establish the relative position of landmarks such as gray and white matter and of surrounding cortical areas; these landmarks were then compared with the MR brain atlas figures. During recordings, area MST and surrounding cortical areas were provisionally identified by the receptive field extent and tuning. Background illumination was switched on between recordings to avoid dark adaptation. The background light was switched off during recordings. The monkey, separated from the experimental apparatus by thick black curtains, was in darkness except for the fixation target and the stimulus.
In-house software controlled the presentation of the fixation point, reward delivery, and the collection of eye position and cortical activity data. Eye position was monitored every 8 ms. Cortical activity was sampled at a frequency of 1,000 Hz and single-cell activity was isolated on-line (MSD 3.19, Alpha Omega). Once a cell was successfully isolated and its receptive field was hand-mapped, several stimulus sequences (tests) were presented. During each test, stimuli were presented continuously for 5-25 min, depending on the test type, while the animal alternated fixation and rest at 2-3 s intervals. Only the cortical activity collected while the animal fixated the central point was subsequently analyzed.
Visual stimulation and data analysis
Visual stimuli were generated by a Silicon Graphics High Impact Indigo 2 computer (Iris 6.2, MIPS R4400, 150 MHz) using software specifically designed for this set of experiments with OpenGL (SGI Performer 2.1). Because the timing of the stimulus presentation could not be reliably retrieved from the computer at the resolution needed for this experiment (1 ms), the timing of the stimuli was controlled by a diode secured on the border of the screen in front of the animal. At each stimulus onset, a white square was displayed for the duration of a frame (1/60 s) and the time of the onset of the white square was stored by the data acquisition system. The white square and the diode were always outside the field of view of the animal. The sequence of stimuli presented and the estimated time of the presentation were stored on the computer. At the end of the recording session, information about the presented stimuli and the collected data (both spikes and stimulus onset times) were merged.
A simulated camera moved within a 3D environment of randomly placed white dots at a speed of 30 deg/s or less. Stimulus sequences were built as paths within this 3D environment in such a way that any sudden or gradual change in the direction of motion was possible and that the random dots changed direction as necessary. In no case was a new set of random dots abruptly introduced during an ongoing stimulus sequence. To achieve this, a cube of random dots was built using 27 smaller cubes, with identical dimensions and identical dot distribution, arranged in a 3 × 3 × 3 grid. Initially, the simulated camera, the position of which determined the field of view within the 3D environment, was placed at the center of the cube, was assigned a depth of visual field of 30 meters, and could be moved in any direction or rotated around any axis. When the camera approached a predefined border within the cube (which depended on the depth of the visual field) beyond which far dots could no longer be seen, it was moved back to the central cube in the same relative position (within the smaller cube) it previously occupied in the outer cube. This allowed for continuous smooth presentation for any desired length of time while restricting the amount of computational resources necessary to compute the position of single dots in the simulated-space model.
Motion in the simulated 3D environment was defined in a six-dimensional (6D) space, composed of two sets of three orthogonal axes, representing translations (left/right, up/down, and forward/backward [expansion/contraction]) and rotations (left/right, up/down, clockwise/anticlockwise rotation, or yaw, pitch, and roll, respectively). The sign of a component in each dimension determined the direction of motion. Each possible motion in a 3D space can be obtained as a linear combination of the six dimensions used. Based on this coordinate system, flow-field stimuli can be grouped into three categories:
1) Cardinal flow fields. Thirteen flow fields, defined as a positive or negative component of only one of the six dimensions, plus a stationary random dot pattern as a control. The set consisted of the flow fields listed in the previous paragraph.
2) Linear-combination flow fields. Flow fields defined as a linear combination (vector sum) of two or more cardinal flow fields. Because the points are fixed and the simulated camera moves according to the specified parameters, at all times the resulting flow field is a linear combination of the defining flow fields. Linear-combination flow fields include all possible flow fields other than cardinal flow fields. They include, but are not restricted to, spirals and shearing patterns. Translations at 45° with respect to the cardinal flow fields are also considered linear-combination flow fields.
3) Transitions. During continuously changing stimulation, one of the 13 cardinal flow fields was morphed into another one by linearly decreasing the contribution of the first flow field and linearly increasing the contribution of the second flow field. They are a special kind of linear-combination flow field, one in which the flow field pattern changes in every frame. The resulting flow is a smoothly changing, often complex flow that can be perceived as a path in the 3D environment. Two cardinal flow fields define a transition; we call them the from-stimulus and the to-stimulus.
By flow field we refer to both a single instance of a moving pattern (defined in terms of a set of parameters and initial dot positions) and a type of moving pattern (defined only by the 6D motion parameters). We used various combinations of cardinal and linear-combination flow fields and transitions to define several tests, each of which addressed a different aspect of temporal processing in area MST. A detailed description of the combinations used is given in RESULTS. In all cases, cardinal and linear-combination flow fields and transitions were presented for 50 frames (833 ms). Within each test, each stimulus was presented 20-40 times. Tests that included transitions always alternated the presentation of a cardinal flow field with the presentation of an associated transition, giving the impression of a smooth extended movement. Not all transitions and linear-combination flow stimuli could be tested on each cell. While recording from a cell, the selection of stimuli was initially random and later was determined on the basis of an on-line analysis of the collected data. All cardinal flow stimuli were tested on each cell.
In addition to the tests reported in RESULTS, we performed
two control tests to map the receptive fields of the cells and to
obtain their speed profile. These data were used in establishing the
basic properties of the cells studied and as an aid to reconstruction. The results obtained with these tests do not directly address the
issues covered in the present study and are in agreement with previously published reports (Duffy and Wurtz 1991a,b
;
Lagae et al. 1994
; Saito et al. 1986
;
Tanaka and Saito 1989
; Tanaka et al.
1996
). We mapped the receptive field using small
patches (each covering 1/25 of the area covered by the flow field
stimuli) of random dots that translated in eight directions in the
frontoparallel plane and that presented at 25 locations within a 5 × 5 grid. We were able to determine the extent of the receptive field
and local direction selectivity at the 25 locations tested.
The speed tuning of the cell to the two cardinal flow-field stimuli that defined the transition with the highest firing rate was obtained by presenting the two flow fields at nine speeds each (4 lower and 4 higher than the speed used in the other tests, at intervals of 6 deg/sec). We used this test to verify that the change in the response we obtained during the transitions could not be accounted for as an effect of speed tuning. In transitions to or from a stationary pattern, the response during the transition was consistent with the speed tuning curve whenever the speed tuning was tested.
Typically, we recorded the activity of each cell for as long as the recording was judged to be stable, in most cases for more than one test and in the majority of cases for 3-5 tests. Stimuli always covered the entire 90° central portion of the visual field and were backprojected from a projector onto a clear acrylic screen with a thickness of 0.635 cm (Draper Cineplex, Cine 10 coating, with a peak gain of 0.1 and a half-gain angle of 44%, which distributed the light from the projector uniformly across the screen surface). Stimuli were presented at a constant frame rate of 60 Hz.
Data were analyzed using software written in Perl and C to perform standard statistical tests (t-test, chi-square). We excluded from analysis the data collected during tests in which the mean firing rate across all the conditions was below 1 spike/s. Among the sample of 217 MST cells, 13 (6%) were rejected. The latency used in further analysis of the data was computed initially for each stimulus by identifying the first three consecutive 40-ms bins that had a firing rate higher than the average firing rate for the test. The latency for the stimulus was defined as the middle of the first bin, as long as it was no greater than 150 ms. The latency for each test was calculated as the average latency of the single stimuli included in the test.
Histology and reconstruction
During the final days of recording from the first monkey,
electrolytic microlesions (10 µA positive and negative for 10 s) were made at several locations within the estimated location of area
MST. After recording was completed the monkey was given an overdose of
pentobarbital sodium and, after respiratory block and cessation of all
reflexes, was transcardiacally perfused. Sagittal brain sections were
cut at 50 µm thickness and alternately stained with cresyl violet,
neutral red, or Klüver-Barrera for cytoarchitecture, and with the
Gallyas method for myeloarchitecture (Gallyas 1979;
Hess and Merker 1983
). Electrode tracks were identified on the basis of the location of the penetrations relative to the entire
recorded area, the spatial relationship to other tracks and marking
lesions, and the depth profile during a penetration. The approximate
location of each recording site on the track was determined based on
the distance from the above-specified landmarks as well as the location
of gray matter. Recording sites and myelin borders were reconstructed
on two-dimensional maps of the recorded hemisphere (Ungerleider
and Desimone 1986
; Van Essen and Maunsell 1980
).
Based on the reconstructions, we decided that 135 of the 170 cells
recorded from the first monkey were located within the MST boundaries.
The second monkey is involved in further experiments and histological
analysis is not yet available.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We recorded from 217 cells in area MST in two hemispheres of two macaques (170 and 47 cells, respectively). Temporal sequences of stimuli and Temporal context compare the responses obtained during continuous versus noncontinuous stimulus presentation and randomized versus nonrandomized stimulus sequences (movies). The remaining sections use tests that involve both transitions and constant stimulation to address the issue of linearity and temporal independence.
Temporal sequences of stimuli
Our tests included continuous stimulation and stimulation interleaved with interstimulus intervals (ISI). Before proceeding with the analysis of linearity and temporal independence, we wanted to verify that the overall tuning and response properties of MST cells remained unaffected by the regimen of stimulus presentation used. We tested the tuning to flow fields and the temporal profile of the response in three tests, which included the same set of stimuli (cardinal flow fields) embedded in a different temporal context:
1) Flow field tuning test with ISI. Stimuli were interleaved with an interstimulus interval of 833 ms during which the screen was left in darkness (with the possible exception of the fixation point).
2) Flow field tuning test without ISI. Stimuli were presented in continuous succession, with abrupt changes in flow-field pattern but without discontinuities in the random-dot pattern (at the stimulus onset, single dots changed direction but were not replaced).
3) Transition-tuning test. Stimuli alternated with transitions, during which the dots gradually changed their direction from that of from-stimulus to that of to-stimulus. In this initial analysis, the response during the transitions was ignored.
By directly comparing the responses in these three categories for those cells in which each of these three stimulus sets were tested (n = 61), we asked whether the tuning properties and temporal profile of the responses were affected by the type of stimulus sequence. We found no evidence of systematic changes in tuning among the three tests, but there were slight changes in the temporal structure (Fig. 1).
|
The average firing rate was similar across conditions: 9.9 spikes/s (standard deviation 7.1) for stimuli with ISI, 9.4 spikes/s (standard deviation 7.8) for stimuli without ISI, and 10.2 spikes/s (standard deviation 7.5) for stimuli alternated with transitions (during transitions the firing rate was 10.4 spikes/s, standard deviation 7.7). Within single cells, the average absolute difference in firing rate was 2.8 spikes/s (standard deviation 3.3) between stimuli with and without ISI, 2.1 spikes/s (standard deviation 2.3) between stimuli without ISI and stimuli alternated with transitions, and 2.2 spikes/s (standard deviation 2.1) between stimuli with ISI and stimuli alternated with transitions. The average absolute difference in firing rate between stimuli and transitions within the same stimulus set was 0.7 spikes/s (standard deviation 0.8).
The tuning properties of the cells obtained in the three tests were similar. We classified stimuli as either highly effective (the average firing rate for the stimulus was significantly higher than the average firing rate for the test plus a standard deviation), non-effective (the stimulus firing rate was significantly smaller than the average firing rate minus a standard deviation), or moderately effective. Correspondingly, we refer to cells as highly responsive, nonresponsive, or moderately responsive. We decided to use this classification, rather than the more common distinction between selective and nonselective responses, to allow for a finer analysis of the data. Because the responses to different stimuli were compared with each other and with the average firing rate across conditions (and not with a reference firing rate such as the baseline), we do not imply that cells were inhibited during the presentation of non-effective stimuli. Non-effective stimuli were simply defined as eliciting a significantly lower response than other stimuli in the same test.
Among the 61 cells tested with the three stimulus sets, 59 (96.7%) were highly responsive to at least one flow type in at least one test. The percentage of cells responsive to at least one flow field and the number of effective flow fields was similar across tests. When tested with stimuli with ISI, 52 cells (85.2%) were highly responsive to at least one flow field and, among those, the average number of highly effective stimuli was 1.9 (standard deviation 1.1) of 13 tested stimuli. Similarly, 51 cells (83.6%) tested with stimuli without ISI were responsive to at least one stimulus and to an average of 2.0 stimuli (standard deviation 1.2). In tests with transitions, 56 cells (91.8%) were responsive to at least one stimulus, with an average of 2.4 effective stimuli per set (standard deviation 1.0). All cells were moderately responsive to at least one stimulus in all of the tests. The number of moderately effective flow fields was 8.3 (standard deviation 1.1) in stimuli with ISI, 8.6 (standard deviation 1.2) in stimuli without ISI, and 7.4 (standard deviation 1.0) in stimuli alternated with transitions.
The tuning of MST cells to flow fields was also very similar in the three tests. We counted the number of stimuli in each cell that were assigned to the same category (highly effective, moderately effective, or noneffective). If cells responded randomly, on average 3.9 stimuli would belong to the same category. The average number of stimuli in the same category was 10.0 of 13 (standard deviation 2.3) when comparing stimuli with and without ISI, 9.4 (standard deviation 2.7) when comparing stimuli with ISI with stimuli with transitions, and 9.7 (standard deviation 2.3) when comparing stimuli without ISI with stimuli with transitions.
The temporal profile of the response was not the same in the three
conditions, however. As previously noted (Duffy and Wurtz 1997), MST cells maintain a constant firing rate throughout a constant stimulus presentation. At the beginning of stimulus
presentation an often nonspecific peak in the response has been
reported (Duffy and Wurtz 1997
).
To establish if such nonspecific peaks were equally present in the three tests, we counted the number of peaks at the beginning of a stimulus presentation, which indicate the presence of a large transient response to the stimulus onset. We defined a peak as a 40-ms bin in which the firing rate was larger than the average firing rate plus two standard deviations and which occurred before the latency plus 30 ms (stimuli without transitions) or in the initial 100 ms (stimuli alternated with transitions).
The largest number of peaks was found in stimuli with ISI (mean 6.7, standard deviation 3.6), followed by stimuli without ISI (mean 4.0, standard deviation 2.1). Fewer peaks were present in stimuli alternated with transitions (mean 1.9, standard deviation 1.3) as there was no abrupt change in the flow-field pattern.
The higher number of peaks in stimuli with ISI demonstrates that the
occurrence of stimulus-nonspecific initial peaks is more common in
noncontinuous stimulation. In 28 cells (45.9%) a peak was present in
over half of the conditions when the cells were tested with stimuli
with ISI, whereas that was the case in only eight cells (43.1%) in
stimuli without ISI and in none of the cells tested with stimuli
alternated with transitions (note that the average number of highly
effective flow fields in which a peak was expected was 1.9-2.4, see
above). Our data confirm the results from Duffy and
Wurtz (1997) and show that this motion-onset response component
can be eliminated by presenting stimuli in a continuous fashion, with
or without abrupt changes in the flow-field pattern.
Temporal context
Because the experiments we conducted often relied on a continuous stimulus presentation paradigm that has not been widely used in area MST, we verified that the effect from preceding stimulation in a sequence of continuously presented stimuli was negligible (i.e., the response to a stimulus did not depend on the sequence of stimuli previously presented). This is a condition that has to be met in order to examine the response to each stimulus in isolation.
In the transition-tuning test, we selected randomly the initial sequence of stimuli, which included two presentations of each of the 13 cardinal flow fields, and 27 transitions. We then repeatedly presented this sequence (which in the rest of the present paper we refer to as a movie) to record the response of the cell to 40 repetitions of each stimulus and 20 repetitions of each transition (Fig. 2 shows an example of a movie sequence). We also used this type of movie sequence in 20 flow-field tuning tests with ISI and in 21 flow-field tuning tests without ISI. This regimen gave us the opportunity to compare the response with each stimulus in two temporal contexts. Within the movie, each stimulus was presented twice, preceded by a different cardinal flow field or transition. If there were an effect of temporal context (stimulus sequence) we would expect the firing rate in the two sets of repetitions of the same stimulus to be different.
|
We first established an acceptable reference measure of trial-to-trial variability of firing patterns by analyzing the responses obtained with completely randomized stimulus sequences. If the cells are not affected by temporal information, we would expect that the trial-to-trial variability would match the variability in firing rate between the two sets of responses to the same stimulus in a different temporal context. The randomized tests only included cardinal flow fields and in some cases stimuli were alternated with an ISI (flow-field tuning test). For each stimulus, we randomly split the response into two groups and tested whether the underlying temporal distribution of the response was the same using the chi-square test. We found that in only 2.4% and 0.3% of stimuli with and without ISI, respectively (n = 1,547 stimuli with ISI; n = 1,911 stimuli without ISI), the response was statistically different.
In the test with fixed stimulus sequences (movies) we obtained generally similar results. In only 2.0% of stimuli alternated with transitions (n = 2,262, 174 cells) and in 6.0% of stimuli without transitions (either with or without ISI, n = 533, 41 cells), the temporal pattern of the response was different (chi-square test) in the two presentations of the same stimulus embedded in a different temporal context. Because the preceding transition may affect the initial firing rate during the stimulus presentation, we repeated the test while excluding the first 80 ms of the response and obtained comparable results.
The comparison between the movie and the randomized presentation sequence shows that the temporal context within the stimulus sequence does not significantly alter the response properties of the cell. During both continuous and noncontinuous stimulation, the response to each stimulus can therefore be analyzed independently from the stimuli preceding it.
Responses during transitions
Having established that the temporal distribution of the responses during continuous stimulation and, in particular, during transitions was not significantly affected by preceding stimulation and that the response properties during transitions and constant stimulation did not substantially change, we turned to the analysis of the responses during transitions (Fig. 2). In this section we look at the responses during individual transitions and at the single-cell level.
Among the 176 cells tested with the transition-tuning test, we recorded the responses to 4,752 transitions. Of those, cells were highly responsive to 603 transitions (12.7%) and moderately responsive to 2,963 transitions (62.3%). As a first step, we tested whether each transition was linear with respect to the preceding from-stimulus and to the following to-stimulus. A transition was defined as being linear if a linear combination of the response obtained during the presentation of the from-stimulus and the to-stimulus could predict the response during the transition. If a transition is linear, it is temporally independent and linear with respect to the stimulus space. If a transition fails the test, the nonlinearity present in the response may be due to dependence on temporal context, tuning nonlinearities, or both.
We conducted the following analyses separately for two groups of transitions, a first group including only highly effective transitions (n = 603) and a second group including highly and moderately effective transitions (n = 3,655). Transitions in the first group have a higher firing rate; transitions in the second group offer a wider view of the responses to transitions. Transitions with a firing rate below average were excluded because in most cases the firing rate was so low that the best fit would have been a horizontal line near the x-axis, which would have artificially increased the number of linear responses.
For each transition, we built a linear model based on the response to the from-stimulus and the to-stimulus that reflected the linear decrease of the from-stimulus component and the linear increase of the to-stimulus component. Essentially, the model was a line from the average firing rate in the from-stimulus to the average firing rate in the to-stimulus (Fig. 4A). The chi-square test was then used to evaluate the goodness-of-fit between the expected model and the data. Among highly effective transitions, a linear response was found in 135 transitions (22.4%) whereas among the second group (highly and moderately effective transitions), 1,596 transitions (44.8%) were linear.
Linear transitions could be either equally distributed among cells or concentrated in a subset of cells. To address this issue, we looked at the responses to all recorded transitions for each cell. In most cells the majority of responses to transitions were nonlinear (Fig. 3). In the first group, 42 cells (25.1%) had over 50% of the linear transitions; in the second group, linear transitions were dominant in 75 cells (42.6%). The higher percentage of linear transitions among highly effective transitions suggests that there is a relationship between firing rate and linearity. The following tests examined the origin of this relationship.
|
The results show that nonlinear transitions were not clustered in a subset of cells but rather that they occurred in almost all cells. Before the issue of the source of nonlinear responses is addressed, the temporal structure of the responses is described in more detail, first among the entire set of transitions and then among subsets of transitions.
Transition models
To characterize the temporal structure of nonlinear transitions, we fit nonlinear transitions against three models that reflect the profiles of responses encountered in previous cell recordings:
1)
Straight-line. A set of lines at different orientations
(from 45° to 45°, tested at 1° intervals, centered at the
middle of the of the transition) such that the area below the line was the same as the area covered by the response profile (Fig.
4B).
2)
Peak. A set of Gaussians with varying spread (standard
deviation ) centered on the bin with the highest firing rate (Fig. 4D). Standard deviation ranged from 0.1 to 10 and was tested
at 0.1 intervals.
|
3
Step. The largest difference in firing rate between two consecutive bins was first identified. Two straight lines were then drawn: the first from the first bin to the discontinuity at the height of the average firing rate before the discontinuity; the second from the discontinuity to the last bin at the height of the average firing rate after the discontinuity. This model was tested only when the difference in firing rate between the average firing rate before and after the discontinuity was larger than half the average firing rate for the transition (Fig. 4C).
As in Responses during transitions, only highly
effective and moderately effective transitions were included in the
analysis. Prior to fitting, the response at each 40-ms bin
(rb) was smoothed over the four
adjacent bins to reduce the effect of noise in the response
![]() |
|
Among highly effective transitions, the straight-line model was the most successful at describing the responses of MST cells (42.6% of all transitions). The peak model accounted for 21.4% of fits and the step model for 13.6%. Among the second group, the straight-line model was also the most successful (26.1%) at describing nonlinear transitions. The combined peak model (13.2%) and step model (15.9%) fit slightly less than one-third of the transitions (Fig. 4, C and D).
We found no evidence that a given model was preferentially found in a subset of cells. Instead, transition models seemed to be randomly distributed among cells (Fig. 6). In the first group, straight-line model fits accounted for more than 50% of transitions in almost half of the cells (46.1%, n = 167), as was expected from the high frequency of straight-line fits. The peak and step models fit more than 50% of the transitions in only 19.2% and 12.6% of cells, respectively. In the second group, the percentage of cells with straight-line model fits in more than 50% of transitions was lower than in the first group (8.5%, n = 176) because of the lower percentage of straight-line fits. Even fewer cells had a majority of peak model fits (4.0%). In no cell did step models account for more than 50% of transitions.
|
At the cell level, both the percentage of linear transitions and the distribution of transition models were randomly distributed, indicating that there were no subgroups of cells with specific response profiles. It may therefore be possible that different transition types could explain the differences in the distribution of transition models. A transition model, for instance, may be associated with a specific transition pattern.
Transition types
We have so far analyzed all the transitions together. Among transitions, however, the type of flow pattern may vary greatly. In particular, there are several groups of transitions that overtly share some features. Some transitions, for instance, are characterized by a stationary pattern in the middle, others by acceleration of a cardinal flow field. We asked whether the same percentage of linear responses and the same proportion of model fits were present in these groups of transitions and in the entire transitions set. We first tested the responses within each group for linearity (Fig. 7) and then fit the nonlinear responses with the three models described above (Fig. 8).
|
|
Some transitions (accelerations: n = 24 of 169 possible transitions) consisted of an acceleration of a cardinal flow field: the from-stimulus or the to-stimulus was a stationary flow field (Fig. 4H). A larger percentage of straight-line fits and a smaller percentage of peak fits than in the overall cell population were found in both highly effective (group I) and effective (group II) transitions. The gradual increase in the firing rate may be interpreted as reflecting a speed-tuning curve.
In stationary and near-stationary transitions (n = 12 + 8) the flow was stationary or almost stationary in the middle of the transition. In stationary transitions, the two flow fields were opposite (for instance, upward and downward translations, or contraction and expansion). Near-stationary transitions involved one rotation and one translation, moving in opposite directions, along the corresponding dimension in the frontoparallel plane. A lower-than-average incidence of linear transitions was found in the stationary and near-stationary transitions; in those transitions the sharp change in the flow field (a full reversal of the flow field in the stationary case) was often reflected in sudden changes in the firing rate. Consequently, the models assigned to stationary transitions were more frequently step (Fig. 4F) or peak (Fig. 4G) than in the whole cell population. The step model typically occurred when the cell strongly responded to only one of the stimuli defining the transition. For instance, a cell responsive to expansion but unresponsive to contraction would have a clear step-like onset of response when the expansion followed a contraction in the transition and a clear fall in firing rate when the expansion ended, followed by a contraction. The peak fits were present in similar cases, but in cells with a stronger sensitivity to low speeds.
In direction, rotation, or near-direction transitions (n = 8 + 8 + 8) the flow gradually changed direction of translation (direction type) or rotation (rotation type) in the frontoparallel plane (Fig. 4E). The two flow fields were orthogonal (e.g., upward translation and leftward translation). Near-direction transitions involved a rotation and a translation, for example, leftward translation to downward rotation (n = 8 + 8). In direction transitions, the higher percentage of linear responses may be due to lack of responsiveness to any intermediate direction of flow. Direction and rotation transitions largely agreed with the average results for the entire cell population, probably because the type of motion change that characterized such transitions was not too unlike the other transitions.
Our data show that the transition type has a strong effect on the percentage of linear transitions and on the ratio of models fit to nonlinear transitions. Such effect suggests that some of the nonlinearities in the responses may be caused by the tuning properties (linear or nonlinear depending on the transition type) of the cells.
Sources of nonlinearities during transitions
Nonlinearities in the responses to transitions were equally spread among cells and, albeit in different degrees, across different transition types. In the next three tests we tried to identify the source of nonlinearities in responses to transitions. By looking at a single nonlinear response during a transition, it is impossible to decide if the nonlinearity is caused by the effect of temporal context, by the cell tuning for the linear-combination flow fields, or by both. To decide among these three possibilities, we tested for temporal independence.
A cell response is temporally independent if the response to any flow-field pattern is not affected by the preceding flow-field pattern. MST cells have been shown to be mostly linear, after the initial stimulus onset, during constant stimulation with stimuli alternated with ISI, because their firing rate remains largely unaffected throughout the stimulus presentation. Examples of dependence on temporal context include the peaks at the stimulus onset, obtained here with stimuli alternated with ISI; the firing rate at the stimulus onset was higher than later in the presentation, although the stimulus did not change. We showed that during continuous stimulus presentation (with abrupt flow pattern changes or with transitions) the initial motion-onset peak is eliminated and the firing rate throughout a constant flow field presentation is uniform.
A response dependent on temporal context would indicate that cells are sensitive to temporal changes in the stimulus or to its second-order properties. If instead those cells were temporally independent, a nonlinear response during a transition might be caused by nonlinearities in the tuning profile of the cells. For instance, a peak in the response to a transition could indicate that a cell is responsive to the flow-field pattern presented at the time of the peak minus the cell latency.
To test whether cells are temporally independent, we devised three experiments. In the temporal-independence test, we presented the flow patterns that occurred during the transition as constant flow fields for the standard stimulus duration of 50 frames (833 ms) and then compared the firing rates thus obtained to the firing rate during the transition. In the temporal-symmetry test, we compared the temporal distribution of the response to a transition and the corresponding reversed transition. Finally, in the heading test, we examined in more detail the responses of MST cells within a restricted subregion of the stimulus space, so that the changes in the responses to similar transitions could be analyzed.
TEMPORAL-INDEPENDENCE TEST.
We tested 51 cells (153 transitions) with both the
temporal-independence test and the transition-tuning test (Fig.
9). We selected the three transitions
associated with the highest firing rates in the transition-tuning test.
For each transition tested, five flow patterns were identified: the
from-stimulus, the flow fields presented at one-fourth, one-half, and
three-fourths of the transition, and the to-stimulus (at 0, 208, 416, 634, and 833 ms, respectively). The 25 stimuli were then presented in
random order, as constant flow field patterns, for 20-40 repetitions each. To compare the data obtained from a transition and the
corresponding five stimuli, we constructed a model from the response
during the transition. The model was constructed by using the firing rate of the cell during the corresponding transition at the relevant points during the presentation (taking into account a cell latency of
80 ms) and the average firing rate to the from-stimulus and the
to-stimulus. We expected that if a cell was temporally independent, the
average firing rate to the constant-stimulation flow fields could be
predicted by this model. We tested this hypothesis with the chi-square
test, comparing the model with the average firing rate of the
corresponding constant stimuli. In only 7.2% of the transitions tested
was there a significant ( = 0.05) difference between the two
sets of firing rates. In the rest of the transitions, the response to
the transition correctly predicted the responses during constant
stimulation. This was the case during both linear and nonlinear
transitions. As a consequence, the nonlinearities in the responses to
transitions were not caused by temporal context, in most cases.
|
TEMPORAL-SYMMETRY TEST. In the temporal-symmetry test, we recorded the responses of cells to a transition and its inverse (the same transition presented backward) and then used the chi-square test to determine whether the distribution of the responses in the second case was a mirror image of the distribution obtained with the first transition, taking into account a latency of 80 ms (Fig. 10). If a cell was temporally independent, we would expect that the response during the reverse transition would be a mirror image of the response during the original transition. While recording from a cell, we analyzed the data and selected the transitions with the highest firing rate. In the broad temporal-symmetry test, we tested all the transitions to and from the cardinal flow fields defining the transition with the highest firing rate. In the restricted test, we tested only the three transitions with the highest firing rates and their inverses.
|
HEADING TEST. In the transition-tuning test we randomly sampled the transition set and chose 27 of 169 transitions. In the heading test we took a different approach: we selected a small region in the stimulus space and tested the transitions between every pair of constant flow fields in each cell. Because MST cells are thought to be involved in the perception of self-motion, and forward motion is a very common type of self-motion, we selected five flow fields simulating forward motion (straight ahead and 30° and 60° to the right and to the left) and the stationary random-dot pattern. In sum, we presented six stimuli and 30 transitions in each test; two axes (forward translation and left/right translation) defined all stimuli and transitions. As a result, we could see how the responses to transitions changed within a small region of stimulus space in which transitions are similar (Fig. 11).
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Area MST is widely agreed to be involved in the processing of
visual input generated by self-motion in the environment
(Bremmer et al. 1999; Duffy 1998
;
Duffy and Wurtz 1991a
,b
; Saito et al. 1986
; Tanaka and Saito 1989
; Tanaka et
al. 1986
, 1989
; Thier and Erikson 1992
). The
flow-field patterns generated by self-motion change frequently as a
result of eye, head, or body movement. As a consequence, cells in area
MST often experience changes in the flow-field pattern and knowledge
about their responses to such changes is important in understanding the
functional role of MST and the computations that take place at the
single-cell level. We analyzed the responses of MST cells to
continuously changing and constant stimuli to understand their role in
processing information about temporal changes in the motion pattern
and, more generally, to establish if temporal information affects their response.
We pursued this aim in three stages. First, we verified that the cell responses were not affected by the presentation paradigm (continuous stimulation vs. stimuli alternated with ISI) or by preceding stimulation. These results provided the necessary justification for the analysis of individual responses to stimuli and transitions embedded in continuous nonrandomized sequences.
Second, we showed that the responses during transitions were, in more than half of the cells, nonlinear with respect to the stimuli defining the transitions. The nonlinearity during transitions could either be due to dependence on temporal context or to tuning nonlinearities to flow-field patterns (cells were tuned to a specific flow-field pattern that was present during the transition; the cells preferred that pattern over the from- and the to-stimulus).
To distinguish between these two hypotheses, we tested more specifically for dependence on temporal context. A cell is temporally independent if its response at a given time is a function of the stimulus being presented at that time minus the latency. The temporal-independence test showed that the responses in most cases were not significantly affected by temporal context. Furthermore, the responses to a transition and its inverse are each a mirror image of the other (temporal-symmetry test). Finally, we tested for temporal independence within a restricted region of the stimulus space (straight-ahead movements) in the heading test. The results obtained confirmed that the responses of the cells are temporally independent and, within the small range of flow fields presented, in most cases the tuning also was linear.
Cardinal flow fields inadequate basis for mapping of visual stimuli
Our results indicate that nonlinearities in tuning are a source of nonlinearity in the cell responses during transitions. By nonlinearities in tuning we mean that the responses to cardinal flow fields often do not correctly predict the responses of a cell during transitions. In our cell sample, over half of the transitions in the transition-tuning test were nonlinear. Dependence on temporal context did not account for these nonlinearities and we therefore concluded that they were, in most cases, caused by nonlinearities in the tuning profile to flow fields. The higher percentage of nonlinear responses to highly effective stimuli than to moderately effective ones also suggests that there was a relationship between nonlinearities and tuning.
Not all nonlinearities in the responses during transitions were caused by tuning nonlinearities, however. Although the flow-field patterns during a transition were linear combinations of the two contributing flow fields, there were a few special cases in which, for instance, the linear combination could result in a stationary random-dot pattern (e.g., in the reversal of a flow-field direction). In these cases, nonlinearities in the response profiles were expected as a consequence of linear tuning to flow fields and temporal independence.
For instance, in a transition from upward motion to downward motion, the midpoint was a stationary random-dot pattern. If a cell was responsive to the second flow, we would expect a nonlinear response: a sudden rise in the firing rate in the second half of the transition, with a peak-shaped response if the cell prefers low speeds, a flat firing rate (step model) if the cell is not strongly sensitive to speed, or a gradually increasing firing rate if the cell prefers higher speeds.
We reported that the type of nonlinear responses (with respect to the
to- and from-stimuli) that would be predicted by linear tuning to flow
fields was more common within the transition types that we analyzed
separately than in the entire sample of transitions. For instance, we
observed a higher percentage of peak and step models in transitions in
which the middle point was stationary. Among accelerations, we found a
higher-than-average percentage of straight-line fits, which most likely
reflects speed-tuning curves and which is in agreement with
psychophysical results that indicate that there are no dedicated
mechanisms devoted to the perception of acceleration (Monen and
Brenner 1994; Simpson 1994
; Werkhoven et
al. 1992
). Although the responses were, in several cases,
consistent with linear tuning to flow fields, for each transition group
there were still substantial nonlinearities present. The fitting of the
three transition models (straight-line, peak, and step) to different
transition types shows, furthermore, that nonlinearities in the
responses were not concentrated within specific transition types.
Our results point toward a more complex tuning function to flow fields.
MST cells were responsive to both cardinal and linear-combination flow
fields, thus generalizing the finding that MST cells may be responsive
to any part of the full spectrum of the 6D flow- field space
(Graziano et al. 1994; Lagae et al. 1994
;
Orban et al. 1992
) and that they do not linearly
decompose linear-combination flow fields (often referred to as complex
flow fields) into cardinal flow fields (Orban et al.
1992
). The presence of complex tuning functions that span the
entire stimulus space points out methodological risks associated with
using only a limited set of flow fields (the cardinal flow fields used
here or other similar sets) to obtain a full characterization of the
tuning properties of MST cells.
Temporal independence
The main result presented here is that MST cells are temporally
independent. Information from temporal context (preceding stimulation)
does not significantly affect the cell responses. Previous work
(Duffy and Wurtz 1997) has shown that the responses to a
single stimulus remained constant for periods of up to 15 s. The
absence of a significant dependence on temporal context when the
stimulus did not change through time (i.e., the response was not
transient) agrees with our results. The initial, nonspecific, transient
peaks that we found in the stimuli interleaved with ISI were the only
effect of temporal context that we identified, and they confirm
previous reports (Duffy and Wurtz 1991a
, 1997
). We also
showed that the initial peak simply reflected the onset of stimulus
motion and could be eliminated by using continuous stimulation.
Duffy and Wurtz (1997) also found that the response to
the same stimulus may be affected within longer time periods by the frequent presentation of specific groups of stimuli. It may be argued
that this is an example of dependence on temporal context that
contradicts our results. We do not think this is the case. The two
experiments look at the temporal effects from a different perspective
and reach different, but by no means opposing, conclusions. We asked
whether the response to a given stimulus is affected by those
immediately preceding it. For each test we generated a random sequence
of stimuli and presented it several times. We then looked at the
responses to the same stimulus embedded in different stimulus
sequences. Although Duffy and Wurtz (1997)
investigated
longer-term effects of temporal context, which was defined as the
entire stimulus set used in the test, they compared the responses to
the same stimulus at the beginning and at the end of the experiment.
Duffy and Wurtz (1997)
looked at the effects of
habituation and novelty; we were interested in temporal independence within a shorter time frame. Within the shorter time frame, we did not
see any significant dependence on temporal context, and the presence of
the longer-term effects described by Duffy and Wurtz
(1997)
does not strengthen or weaken this result.
If MST cells were temporally dependent, they might be expected to process information about specific paths in the 3D environment or to respond differently to changing and constant flow fields. However, we were able to show that they are temporally independent and, therefore, that they do not explicitly encode the second-order properties of the stimuli. In other words, MST cells are tuned to specific flow-field patterns but they do not code for specific changes in the patterns. Presumably, such second-order properties and path information is processed at the cell population level or elsewhere.
Consequences of temporal independence for continuous and continuously changing stimulus presentation
Temporal independence in MST cells has important methodological
consequences. Because cells respond only to the stimulus currently being presented, it is possible to characterize the tuning properties of MST cells using continuous stimulation with frequent changes in the
flow field. In these experiments, the flow-field pattern changed at
every frame during transitions, but other regimens are equally
feasible. The major advantage of continuous stimulation is that it
allows the amount of data collected from a cell in a fixed amount of
time to be increased (Britten and Heuer 1999; Hoffmann and v. Seelen 1978
; Paolini and Sereno
1997a
,b
, 1998
; Paolini et al. 1998
). An
additional advantage lies in the elimination of the nonspecific initial
peak [previously described by Duffy and Wurtz (1997)
],
which complicates data analysis.
In addition to presenting stimuli continuously, the data presented here show that it is possible to present stimuli whose parameters gradually change in time, as long as the perception of continuous movement is preserved. Here we limited ourselves to the use of flow fields, but it may be possible to use other stimuli in other cortical areas as long as it is possible to show that cells are temporally independent. Continuously changing stimulation offers a fast method for mapping the tuning of temporally independent cells. The responses during transitions can be interpreted as showing the tuning of the cell to two cardinal flow fields and to all their linear combinations that are included in the transition.
![]() |
ACKNOWLEDGMENTS |
---|
This work was supported in part by Deutsche Forschungsgemeinschaft Grant SFB509/B1 and Human Frontier Science Program Grant R6-71/96B.
![]() |
FOOTNOTES |
---|
Address for reprint requests: M. Paolini (E-mail: paolini{at}neurobiologie.ruhr-uni-bochum.de).
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
Received 8 September 1999; accepted in final form 17 April 2000.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|