Neuronal Activity in Medial Frontal Cortex During Learning of Sequential Procedures

Kae Nakamura1, Katsuyuki Sakai1, 2, and Okihide Hikosaka1

1 Department of Physiology, Juntendo University, School of Medicine; and 2 Department of Neurology, Division of Neuroscience, Graduate School of Medicine, University of Tokyo, Tokyo 113, Japan

    ABSTRACT
Abstract
Introduction
Methods
Results
Discussion
References

Nakamura, Kae, Katsuyuki Sakai, and Okihide Hikosaka. Neuronal activity in medial frontal cortex during learning of sequential procedures. J. Neurophysiol. 80: 2671-2687, 1998. To study the role of medial frontal cortex in learning and memory of sequential procedures, we examined neuronal activity of the presupplementary motor area (pre-SMA) and supplementary motor area (SMA) while monkeys (n = 2) performed a sequential button press task, "2 × 5 task." In this paradigm, 2 of 16 (4 × 4 matrix) light-emitting diode buttons (called "set") were illuminated simultaneously and the monkey had to press them in a predetermined order. A total of five sets (called "hyperset") was presented in a fixed order for completion of a trial. We examined the neuronal activity of each cell using two kinds of hypersets: new hypersets that the monkey experienced for the first time for which he had to find the correct orders of button presses by trial-and-error and learned hypersets that the monkey had learned with extensive practice (n = 16 and 10 for each monkey). To investigate whether cells in medial frontal cortex are involved in the acquisition of new sequences or execution of well-learned procedures, we examined three to five new hypersets and three to five learned hypersets for each cell. Among 345 task-related cells, we found 78 cells that were more active during performance of new hypersets than learned hypersets (new-preferring cells) and 18 cells that were more active for learned hypersets (learned-preferring cells). Among new-preferring cells, 33 cells showed a learning-dependent decrease of cell activity: their activity was highest at the beginning of learning and decreased as the animal acquired the correct response for each set with increasing reliability. In contrast, 11 learned-preferring cells showed a learning-dependent increase of neuronal activity. We found a difference in the anatomic distribution of new-preferring cells. The proportion of new-preferring cells was greater in the rostral part of the medial frontal cortex, corresponding to the pre-SMA, than the posterior part, the SMA. There was some trend that learned-preferring cells were more abundant in the SMA. These results suggest that the pre-SMA, rather than SMA, is more involved in the acquisition of new sequential procedures.

    INTRODUCTION
Abstract
Introduction
Methods
Results
Discussion
References

Many behaviors rely on learning of a sequence of movements. Studies using primates have revealed that several brain regions are involved in the performance of sequential movements. Neurons that change their activity with particular transitions or combinations of movement sequences rather than movements per se have been found in the prefrontal cortex (Joseph and Barone 1987), premotor cortex (Mushiake et al. 1991), supplementary motor area (Mushiake et al. 1990), caudate nucleus (Kermadi and Joseph. 1995), globus pallidus (Mushiake and Strick 1995), and dentate nucleus (Mushiake and Strick 1993). These studies focused on how sequential movements are controlled because the monkeys already had learned the sequential movements with extensive practice before the experiments started. However, acquisition and control of movement sequences could possibly be mediated by separate neural mechanisms (Keele and Summers 1976; Shadmehr and Brashers-Krug 1997; Shadmehr and Holcomb 1997; Summers 1981).

We hypothesized that the areas for acquisition of new procedures (acquisition mechanism) and the areas for storage of long-term memories and their retrieval (storage mechanism) exist separately. This hypothesis can be tested if the monkey has a repertoire of well-learned procedures and at the same time has an opportunity to learn totally new procedures. Neurons related to the acquisition mechanism would become active while the animal is learning new procedures; neurons related to the storage mechanism would active while the animal is performing learned procedures.

The sequential button task, called 2 × 5 task (Hikosaka et al. 1995) is suitable to test this hypothesis because a large number of motor sequences can be generated. The task was to press five consecutive pairs of target buttons (indicated by illumination), in the correct order for each pair. Each pair was called the "set" and the whole sequence was called "hyperset." This task required the subject to find out correct order in which to press the button by trial and error. In a previous paper (Hikosaka et al. 1995), we showed that monkeys could learn many hypersets and, with daily practice, acquired excellent procedural skills for 10-20 learned hypersets. Learning occurred for each hyperset repeatedly, suggesting that the long-term memories were generated for individual sequences. Important here was that, even after the mastery of the procedural skills, we could ask the monkeys to learn new hypersets repeatedly that were generated by the computer. These behavioral studies have provided us with a good experimental system to test the separate acquisition/storage mechanisms described above.

In this paper, we focus on the role of medial frontal cortex in acquisition and memory of sequential procedures. Many studies have demonstrated that the medial frontal region, especially the supplementary motor area (SMA), is important for the control of sequential movements in primates (Halsband et al. 1994; Mushiake et al. 1990, 1991; Tanji and Shima 1994) and human (Lang et al. 1990; Roland et al. 1980; Shibasaki et al. 1993). Recent anatomic and physiological studies indicated that the classical SMA is subdivided into two distinct areas, the presupplementary motor area (pre-SMA), located rostrally, and the SMA proper (SMA), located caudally (Matelli et al. 1991; Matsuzaka et al. 1992; Rizzolatti et al. 1990). These areas have different cortical and subcortical connections (Bates and Goldman-Rakic 1993; Dum and Strick 1991; He et al. 1995; Hutchins et al. 1988; Lu et al. 1994; Luppino et al. 1993, 1994; Matelli and Luppino 1996; Matelli et al. 1995). We wanted to know whether the pre-SMA and SMA have different roles in learning and memory of sequential movements.

In this paper, we describe the neuronal activity in these two areas. We asked two main questions: are there any differences in neuronal activity for the performance of new sequences and well-learned sequences? And are there any changes of neuronal activity during the acquisition of new sequences? We will report that many neurons, especially in the pre-SMA, were preferentially active for new sequences, suggesting that the pre-SMA is related to the acquisition mechanism.

    METHODS
Abstract
Introduction
Methods
Results
Discussion
References

Experimental animals

We used two male Japanese monkeys (Macaca fuscata): monkey GA (5.5 kg) and monkey BO (10.5 kg). A total of four hemispheres were surveyed. The monkeys were kept in individual primate cages in an air-conditioned room where food was always available. At the beginning of each experimental session, they were moved to the experimental room in a primate chair. The monkeys were given restricted amounts of fluid during periods of training and recording. Their body weight and appetite were checked daily. Supplementary water and fruit were provided daily. Throughout the experiment the monkeys were treated in accordance with the Guiding Principles for Research Involving Animals and Human beings by the American Physiological Society.

Surgery

The experiments were carried out while the monkey's head was fixed and its eye movements were recorded. For this purpose, a head holder, a chamber for unit recording, and an eye coil were implanted under surgical procedures. The monkey was sedated by intramuscular injections of ketamine (4.0-5.0 mg/kg) and xylazine (1.0-2.0 mg/kg). General anesthesia then was induced by intravenous injection of pentobarbital sodium (5 mg·kg-1·h-1). Surgical procedures were conducted in aseptic conditions. After exposing the skull, 15-20 acrylic screws were bolted into it and fixed with dental acrylic resin. The screws served as anchors by which a head holder and a chamber, both made of delrin, were fixed to the skull. A scleral eye coil was implanted in one eye for monitoring eye position (Judge et al. 1980; Robinson 1963). The recording chamber was implanted tangentially to the cortical surface, centered on the midline of the frontal cortex. The monkey received antibiotics (sodium ampicillin 25-40 mg/kg intramuscularly each day) after the operation.

Apparatus

A detailed description of the apparatus and behavioral paradigm was presented in the previous paper (Hikosaka et al. 1995). Briefly, the monkey sat in a primate chair and faced a black panel on which 16 light-emitting diode (LED) buttons were mounted in a 4 × 4 matrix. At the bottom of the panel was another LED button that was used as a home key. To have the monkey use only one hand for a button press, a vertical Plexiglas plate was attached to the chair in an oblique direction between the plate and the unused hand. To reverse the hand that was used, the plate was replaced to the other side. The monkey's head was fixed with a head holder connected to the primate chair. A metallic pipe for supply of reward (water) was positioned in front of the monkey's mouth.

Behavioral paradigms

2 × 5 TASK. The monkeys' task, the 2 × 5 task was to press five consecutive pairs of buttons in the correct order, which they had to discover by trial and error in a block of trials. Figure 1A shows an example of the sequence of events in a single task trial. The whole sequence was called a hyperset; each pair was called a set. Time line showing task periods and events are illustrated in Fig. 1B. At the start of a trial, the home key was illuminated. After the monkey pressed the home key for 1 s, 2 of the 16 target LEDs, the first set, turned on simultaneously. The monkey had to press the illuminated buttons in the correct order; the animal had to choose one of two buttons as a first press followed by the second press. If successful, the two buttons were extinguished one by one as they were pressed, and another pair of LEDs, a second set, was illuminated and the monkey had to press them in the correct order again. Each hyperset consisted of five sets, presented in a fixed order. Liquid reward was given after successful completion of each set. The amount of reward was increased toward the final (5th) set, which encouraged the monkey to complete the whole hyperset. Also we gave additional reward to encourage the animal to do the task as quickly as possible; the amount of given water was increased as the performance time was shorter. If the wrong button was pressed at any point in the hyperset, the trial was regarded to be unsuccessful and was aborted, and the monkey had to start again from the home key to initiate a new trial. Each hyperset was presented repeatedly in a block until 10-20 successful trials had been performed. A different hyperset then was used for the next block. Successive trials were separated by an interval of 0.5-3 min by inactivating the panel.


View larger version (20K):
[in this window]
[in a new window]
 
FIG. 1. A: sequence of events for a representative trial of the 2 × 5 task. For each set, the monkey had to press the 2 illuminated buttons in the correct order (denoted as 1 and 2) to proceed to the next set. Monkey had to find the correct order by trial and error. B: time line showing task periods and events. After pressing a home key for 1 s, the 2 light-emitting diode (LED) buttons of the 1st set were illuminated simultaneously. Each illuminated button was turned off when the monkey pressed it. We analyzed the neuronal activity for the movement period: from the onset of the 1st set to the 2nd button press in the 5th set.

NEW AND LEARNED SEQUENCES. We hypothesized that there are separate neural mechanisms underlying procedural learning and memory, one for acquisition of new procedures and the other for storage of long-term memories and their retrieval. The prerequisite for testing this hypothesis was the experimental situation in which the monkey had acquired long-term procedural memories and at the same time had opportunities to learn new procedures repeatedly.

The 2 × 5 task was ideal for this purpose because new sequences (hypersets) can be generated practically as many as possible [(5P2)5]. We asked the monkeys to perform newly computer-generated hypersets (new hypersets). Monkey BO and monkey GA, respectively, had experienced 218 and 106 new hypersets before the unit recording started and 1,109 and 603 new hypersets during the recording experiments. Most of them were performed just once (1 block); and half of them were performed by the right hand, the other half were by the left hand.

To create long-term memories, we asked the monkeys to practice a standard group of hypersets (16 for monkey BO and 10 for monkey GA) almost every day for >2 yr for monkey BO and 8 mo for monkey GA before recordings started. As a result, the monkey became very skillful in performing them. We called them "learned hypersets." Half of the learned hypersets were performed always by the right hand, the other half by the left hand.

Figure 2 shows the representative performance of monkey BO for a new hyperset (A) and a learned hyperset (B). For the new hyperset, the number of completed sets increased through trial-and-error processes, and after the 15th trial the monkey no longer made errors (Fig. 2Aa). The time for the completion of one successful trial (the performance time) decreased gradually as learning proceeded (Fig. 2Ab). A similar learning process occurred every time the monkey performed a new hyperset. The mean number of errors to criterion (10 successful trials) for new hypersets was 7.4 ± 5.4 (mean ± SD) for monkey BO and 10.3 ± 5.9 for monkey GA. The mean performance time was 4.6 ± 0.8 s and 5.0 ± 0.5 s, respectively. On the other hand, for the learned hyperset, the monkey performed learned hyperset with few errors, as shown in Fig. 2Ba. The monkey completed the whole hyperset (5 sets) typically with no error, and the performance time was much shorter than that for the new hyperset from the first trial. The number of errors to criterion for the learned hypersets was 0.6 ± 1.0 and 0.5 ± 1.2, for monkeys BO and GA, respectively. The mean performance time was 3.5 ± 0.9 s and 3.6 ± 0.3 s for learned hypersets, for monkeys BO and GA, respectively.


View larger version (24K):
[in this window]
[in a new window]
 
FIG. 2. Monkey's performance for a new hyperset (A) and a learned hyperset (B). Number of completed sets (a) and performance time for a successful trial (b) are plotted against trial number. For the new hyperset, the number of completed sets increased and the performance time decreased as learning proceeded. For the learned hyperset, the monkey completed the whole 5 sets and the performance time was minimum from the initial trial.

SIMPLE REACTION TASK. In addition to 2 × 5 task, we used a visually guided, simple reaching task ("simple reaction task"). After pressing the home key for 1 s, 1 of 16 buttons turned on. The monkey was required to press the illuminated button to obtain reward. The location of the target was chosen pseudorandomly such that each 1 of 16 targets appeared once for 16 consecutive trials.

Intracortical microstimulation

To determine the locations of the pre-SMA and SMA, we performed intracortical microstimulation before a series of extracelluar recording experiments. The stimuli were trains of cathodal pulses generated by a constant current stimulator. The following parameters were used: trains of 20-60 cathodal pulses (duration: 200 µs), at 300 Hz, 20-80 µA. The current strength was controlled on an oscilloscope measuring the voltage drop across a 10-kOmega resistor in series with the stimulating electrode. The stimuli were applied while the monkey was sitting in the chair, alert, while two investigators were observing evoked body movements. The threshold current was determined at the current intensity by which a body movement was evoked in about half of the stimulation trials.

Electrode penetrations were usually spaced at 2-mm intervals in the rostrocaudal direction (Fig. 3). We also performed additional stimulation experiments during the recording session to make sure that the evoked movements were in the upper limb. In each penetration, the first stimulation was carried out at the site at which the first action potentials were recorded. The subsequent stimulation was made every 500 µm for the length of 8-10 mm.


View larger version (43K):
[in this window]
[in a new window]
 
FIG. 3. Distribution of stimulus-evoked body movements (A) and task-related neurons (B) in monkey BO. Data are shown on representative coronal sections as viewed from the caudal end (left and right hemispheres are shown on left and right; distance between sections: 2 mm). Arrows (PA) indicate the rostrocaudal level corresponding to the genu or arcuate sulcus. Border between the presupplementary motor area (pre-SMA) and SMA was determined by the results of intracortical microstimulation. A: kinds of stimulus-evoked body movements are indicated by different symbols, their sizes indicating the threshold currents. Small symbol, >60 µA; medium, 40-60 µA; large, <40 µA. Dots indicate the sites from which no movement was evoked. B: recording sites are indicated along electrode tracks (vertical lines) the entry points of which are indicated by black dots on the surface. Blue rectangles indicate new-preferring cells; red rectangles indicate learned-preferring cells; black short bars indicate other task-related cells. Note that neurons were recorded mostly in the medial wall of the frontal cortex and above the cingulate sulcus. C: penetration sites and distribution of new-preferring (New > Learned) and learned-preferring (Learned > New) cells. The top view of the brains (anterior upward) of 2 monkeys are presented. Ratios of new- and learned-preferring cells were calculated by dividing their number by the total number of task-related cells recorded at each penetration site. Ratios are expressed by the sizes of squares (new preferring) and circles (learned preferring). Sulci were drawn according to the histology (monkey BO) and the magnetic resonance imaging (MRI; monkey GA). PS, principal sulcus; ARC, arcuate sulcus; CS, central sulcus; PA, genu of arcuate sulcus.

Single-unit recording

For monkey BO, glass-coated, elgiloy electrodes (1.0-2.0 MOmega measured at 1 kHz) were inserted through the exposed dura to record single neurons in the medial frontal cortex. It was difficult, however, using this method to estimate the depths of recorded neurons because the electrode, as passing through the dura, tended to depress the brain surface. For monkey GA, after determining the locations of the pre-SMA and SMA by intracortical microstimulation, we implanted Teflon guide tubes (outer diameter: 0.85 mm, inner diameter: 0.6 mm), which were fixed on the skull using dental acrylic resin so that their tips were positioned below the dura and close to the surface of the brain. They could be removed and re-implanted at different locations within the chamber. The operation was performed under general anesthesia with ketamine and xylazine. Single-unit recordings were performed using tungsten electrodes (diameter: 0.25 mm, 1-5 MOmega , measured at 1 kHz, Frederick Haer) through these guide tubes. The procedure allowed us to estimate the depths of recorded neurons because the depression of the brain surface was minimized. We found that electrode penetrations performed several weeks apart through the same guide tube yielded neurons with similar responses at similar depths. Single-unit potentials were amplified, filtered with a band-pass of 500 Hz to 5 kHz, and digital-sampled using a window discriminator.

Eye-movement recording

Eye movements were recorded using the search coil method (Enzanshi Kogyo MEL-20U) (Judge et al. 1980; Matsumura et al. 1992; Robinson 1963). Eye positions were digitized at 500 Hz and stored into an analog file continuously during each block of trials. On the computer monitor were presented, as a two-dimensional display, the states of the 4 × 4 target arrays (e.g., whether they are illuminated or pressed) and the current and recent eye positions. Horizontal and vertical eye positions also were displayed, for each set, relative to the time before and after the onsets of target LEDs.

Control of experiments and data acquisition

The behavioral tasks as well as storage and display of data were controlled by a computer (PC 9801RA, NEC, Tokyo). The time and nature of task-related events (e.g., onset and offset of LED targets, pressing and releasing of buttons, neuronal activation) were stored into an event file for off-line analysis. Eye positions were digitized at 500 Hz and stored into an analog file continuously during each block of trials.

Experimental procedures

To record neuronal activity, the electrode was advanced while the monkey performed the 2 × 5 task until task-related activity was found. We were careful to examine the task-relation of neuronal activity for both new hypersets and learned hypersets because there were cells that were activated for only one of them. If the neuron exhibited any change in activity at any period during a trial by visual inspection, data acquisition was initiated. Otherwise, the electrode was advanced to find the next neuron.

To examine whether the cell activity was related to the acquisition of new sequence or performance of well-learned sequence, we had the monkey perform several new hypersets and learned hypersets while a single cell was being recorded. For each cell in monkey BO, we examined five new hypersets, five learned hypersets, and the simple reaction task performed by the hand contralateral to the recording site. For each cell in monkey GA, we examined three new and three learned hypersets and the simple reaction task by the contralateral hand and one new and one learned hypersets performed by the ipsilateral hand. New hypersets and learned hypersets were examined alternately. Occasionally, we examined the neuronal activity for partially changed learned hypersets; e.g., set 2 and set 3 were changed. To ensure the condition of recording was not changed throughout all tasks, we presented the first hyperset again at the end of the recording session.

Data analysis

TASK RELATIONS. We focused our analysis on the period from the onset of the stimuli in set 1 to the second button press in set 5 (see Fig. 1B), which will be called "movement period." We compared the neuronal activity during the movement period with "baseline activity," which was obtained for a 1-s period between hypersets (blocks) during which the monkey was not performing any task while resting his hand on the home key gently.

DIFFERENCE BETWEEN NEW AND LEARNED HYPERSETS. To determine whether a recorded neuron showed preferential activity for new hypersets or learned hypersets, we first calculated the discharge rates during the movement period for each trial. A statistical comparison was made for each cell between the pooled data for the first five successful trials of new hypersets and learned hypersets (Mann-Whitney U test, P < 0.01). Only the first five trials were examined because the neuronal activity often changed rapidly while the monkey was learning the new hypersets; i.e., learning-dependent change (see next section).

LEARNING-DEPENDENT CHANGE. A learning-dependent change for new hypersets was first assessed by visual inspection. A statistical comparison was performed for the discharge rates during the movement period between the initial five successful trials and the following five successful trials (Mann-Whitney U test, P < 0.05). A cell showing a change in activity for at least one hyperset was categorized as showing learning-dependent change.

There were three exceptions for the analysis. In the first case, the change was prominent in the very early stage of learning especially before the first success of a trial; i.e., when only the initial part of the hyperset (1, 2, 3, or 4 sets) was completed. In this case, we calculated the discharge rates only for the completed sets and compared the values for the initial five trials with those for the following five trials. In the second case, the change in discharge rate was observed for particular sets. In this case, we compared the discharge rates for individual sets separately. When we found significant changes (Mann-Whitney U test, P < 0.05) for more than three of five sets, we determined that the cell showed a learning-dependent change. In the third case, the change was slow so that the completion of 10 successful trials was not enough to observe a significant change. In this case, we compared the discharge rate for the initial five trials and later five trials (typically 16-20 trials).

Histology

After recording and injection were completed, monkey BO was anesthetized with an overdose of pentobarbital sodium and perfused through the heart with 4% Formalin. The brain was blocked and equilibrated with 30% sucrose. Frozen sections were cut at 50 µm in the planes parallel to the electrode penetrations so that complete tracks were visible in single sections. The sections were stained with thioneine. Reconstruction of the location and extent of SMA and pre-SMA was based on microlesions (5 µA for 200 s) made at every 2 mm within the chamber, 1-3 mm deep from the surface of the cortex. Individual recording and injection sites were estimated based on these microlesions. Monkey GA is still being used for further experiments.

MRI

After the implantation of the recording/injection chambers, we obtained magnetic resonance images (MRI) for both monkeys (Hitachi Laboratory MRIS, 2.11 tesla for monkey BO; Hitachi AIRIS, 0.3 tesla for monkey GA) using the procedure described by Kato et al. (1995). We confirmed in monkey BO that the recording sites estimated on the MR images well corresponded with the histologically identified microlesions. In monkey GA, the reconstruction of recording sites was based on the MR images.

    RESULTS
Abstract
Introduction
Methods
Results
Discussion
References

We found neurons in the medial frontal cortex that were activated selectively during performance of either new or learned hypersets. The following sections describe the locations at which we found such neurons and their differential responses during learning of new hypersets and performance of well-practiced hypersets.

Location of recording sites in SMA and the pre-SMA

We distinguished the SMA and the pre-SMA as previously proposed (Luppino et al. 1991; Matsuzaka et al. 1992; Matsuzaka and Tanji 1996). In the SMA was found a somatotopic representation such that the face-arm-leg regions were arranged rostrocaudally (Luppino et al. 1991); movements were elicited with low thresholds (20-40 µA) at 20 pulses. Rostral to the face region of the SMA was another focus from which arm movements were evoked, which we determined to be the pre-SMA (Luppino et al. 1993); larger currents (40-80 µA) and more pulses (40-60 pulses) were needed to evoke movements (Fig. 3). Movements evoked from the SMA tended to be brisk and consistent across trials, whereas movements evoked from the pre-SMA tended to be more complex, slow, or variable across trials. In the rostral part of the pre-SMA, even the strong stimulation was often ineffective; in such cases, the effects of stimulation could be observed only as an arrest of voluntary hand movements. We eliminated the data obtained from the area where eye movements were evoked, which was considered to be the supplementary eye field.

Recording experiments revealed two foci of task-related neurons that corresponded to the arm region of the SMA and the pre-SMA determined by the microstimulation (Fig. 3).

General characteristics of medial frontal cortical cells

Among 2,098 cells recorded from 4 hemispheres, 728 cells showed task-related activity. Among them, we analyzed neuronal activity for 345 cells for which we recorded complete data sets (Table 1). We determined that 116 cells were from the pre-SMA and 69 cells were from the SMA (monkey BO), and 99 cells were from the pre-SMA and 61 cells were from the SMA (monkey GA). In the present study, we analyzed the spike activity of single cells during the movement period (period from the onset of the stimuli in set 1 to the 2nd pressing in set 5, see Fig. 1B). There were some cells that showed spike activity specifically during the intertrial intervals, but we did not analyze them.

 
View this table:
[in this window] [in a new window]
 
TABLE 1. Activity of new and learned hyperset preferring neurons

For monkey GA, we examined neural activity while the animal performed the task using the hand on each side (contralateral or ipsilateral to the recording site). Among 34 SMA cells examined, 18 (52.9%) were activated more strongly when the contralateral hand was used; 16 (47.1%) cells showed no preference. Among 28 pre-SMA cells, 10 (35.7%) showed preference for the contralateral hand, whereas 18 (64.3%) showed no preference. For monkey BO, only the contralateral hand was examined.

We also examined neural activity for a simple reaction task that required nonsequential, visually guided button pressing. All of the cells that were activated for the simple reaction task also were activated for the 2 × 5 sequential button press task. Conversely, about half of the cells that showed significant neuronal activity for the 2 × 5 sequential task were activated also for the simple reaction task (pre-SMA: 55.5%, SMA: 64.4%).

We tried to correlate the cell activity with the locations of individual stimuli or their sequences, but we could not find obvious correlation by visual inspection. In some cells, however, activity was greater for earlier sets (set 1 or 2) than for later sets (set 4 or 5) for all hypersets examined. Such cells were found more frequently in the pre-SMA (27 of 215 cells) than in the SMA (5 of 130 cells). We also found three cells in the pre-SMA that were activated or suppressed specifically for set 5.

The most salient finding in the present study was that many cells were activated differently for new hypersets and learned hypersets as shown in Table 1. We first will describe neurons that were activated preferentially for new hypersets and then neurons preferentially activated for learned hypersets.

Neurons preferentially activated for new sequences

COMPARISON BETWEEN NEW AND LEARNED SEQUENCES. We found that a group of neurons that were activated more strongly for performance of new hypersets than learned hypersets. Figure 4 shows a typical example obtained from one left pre-SMA cell of monkey GA. For new hypersets (Fig. 4A), the neuron started discharging after the illumination of stimuli for each set. The discharge rate increased gradually until the monkey pressed the first button. In contrast, the same neuron showed almost no spike discharge for four learned hypersets (Fig. 4B) except for the first set in the first trial.


View larger version (49K):
[in this window]
[in a new window]
 
FIG. 4. Activity of a left pre-SMA cell for new hypersets (A) and learned hypersets (B). Spike activity, shown by rasters and histograms, are aligned at the time when the monkey pressed the 1st button for each set. Only activity for correctly executed trials are shown, the 1st trial at top, the last at bottom. Inverted triangles in the raster indicates other task-related events (stimulus onset and second button press). For the new hypersets, the cell showed phasic activity for every set before the 1st button press; for the learned hypersets, it was nearly silent except for the 1st trial. Note that the activity for the new hyperset decreased as learning proceeded. Top 3 rows: data when the monkey used the hand contralateral to the recording site. Bottom row: data when the ipsilateral hand was used. Correct orders of button presses for the hypersets used are shown below.

The neuron was activated for all four of the new hypersets but remained nearly silent for all of the learned hypersets. Interestingly, the differential activation for new and learned hypersets was observed even when the hand ipsilateral to the recording site was used (Fig. 4). Furthermore, similarities in stimuli had little significance for the differential activation: for example, the same stimuli appeared in set 5 of hyperset R1 (learned) and in set 3 of hyperset L375, but the neuron's activity was completely different.

To evaluate whether a neuron differentiated between new sequences and learned sequences, we got the pooled data (i.e., discharge rates) obtained from the first one to five successful trials across several hypersets and performed a statistical comparison (Mann-Whitney U test) between new hypersets and learned hypersets. Using a statistical criterion (P < 0.01), we classified neurons into three groups: new-preferring cells, learned-preferring cells, and nonselective cells. Applying this analysis, the neuron shown in Fig. 4 was determined to be new preferring (P < 0.0001).

Most of the new-preferring cells (82%) showed dominant activity in the same time period of most of five sets. A majority of them (90%) showed activity between the stimulus onset and the first button press (as in Fig. 4), when the animal was uncertain and more demand of decision making was required. A smaller portion of cells (10%) showed activity between the first button press and the second button press.

We examined whether the activity for new hypersets reflects attention to the visual stimulus by using the simple reaction task: 73. 7% (monkey BO) or 56.1% (monkey GA) of the new-preferring cells showed activity also in this task.

Learning-dependent decrease of neuronal activity

The new-preferring cells decreased their activity as learning proceeded. The raster display in Fig. 4A shows that activity decreased as the monkey became familiar with the new hyperset. This is shown graphically in Fig. 5A, which included the data for unsuccessful trials as well. The neural activity was high initially when the monkey was still unable to complete the whole hyperset (5 sets) and then decreased gradually as performance was improved. The decrease in cell activity continued even after the monkey could complete the whole hyperset; note, however, that the performance time continued to decrease. Such a learning-dependent decrease in cell activity was repeated every time a new hyperset was introduced (as shown in Fig. 4).


View larger version (29K):
[in this window]
[in a new window]
 
FIG. 5. Learning-dependent decrease in pre-SMA cell activity (a) in comparison with the monkey's performance [number of completed sets (b) and performance time (c)] (data shown in Fig. 4, top, were analyzed). Abscissa indicates the trial number. Neuronal activity (a) indicates the discharge rate for all performed sets for each trial. For the learned hyperset (B), the neuronal activity rate was negligible except for the 1st trial.

For learned hypersets, the neuron showed minimal activity except for the first trial (Fig. 5B). In this case, behavioral correlates of such "first-trial activity" are unclear as the monkey made no error and the performance time was short from the first trial. In some cases, however, we observed that the performance time was longer in the first trial (when neural activity was present) than in the following trials (when neural activity was absent).

RESTORATION OF NEURONAL ACTIVITY BY PARTIAL CHANGES IN LEARNED SEQUENCES. Our previous study suggested that the monkey learned a hyperset as a single unique sequence (Rand et al. 1998). This suggests that, if part of a learned hyperset is changed, it would be regarded as a new hyperset. We expected, therefore, that a new-preferring cell then would become more active for the partially modified learned hyperset.

The neuron shown in Fig. 6 also was recorded in the pre-SMA and was determined to be new preferring (compare Fig. 6Aa and 6Ab). We then replaced parts of the learned hyperset (L-2, shown in Fig. 6Bb): first, sets 2 and 3 (Fig. 6Bc); second, sets 4 and 5 (Fig. 6Bd). The neuron now became more active for the modified hypersets (compare Fig. 6A, c and d with b). Interestingly, the neuronal activity also increased even for the unchanged parts of the modified hypersets (sets 1, 4, and 5 in Fig. 6Ac; sets 1-3 in Fig. 6Ad). The data suggest that the activity of new-preferring cell reflects the novelty of the whole sequence rather than the novelty of the individual sets. We examined 21 new-preferring cells in the same way (n = 11 for monkey BO, n = 10 for monkey GA), and all of them showed the same effect.


View larger version (48K):
[in this window]
[in a new window]
 
FIG. 6. Modulation of pre-SMA cell activity when parts of a learned hyperset were changed. A: this pre-SMA neuron was preferential for new hypersets as shown in a and b. When the stimuli of sets 2 and 3 of the learned hyperset were changed (c), the neuronal activity increased for the changed sets (sets 2 and 3) together with 2 adjacent sets (sets 1 and 4) and set 5 as well. When the sets 4 and 5 were changed (d), the neuronal activity also increased for sets 1-3. Sequences used are shown in B.

Neurons preferentially activated for learned sequences

A group of neurons showed preferential activity for learned hypersets (learned-preferring cells). Figure 7 shows one example. This neuron was recorded in the left SMA of monkey GA. A statistical analysis (Mann-Whitney U test) indicated that the activity of this neuron was significantly higher for learned hypersets than for new hypersets (Mann-Whitney U test P < 0.01).


View larger version (42K):
[in this window]
[in a new window]
 
FIG. 7. Activity of a left SMA cell that was stronger for learned hypersets (B) than for new hypersets (A). Data for each set were aligned at the time when the stimuli of each set turned on.

Learned-preferring cells tended to show differential activity for individual sets (as shown in Fig. 7), unlike new-preferring cells whose activation pattern tended to be similar between sets (activated from the stimulus onset to the first button press, as shown in Fig. 4).

Some of the learned-preferring cells (11 of 18) increased their activity as learning proceeded, as illustrated in Fig. 8. The monkey learned the same hyperset in two sequential blocks, and the increase in neuronal activity and the improvement of the performance occurred concurrently.


View larger version (18K):
[in this window]
[in a new window]
 
FIG. 8. Learning-dependent increase in SMA cell activity (A) in comparison with the monkey's performance (B: number of completed sets, C: performance time). Monkey performed 1 new hyperset in 2 consecutive blocks.

Distribution of new- and learned-preferring cells

Figure 9 shows the proportion of new- and learned-preferring cells separately for the pre-SMA and SMA for monkey BO and GA. The proportion of new-preferring cells was larger in the pre-SMA in both monkeys: pre-SMA, 22.4% (BO) and 39.4% (GA); SMA, 5.8% (BO) and 14.8% (GA) (chi 2 test, chi 2 = 19.9, df = 2, P < 0.0001). The proportion of learned-preferring cells was smaller, but there was some trend that learned-preferring cells were more abundant in the SMA: pre-SMA, 4.3% (BO) and 3.0% (GA); SMA, 8.7% (BO) and 6.6% (GA).


View larger version (32K):
[in this window]
[in a new window]
 
FIG. 9. Proportions of new-preferring (New > Learned) and learned-preferring cells (Learned > New) relative to the total number of task-related cells in the pre-SMA and SMA in 2 monkeys (BO and GA).

First-trial effect

As evident in the cell shown in Fig. 4, many of the new-preferring cells (n = 27 in monkey BO, n = 40 in monkey GA) showed vigorous activity for learned hypersets but only in the first trial (or even for only set 1). In contrast, only one learned-preferring cell (1 pre-SMA cell of monkey GA) showed this first-trial effect. Such cells were found mostly in the pre-SMA (85% for BO, 85% for GA). The data might suggest that the first-trial activity is related to retrieval of information from long-term memory. However, when a simple reaction task (reaching and pressing of one illuminated button, requiring no memory retrieval) was tested, many of the first-trial active cells (61%) again showed activity only for the first trial (data not shown).

Population data

In Fig. 10, we calculated population activity and learning curves separately for new-preferring cells (n = 25) and learned-preferring cells (n = 6). The data confirmed the findings described above. Activity of new-preferring cells was greater for new hypersets than for learned hypersets, according to the definition; the reverse is true for the learned preferring cells. However, new-preferring cells tended to be active in the first trial of learned hypersets (see First-trial effect) (analysis of variance, P < 0.0003). For new hypersets, activity of new-preferring cells decreased as learning proceeded, while activity of learned-preferring cells tended to increase.


View larger version (25K):
[in this window]
[in a new window]
 
FIG. 10. Population activity (a) and performance curves (b and c) for 25 new-preferring cells (A) and 6 learned-preferring cells (B) recorded in monkey GA. We calculated the population activity after normalizing the activity of a given cell to its maximal discharge per trial (which occurred either 1 of the hypersets tested). Trial number was normalized by aligning the activity and behavioral parameters at the 1st time when the monkey achieved a correct trial (indicated by trial number 0). Mean values of the number of completed sets were calculated and plotted for normalized trial numbers (b). Performance times were calculated only for the successful trials (c). Error bars: ±1 SE.

    DISCUSSION
Abstract
Introduction
Methods
Results
Discussion
References

Regional difference in learning-related function

In the present study, we found a regional difference within the medial frontal cortex, in terms of acquisition of sequential procedures. There was a clear trend that cells in the rostral part of the medial frontal cortex were activated more for new than learned hypersets; most neurons in the caudal part did not distinguish these kinds of hypersets. The rostral and caudal parts would correspond, respectively, to the pre-SMA and the SMA, the distinction proposed by Rizzolatti et al. (1990), Luppino et al. (1991), and Matsuzaka et al. (1992) according to the result of intracortical microstimulation.

Although we excluded the data for the cells that were recorded in the region where eye movements were evoked by electrical stimulation, there still remains the possibility that the area that we determined to be the pre-SMA contains some cells in the supplementary eye field (SEF). In fact, in monkey BO in which neurons were searched in wider areas, learned-preferring cells tended to be found even in the rostral part, but in the dorsal surface (perhaps including the SEF), rather than the medial wall.

Learning-related activity in pre-SMA neurons

The activity of the pre-SMA neurons differentiated new sequences from learned sequences. Furthermore, their activity for a new sequence tended to decrease as the monkey learned the new sequence. For individual pre-SMA neurons, a similar activity change was observed each time a new sequence was introduced which the monkey had to learn. A possible explanation would be that the pre-SMA cell activity is contingent on the way in which the hand moved: quick and continuous movements for learned sequences versus slow and discontinuous movements for new sequences. We think, however, that this possibility is unlikely. The fact that pre-SMA neurons were active regardless of the side of the performing hand suggests that they carry information remote from motor outputs. Further, the magnitude of neuronal activity does not appear directly related to the kinematic parameters of hand movements (Fig. 4).

The differential activity of pre-SMA neurons then would be related to learning itself. In a previous behavioral study (Hikosaka et al. 1995), we suggested that learning of the 2 × 5 task occurred at three levels: short-term and sequence-selective learning that occurred by repeating a particular hyperset during a block of experiment---our monkeys learned, to some degree, to perform a new hyperset within a several minutes; long-term and sequence-selective learning that took place for each hyperset across days---by daily practice, they further improved their skills for performing the particular hyperset; and long-term and sequence-unselective learning that was indicated by the improvement of performance for new hypersets---they performed gradually better with more experiences in the 2 × 5 task. These results, taken together, suggest that the pre-SMA is related to learning, especially short-term sequence-selective learning. This is supported by our experiment which showed that local inactivation of the pre-SMA led to selective deficits in learning of new sequences (Miyashita et al. 1996b).

However, there are several points that remain unsolved. It is unclear whether the activity of pre-SMA is related to the formation of sequence itself. Functional magnetic resonance imaging (fMRI) studies on human subjects from our laboratory have indicated that the pre-SMA became active during learning of sequential as well as nonsequential procedures (unpublished observation). It is also unclear whether the pre-SMA is related to visuomotor transformation, specifically, the transformation of information from visual to motor coordinates. Rizzolatti et al. found that the important variable for activation of reaching-grasping neurons in area 6abeta (which corresponds to pre-SMA) was whether the object was at a reachable distance, not physical characteristics (the size or the type of grip) nor location of the target object. In other words, area 6abeta does not seem to be related to the visuomotor transformation per se but could be involved in higher order recognition process (Rizzolatti et al. 1990).

What kind of information do pre-SMA neurons encode then? Many processes may not be related to sequence information but may be prerequisites for learning new sequences.

NOVELTY DETECTION. Learning is initiated when the subject encounters a new environment; it is unnecessary in a familiar environment. Therefore, it is very important to differentiate new from familiar environments, which would require the comparison between current sensory inputs and long- or short-term memories. In fact, many pre-SMA neurons behaved like a novelty detector: their activity decreased rapidly as the monkey started learning.

SELECTIVE ATTENTION TO VISUAL CUES. For the new hyperset, the monkey would move their eyes and hand in response to the visual instruction; for the learned hyperset, the eye and hand movements would be generated in a preprogrammed manner (Miyashita et al. 1996a). Thus the learning-related decrease in pre-SMA cell activity might reflect the decrease in the monkey's attention to the visual instruction. In fact, Matsuzaka et al. (1992) showed that cells with visual response were more abundant in the pre-SMA than in the SMA. We also found that a considerable portion of new-preferring cells were active in the simple reaction task, which required attention but not learning.

DECISION MAKING. In the initial stage of new learning, the monkeys had to choose and press one of the two illuminated buttons. No information was given as to which button was more likely to be correct, and such an uncertain situation was present for each set. Consistent with this idea, many pre-SMA neurons discharged before the first button press for each set when the animal had to make a decision (Fig. 4). This aspect would be supported by human imaging studies (Deiber et al. 1991) showing that the human medial frontal area that would correspond to the pre-SMA becomes active when the subject had to choose one out of four movements voluntarily.

ERROR DETECTION. The detection of errors is crucial in trial-and-error learning. During new learning of a hyperset, the errors occurred at earlier trials and therefore its frequency was high in the early stage, which would correspond to the learning-related decrease in pre-SMA cell activity. We found, however, few neurons in the pre-SMA that fired selectively after errors.

MEMORY ENCODING AND RETRIEVAL. During learning, the sequence information must be maintained as a memory and at the same time must be retrieved as a motor command. As discussed earlier, the memory in this case would be a short-term one. Human imaging studies have shown that the area corresponding to the pre-SMA was activated when encoding and retrieval of short-term memory is required (Buckner et al. 1996; Fiez et al. 1996; Paulesu et al. 1993). The well-documented inputs to the pre-SMA from the dorsolateral prefrontal cortex (Bates and Goldman-Rakic 1993) might provide the short-term working memory signals.

SHIFT OF MOTOR PLAN. Once an error is detected, the monkey had to shift or change the motor plan in the next trial. Pre-SMA neurons might be related to this process, as they responded to a sensory signal that instructed the monkey to change the ongoing action to a new one (Matsuzaka et al. 1996; Shima et al. 1996). This result is consistent with our finding that the same pre-SMA neurons became active only at the very first trial of a learned hyperset, at which the monkey was required to update the procedure, that is, to discard the previous sequence and set the new sequence.

AROUSAL. The "habituation"-like behavior of pre-SMA neurons would raise the possibility that they encode "arousal" or "vigilance" (Thompson and Spencer 1966; Vinogradova and Sokolov 1975). However, the learning-related activation was relatively localized in the pre-SMA, as shown in this study and the human fMRI study (Hikosaka et al. 1996), which is inconsistent with the view that arousal is the general increase in the level of brain activity.

To summarize, the activity of pre-SMA neurons may reflect these cognitive processes, which would be tightly related with each other for the acquisition of new sequences in our task. We further speculate that the pre-SMA controls the conditional or spatial visuomotor transformation that is carried out by the dorsal (Mitz et al. 1991) as well as ventral (Rizzolatti et al. 1988) premotor areas. In this way, the pre-SMA would control, rather than execute, motor programs. A similar idea has been proposed by Rizzolatti et al. (1996). Such a control mechanism would allow efficient learning of a sequential procedure because the performance of the procedure initially is dependent on sensory information but eventually becomes automatic (Anderson 1982).

Role of SMA in learning and memory

Many studies have shown that the SMA is related to sequential movements. In the monkey SMA, Tanji and his colleagues found neurons that became active selectively before a particular sequence of movements or at a particular transition of movements; the result implies that the information for learned motor sequences is stored in the SMA (Tanji and Shima 1994). Different lines of research in human subjects would support this view. Impairment of alternating hand movements is an enduring sign after the lesion of the medial frontal cortex including the SMA (Laplane et al. 1977).

Functional imaging studies on human subjects also suggest that the SMA is a storage site of movement skill (Grafton et al. 1992, 1994, 1995; van Mier and Petersen 1996) or sequence (Jenkins et al. 1994; Seitz and Roland 1992). The SMA is activated by execution of mental imagery of sequential movements (Roland et al. 1980). Using a trial-and-error sequential movement task, it was shown that activation of the SMA was higher when prelearned sequences were performed than when new sequences were performed (Jenkins et al. 1994). van Mier and Petersen (1996) also reached the same conclusion using a maze task.

We expected to obtain similar results, but our results were unclear. SMA cells usually did not differentiate between new and learned sequences. Furthermore, there was no indication for the dominance of new- or learned-preferring cells. The present result was consistent with the result of the fMRI study (Hikosaka et al. 1996) in which activation of the SMA, unlike the pre-SMA, was related to sensorimotor processes, not learning processes.

There are at least two possibilities that might account for the difference. First, learned sequences in our study may have been so well learned that the memory for the sequences was stored somewhere else. In fact, Aizawa et al. (1991) showed that, after extensive practice of a motor task, SMA neurons, which were previously active, no longer showed task-related activity, while M1 neurons did (Aizawa et al. 1991). Although the monkeys in the study of Tanji and Shima (1994) were well trained for performing learned sequences, the speed of the performance was controlled, unlike our experiments. If such fast movement sequences are controlled by other brain areas, such as the cerebellum, as suggested by another study from our laboratory (Lu et al. 1998), the role of the SMA may be less important as a memory site.

The second possibility is that the inconsistency may be due to the difference in the kinds of sensory stimuli. In the studies of Jenkins et al. (1994) and van Mier and Petersen (1996), the subjects closed their eyes, whereas our study was strongly dependent on visual stimuli used. The results of the study by Mushiake et al. (1991) are consistent with this idea. In their study, SMA neurons were usually inactive when the monkey was ready to perform a motor sequence according to explicit instructions, but they may be active when the performance was based on memory.

Relation to other animal studies

Learning-dependent changes in neural activity have been shown in the premotor cortex (Germain and Lamarre 1993; Mitz et al. 1991) and the supplementary eye field (Chen and Wise 1995a,b). Chen et al. found that some of neurons in the SEF changed their activity while the monkey learned to associate a new picture with a particular direction of saccade. Although the task used by Chen and Wise and the task we used may appear dissimilar, they may contain common aspects in that the monkey had to associate a visual stimulus with a particular movement pattern. A learning-related decrease in neural activity also has been found in the orbito-frontal cortex (Tremblay and Schultz 1996).

The tight relationship between response decrement (habituation) and memory formation also has been suggested in the inferior parietal cortex of monkeys (Miller et al. 1993) and the caudo-neostriatum of birds (Chew et al. 1995). A critical question remains unsolved how such a habituation-like decrement of neural activity occurs. Neurons in the monkey pre-SMA and other areas listed above were in general broadly tuned to the stimuli or sequences, and yet the decrement of activity occurred selectively to the stimuli or sequences that have been experienced.

Relation to human studies

We also applied the same learning paradigm to a human functional MRI study, with slight modification of task procedures (Hikosaka et al. 1996; Sakai et al. 1998). The results were consistent with the conclusion in the present study. We found learning-related activation that was localized in a small region anterior to the SMA, which we regarded as the human homologue of the pre-SMA. In contrast, the human SMA was activated with sensorimotor processes rather than with sequence learning. Furthermore, the activation of the human pre-SMA decreased as the subject learned a visuomotor sequence. These results are virtually the same as the results obtained in monkeys in the present study, suggesting that humans and monkeys share the same learning mechanism in the pre-SMA.

A similar conclusion may be drawn from other human PET studies, although many of them did not identify the rostral portion of the SMA as the pre-SMA. The area corresponding to the pre-SMA (see Picard and Strick 1996) showed a rCBF (cerebral blood flow) increase when the pianists played an unfamiliar musical piece, but not practiced piece (Sergent et al. 1992, 1993). This area has shown increased activity when subjects performed remembered sequential saccades immediately after viewing the sequence (Petit et al. 1996). Additionally, the pre-SMA activity was shown to decrease when subjects had learned to perform a sequential finger-to-thumb opposition task (Friston et al. 1992).

However, these human imaging studies did not allow us to examine changes of neural activity in fine time resolution that were demonstrated in the present study. Further, there is no easy way, if human subjects are used, to examine whether the learning-related activation is necessary for the learning. Animal experiments are critical in this sense by which we could show that learning of sequential procedures was disrupted by local inactivation of the medial frontal cortex (Miyashita et al. 1996b).

Parallel mechanisms for learning of new sequences

Other studies from our laboratory have indicated that different brain regions other than the medial frontal cortex also are related to learning of sequential procedures. Muscimol injections in the anterior part of the striatum disrupted learning of new sequences, not performance of learned sequences (Miyachi et al. 1997). The pre-SMA is known to project to the anterior part of the striatum (Parthasarathy et al. 1992), which presumably sends information back to the pre-SMA. A functional MRI study on human subjects, using the same paradigm (Sakai et al. 1998), indicated that the pre-SMA becomes active during learning of new sequences together with dorsolateral prefrontal cortex, precuneus (medial part of parietal cortex), and intraparietal cortex. Thus multiple brain regions, including the frontal and parietal cortex and basal ganglia, appear to contribute to the acquisition of new sequences in parallel.

    ACKNOWLEDGEMENTS

  We are grateful to Dr. M. Kato for designing the computer programs. We thank Drs. Longtang Chen, Jeremy Goodridge, and Carol Colby for encouraging and extensive comments on our manuscripts. We also thank Dr. J. Tanji and his collaborators for many important pieces of advice for the experiment.

  This study was supported by the Uehara Memorial Foundation, a Grant-in-Aid for Scientific Research on Priority Areas from The Ministry of Education, Science and Culture of Japan, and The Japan Society for the Promotion of Science (JSPS) Research for the Future program.

    FOOTNOTES

   Present address of K. Nakamura: Center for the Neural Basis of Cognition, Dept. of Neuroscience, University of Pittsburgh, 115 Mellon Institute, 4400 Fifth Ave., Pittsburgh, PA 15213-2683.

  Address for reprint requests: O. Hikosaka, Dept. of Physiology, Juntendo University, School of Medicine, 2-1-1 Hongo, Bunkyo-ku, Tokyo 113, Japan.

  Received 8 April 1998; accepted in final form 5 August 1998.

    REFERENCES
Abstract
Introduction
Methods
Results
Discussion
References

0022-3077/98 $5.00 Copyright ©1998 The American Physiological Society