Department of Psychology, University of Colorado, Boulder, CO, , 1 Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA, , 2 Department of Psychology, Washington University, St Louis, MO and , 3 Department of Psychology, Princeton University, Princeton, NJ, USA
Randall C. O'Reilly, Department of Psychology, University of Colorado Boulder, 345 UCB, Boulder, CO 80309, USA. Email: oreilly{at}psych.colorado.edu
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The ID/ED task (Roberts et al., 1988; Owen et al., 1993
; Dias et al., 1997
) is a refinement of the widely studied Wisconsin card sorting task (WCST). Dynamic categorization tasks like these involve the periodic switching of response rules, with each rule in the ID/ED task defined as responding to a target stimulus in the context of several other stimuli. The ID/ED task involves two kinds of switches intradimensional (ID) and extradimensional (ED). A dimension in this context represents a category of stimuli sharing the same general set of features (e.g. color, shape). For example, shapes constructed from homogeneous regions of color are considered one dimension, while shapes constructed from solid black lines are considered another dimension. An ID switch therefore involves changing the target stimulus to another within the same dimensional category (e.g. from one colored shape to another), while an ED switch involves changing the target stimulus to one from a different dimension (e.g. from a color shape to a solid black line figure). Dias et al. (Dias et al., 1997
) showed that orbital frontal lesions selectively impaired a particular type of ID switches, while dorsolateral frontal lesions selectively impaired specific ED switches.
These data clearly have the potential to inform the issue of how the PFC is organized, given that these two different frontal areas appear to be selectively involved in different aspects of the same task. One can categorize the existing proposals for understanding the organization of frontal cortex into at least two groups. One group, exemplified by Goldman-Rakic (Goldman-Rakic 1987), suggests that different areas of frontal cortex encode different kinds of representational content (e.g. spatial versus object representations), while performing the same kind of essential processing function (e.g. working memory). Another suggests that different frontal areas contribute qualitatively different processing functions [e.g. inhibition versus working memory (Fuster, 1989
; Diamond, 1990
) or simple maintenance versus complex processing (Petrides, 1994
)]. Dias et al. (Dias et al., 1997
) interpreted their data as supporting a differential-processing model involving affective inhibition (in the orbital region) and attentional selection (in the dorsolateral region). In contrast, our model demonstrates that these findings are consistent with a content-based organization (with a common processing function of working memory across different areas).
Thus, our model shares the same basic approach as the working memory model of Goldman-Rakic (Goldman-Rakic, 1987), in that we view the single essential function of frontal cortex as that of maintaining information in an active state over time (i.e. activation-based working memory). Indeed, we have developed a set of biologically based computational mechanisms for understanding how the PFC is specialized for this active maintenance function (Cohen et al., 1996
; O'Reilly et al., 1999
; Braver and Cohen, 2000
; Frank et al., 2001
). However, our model reflects a somewhat different view regarding the specific organization of representations within PFC. We take the view that an important principle of organization may be the level of abstraction of representations, rather than the specific sensory modality or domain of the information represented. Thus, some areas may be responsible for representing specific featural information, while others are responsible for representing more abstract categories (e.g. featural dimensions) [see (Koechlin et al., 1999
; Christoff and Gabrieli, 2000
) for related ideas].
![]() |
The ID/ED Dynamic Categorization Task |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
In the Dias et al. study (Dias et al., 1997), marmosets with lesions in either dorsolateral or orbital (ventromedial) frontal cortex were tested on three of these changes: IDR, IDS and EDS. They found selective impairment of IDRs with orbital lesions, selective impairment of EDS with dorsolateral lesions, and no significant impairments on IDSs with either frontal lesion (Fig. 3
).
|
It suggests that distinct regions of the prefrontal cortex carry out independent but complementary forms of cognitive processing of complex visual stimuli in changing environmental circumstances. Thus, regions within the orbital prefrontal cortex in marmosets enable the rapid reversal of affective associations for specific visual stimuli, whereas the higher-order shifting of attention between supraordinate features of visual stimuli, such as their perceptual dimensions, is mediated by regions within the lateral prefrontal cortex. (p. 9296).
As we noted in the Introduction, this way of characterizing the different frontal areas is consistent with functionally based theories (Fuster, 1989; Diamond, 1990
; Petrides, 1994
), although it also does assume a representational distinction between specific visual stimuli and supraordinate features. Dias et al. (Dias et al., 1997
) argued that their results were inconsistent with a working memory (on-line processing) account, because the apparent memory demands across the different tasks (IDR, IDS, EDS) are all roughly equivalent, whereas deficits were observed only when inhibition or higher-order dimensional shifting was required. We argue that this analysis is based on a simplified notion of working memory that suggests it is only important for the raw maintenance of information. In contrast, we think that the maintenance of information in frontal cortex, when combined with representational distinctions along the lines suggested by Dias et al. (Dias et al., 1997
), has important implications for processing elsewhere in the system, in ways that can explain the Dias et al. pattern of results.
![]() |
A Theory of Perseverations in the ID/ED Task |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We argue that this kind of flexibility is essential for rapid switching in dynamic categorization tasks. Specifically, we think the PFC maintains a representation of the currently relevant dimension or feature in activation-based working memory, and that this activation provides top-down support or biasing (Cohen and O'Reilly, 1996) of the corresponding perceptual processing and action selection pathways, facilitating sorting along this dimension or feature. Categorization behavior may be altered either as a result of weight changes in these modulated pathways, or through changes in the activation-based biasing provided by the working memory system. While the active representation can be relatively rapidly switched when the sorting rules change, a weight-based solution must slowly unlearn the previous weights and learn the new ones. Thus, the perseveration observed in the ID/ED and WCST tasks can be accounted for by the loss of the more flexible, prefrontally mediated activation-based working memory, causing behavior to depend solely on the less flexible weight-based learning supported by posterior cortex and other brain areas [see (Munakata, 1998
; Munakata et al., 2001
) for similar ideas].
Importantly, the effects of frontal damage are only evident when there are prepotent responses that must be overcome this is when the frontal top-down support of the new target (dimension or feature) is critical for overcoming the strength of the prior target. Such prepotent responses are present only after the first rule is learned in WCST, and in ID/ED only in the conditions of IDR and EDS (which involves a reversal of the relevant dimension, as noted previously), but not IDS (which involves all new stimuli and no reversal of the relevant dimension). The fact that activation-based top-down support interacts with the strength of existing associations means that an assessment of the role of working memory in this task based on the idea that it only maintains information [as provided by Dias et al. (Dias et al., 1997)] is incomplete. Maintained representations influence processing elsewhere in the brain, and this influence is felt in some conditions more than others, depending on the strength of learned associations in those other areas. Although it may be intuitively appealing to describe such conditions as requiring inhibition of prepotent responses, we can provide a more parsimonious overall account of frontal function by thinking instead in terms of sustained activations supporting weaker responses (Cohen and O'Reilly, 1996
; Munakata, 1998
; O'Reilly and Munakata, 2000
; Miller and Cohen, 2001
) [cf. the biased-competition model of Desimone and Duncan (Desimone and Duncan, 1995
)].
The flexibility conferred by PFC is specific to the representational content of PFC areas. We suggest that these areas are organized along a gradient of abstraction, such that: (i) orbital frontal cortex in the marmoset is particularly important for supporting feature-level representations, so that lesions here impair reversals at the feature level (IDR); and (ii) dorsolateral areas in the marmoset support dimension-level representations, so that lesions here impair reversals at the dimension level (EDS). As noted, these representational distinctions were also assumed by Dias et al. (Dias et al., 1997). We discuss possible extensions of this account to other primate species in the Discussion.
The Dynamic Gating Mechanism
The flexibility of activation-based working memories depends critically on the presence of a dynamic gating mechanism, which controls the updating and maintenance of working memory representations. When the gate is open, working memory can be updated, and when it is closed, any currently active working memories are protected from interference (from noise, ongoing processing, irrelevant stimuli, etc.). This gate is needed because one setting of connection strengths into the working memory system cannot satisfy both the need for rapid updating and robust maintenance (Cohen et al., 1996; O'Reilly et al., 1999
; Braver and Cohen, 2000
; O'Reilly and Munakata, 2000
). Biologically, we have shown how this gating mechanism can be implemented either through the phasic dopamine neuromodulation of the frontal cortex by the ventral tegmental area (VTA) (Durstewitz et al., 1999
; Braver and Cohen, 2000
; O'Reilly and Munakata, 2000
) (which is used in the present model), or through the interactions between the basal ganglia and frontal cortex (Frank et al., 2001
). In either case, the updating properties of the gating mechanism are shaped by a reinforcement-based learning mechanism, which plays a critical role in the present model by triggering the updating of working memory representations when the categorization rule changes.
Basic Properties of the Model
Figure 4 shows the structure of the model, which is implemented using the Leabra framework (O'Reilly, 1998
, 2001
; O'Reilly and Munakata, 2000
) (see Appendix). Leabra integrates a number of widely used neural network mechanisms into one coherent framework, and has been used to simulate over 40 different psychological phenomena in O'Reilly and Munakata (O'Reilly and Munakata, 2000
). Thus, all the basic mechanisms in the model have strong independent motivation; we note below where specific features of the algorithm play an important role.
|
The PFC is organized into two areas: orbital areas that represent featural information, and lateral areas that represent more abstract dimension-level information. These areas are reciprocally interconnected with the posterior cortex units, and their activity thus biases the posterior cortex units. The featural nature of the orbital representations is accomplished by having the individual PFC units connect in a one-to-one fashion with the featural units within the two dimensions. We only included one set of four such featural units per dimension so that this area does not encode the location of the features, only their identity. The dimensional nature of the lateral representations comes from the fact that there are only two units in this layer, one for each dimension, with each unit fully connected with the feature units in the posterior cortex layer from the corresponding dimension. Throughout, the network connectivity generally obeys the principle of bidirectional cortical connections (Felleman and Van Essen, 1991; O'Reilly and Munakata, 2000
). The exceptions are the VTA which projects to the PFC (see below), the dimensional PFC which projects to the featural PFC but not vice versa, in accord with the ideas and data reviewed by Gobbel (Gobbel, 1997
) [see also Frank et al. (Frank et al., 2001
)], and the input layer which is fixed and therefore does not receive backprojections.
During initial learning, the network has no difficulty activating the target item representation because all posterior cortex units are roughly equally likely to get activated, and the correct item will get reinforced through learning (both the Hebbian and error-driven learning mechanisms in Leabra will cause the target item representation to become stronger). However, if the target is then switched to one that was previously irrelevant (i.e. a reversal), then the irrelevant item will not tend to be activated in the posterior cortex layer, making it difficult to learn the new association. The top-down PFC biasing can overcome this problem by supporting the activation of the new target item, giving it a competitive edge in the limited activation competition.
The VTA layer represents the ventral tegmental area, which provides a dynamic gating mechanism via dopamine neuro-modulation to the PFC. It has been shown that the VTA fires dopamine bursts for stimuli that are predictive of reward (Schultz et al., 1993), in a way that is generally consistent with the temporal differences reinforcement learning mechanism (Sutton, 1988
; Montague et al., 1996
). If rewards are expected but not delivered (i.e. due to a behavioral error), the dopamine neurons exhibit reduced firing, corresponding to a negative error signal. We reasoned that task-relevant information that should be maintained is a reliable predictor of reward, and should thus elicit dopamine firing, resulting in the updating of working memory (O'Reilly et al., 1999
; Braver and Cohen, 2000
), and that the negative error signal should reset working memory representations. The net effect is to produce a form of trial-and-error search by activating and deactivating PFC representations.
The VTA unit directly modulates the strength of weights in the PFC according to changes in its activity. When VTA transitions from not expecting reward to getting a reward (0 to 1), the weights from the posterior cortex units to the PFC transiently increase, thereby encoding the current pattern of posterior cortex activity. If an error is made after correct performance, the negative change in expected reward (1 to 0) causes the PFC gating to decrease significantly in strength, including the gain on the recurrent self-maintenance weights. This effectively clears the PFC activations. If there is no change in the VTA activation, the PFC will maintain its current values. There is also noise so random activation of PFC can occur, especially when there is nothing already in the PFC. Detailed equations are presented in the Appendix.
The model was trained and tested following the procedures of Dias et al. (Dias et al., 1997). The network received two blocks of training followed by the rule change. The first block had only features from one dimension (one of which was the target), and the second block added the two features from the other dimension (but the target remained the same). Each block was trained until a criterion of two epochs (passes through all training items) without error. In the third block, the categorization rule changed as an IDS, IDR, EDS or EDR, and we measured the number of epochs needed to learn the new rule. Three types of networks were run: intact, feature-level (orbital PFC) lesion, and dimension-level (lateral PFC) lesion. Lesions were implemented by effectively removing the corresponding PFC layer. Ten different networks with random initial weights were run.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Note that even though different orbital PFC units encode different dimensions (Fig. 4b), and could therefore help to switch to another dimension, there are two properties that work against this. First, as shown in Figure 4b
, the dimensional units project reciprocally to the entire set of features within a dimension, and are thus much more effective at switching across dimensions than the featural representations that only project to a small subset of a dimension. In short, any different activation in the dimensional layer will produce a switch to a new dimension, whereas many different activations in the featural layer will not shift across dimensions. Second, the overlap among features within a given dimension produces a bias towards activating other featural PFC units within the same dimension.
To illustrate this explanation in specific cases, Figure 6a shows the state of the network after initial target acquisition. The target is the triangle filled shape, and this is what is activated in posterior cortex and the orbital PFC, with the shape dimension active in lateral PFC. Then, in the IDR condition (Fig. 6b
), the input stimuli are the same, but the network's initial responses (based on the old triangle target) are wrong, which rapidly destabilizes the PFC representations. When the PFC activates the other shape representation (squarish shape), this provides top-down support for the corresponding representation in posterior cortex, which then facilitates shifting the responding to be based on this target. In the EDS condition (Fig. 6c
), the input stimuli change to all new items, and responding is initially random but still focused on the old, wrong dimension (shape instead of lines). As errors occur, the PFC representations are destabilized, and when the new dimension (lines) is activated in lateral PFC, this provides top-down support to that dimension in posterior cortex, facilitating learning of the correct output mapping. The detailed time course of this lateral PFC-mediated switching is shown for one EDS run in Figure 7
, where it is clear that the dimensional unit switching leads to switching in the other layers.
|
|
The results of the EDR condition constitute an interesting prediction, showing that only dimensional (lateral) PFC lesions cause impairments, despite the fact that it is like the IDR condition in having the same stimuli present. The notion that orbital PFC is responsible for inhibiting affective associations associated with stimuli, as proposed by Dias et al. (Dias et al., 1997), would appear to predict that orbital PFC lesions would impair performance in this EDR condition. Instead, we suggest that because the dimensional PFC units do not encode featural level details anyway, they will be unaffected by having the same stimuli present. Thus, their role in switching in EDR should be similar to that in EDS, as is suggested by the model results.
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
At a more general level, this model provides an important step toward characterizing the kinds of control mechanisms that enable activation-based working memory to confer greater flexibility as compared to weight-based adaptations. Although the activation-based working memory framework instantiated in our model is consistent with other active or working memory ideas (e.g. Goldman-Rakic, 1987), it extends these notions by accounting for empirical data that implicates the frontal cortex in overcoming perseverations through greater dynamic flexibility.
There are other important advantages of activation-based working memories (O'Reilly and Munakata, 2000). Specifically, working memory is useful because it can be rapidly updated to reflect the ongoing products and demands of processing, and it is generally consciously accessible and can be described in a verbal protocol (Miyake and Shah, 1999
). Furthermore, as exploited in the present model, the active nature of working memory provides a natural mechanism for cognitive control (or task-based attention), where top-down activation can influence processing elsewhere to achieve task-relevant objectives (Cohen et al., 1990
; Cohen and O'Reilly, 1996
; O'Reilly et al., 1999
). Thus, working memory and cognitive control can be seen as two different sides of the same coin of actively maintained information.
However, these advantages of activation-based working memories also have concomitant disadvantages (O'Reilly and Munakata, 2000). For example, because these memories do not involve structural changes, they are transient, and therefore do not provide a suitable basis for long-term memories. Also, because information is encoded by the activation states of neurons, the capacity of these memories scales as a function of the number of neurons, whereas the capacity of weight-based memories scales as a function of the number of synaptic connections, which is much larger.
Because of this fundamental tradeoff between activation-and weight-based memory mechanisms, it makes sense that the brain would have evolved two different specialized systems to obtain the best of both types of memory. This is particularly true if there are specific mechanistic specializations that are needed to make each type of memory work better for example the dynamic gating mechanisms needed for rapid updating and robust maintenance in frontally mediated activation-based working memory. This type of tradeoff-based reasoning has also been exploited in the context of weight-based long-term episodic memory (O'Reilly and Rudy, 2001).
Our model also provides support for the general principle that different areas of frontal cortex might be organized according to different levels of abstractness. In the marmoset, we argued that this organization exists along the ventral (more specific) to lateral (more abstract) axis. However, these organizational axes may differ in other species. For example, it is possible that the lateralventral axis in monkeys is more closely aligned with the anteriorposterior axis in humans. Therefore, we want to emphasize instead the general principle that different frontal areas may be organized according to different levels of abstraction, without making specific claims as to the correspondence of the marmoset brain areas with those in other primates. With this idea in mind, we can explore how well abstraction might align with other interpretations of frontal organization in more familiar primate species.
In the rhesus macaque monkey, a number of neural recording studies suggest that more ventral areas encode more specific object or pattern information (Mishkin and Manning, 1978; Wilson et al., 1993
). Other researchers have hypothesized, based on a variety of data, that the dorsolateral areas in humans are involved in more complex, abstract processing, whereas the ventral areas are used for simpler memory processes that require maintaining specific information (Petrides, 1994
). In contrast to this generally dorsalventral organization, others have suggested an anteriorposterior organization where anterior prefrontal areas have the most temporally-extended representations (Koechlin et al., 1999
). We can interpret this temporal extension as a form of abstraction (Frank et al., 2001
; O'Reilly et al., 1999
), where more anterior/dorsal frontal areas could encode information that is relevant over longer delays (i.e. more abstract goals or plans that encompass many subgoals or subplans), while more posterior/ventral areas encode information relevant over briefer intervals. A major project for future work is to explore how much of this other data can be accounted for using explicit computational models incorporating the principle of differing levels of abstraction.
There are other aspects of the Dias et al. data (Dias et al., 1997) that the model cannot address in its present form. Specifically, the monkeys were also tested on additional reversals and shifts, with the result that the second reversal did not produce the same patterns of frontal deficits as the first. We suspect that this result emerges as a result of the posterior system establishing a more equal balance among the representations involved, and thus do not consider it a challenge to the basic principles captured in our present model. Simulating this data would require a more complex network capable of representing at least 16 different stimuli, whereas our current model only handles four.
More generally, there are a number of scaling issues raised by the model we used a small number of units to simulate a complex phenomenon that likely involves millions of neurons in the monkey brain. This is one of a number of challenges that are often leveled at neural network models of this sort, and a full discussion is beyond the scope of this paper [see O'Reilly and Munakata (O'Reilly and Munakata, 2000) for one such discussion]. One of the most important ways of addressing such concerns is to ask, will the fundamental principles behind the model's behavior change with scaling?. In this case, we think not the most basic principles of the differences between weight-versus activation-based memories should not depend critically on scaling parameters, and we can understand the model's behavior in terms of these principles. Thus, we must be careful to think of the model as just one possible concrete implementation of more general principles.
There are various aspects of the model that could be improved with future research. For example, we have hand-coded the frontal representations in this model (which therefore do not have learned connections as shown in Fig. 4), but it should be possible for these representations to develop naturally through learning in response to a combination of initial architectural constraints and task demands. Demonstrating this kind of learning is important for avoiding hidden homunculi in the model, and is therefore a topic of active research in our group.
Predictions
The model and the broader theory in which it is framed make a number of testable predictions. For example, we simulated the EDR condition in the model [which was not run by Dias et al. (Dias et al., 1997)], and found that only lateral (dimensional) PFC lesions impaired performance on this condition. Orbital (featural) PFC lesions had no apparent effect, despite the fact that this condition involves all the same stimuli, and therefore appears to require inhibiting affective associations with the previous target item [which is how Dias et al. (Dias et al., 1997
) interpreted the role of the orbital PFC]. We interpreted this finding as showing that for the network, the dimensional switching represents the dominant difficulty for the EDR case, and that because the dimensional PFC units are relatively abstract anyway, their ability to switch is relatively unaffected by featural-level changes. This finding thus represents a testable prediction from the model, and one that appears to distinguish it from the predictions that Dias et al. (Dias et al., 1997
) would make.
At a more general level, the overall framework behind the model makes a number of predictions. For example, we predict that electrophysiological recordings would reveal differences in the extent to which neurons in different areas of the PFC exhibit abstract, categorical representations of stimuli. This could be tested by using an experiment similar to that performed by Freedman et al., who have shown that prefrontal neurons in the macaque encode abstract categories of stimuli such as cats versus dogs (Freedman et al., 2001).
Similarly, neuroimaging studies in monkeys or humans could be used to test our ideas. For example, as mentioned above, we have suggested that neuroimaging studies showing activation of more anterior PFC areas in humans (Koechlin et al., 1999; Christoff and Gabrieli, 2000
) can be interpreted as reflecting an organization according to different levels of abstraction. Assuming such an organization, our framework would predict that one might be able to find evidence of a anteriorposterior organization for IDR versus EDS activation in the ID/ED task in humans. Indeed, one such study found suggestive evidence consistent with this prediction (Rogers et al., 2000
). They found that when EDS and IDR were directly compared, there was more dorsal/anterior PFC activation during EDS, but no more ventral PFC activity during IDR. The lack of a ventral difference for IDR versus EDS suggests that these regions were active for both IDR and EDS, which is consistent with their non-shifting control comparison as well.
Other Models
Although we are not aware of any other computational models of the ID/ED task specifically, there are several published models that share some features with the present model. First, the hypothesized role of frontal cortex in our model is very similar to that proposed by Cohen et al. in their model of the Stroop task (Cohen et al., 1990). They suggested that frontal cortex is specifically important for overcoming the prepotent process of word reading to support the weaker process of color naming. We have subsequently generalized this Stroop model in terms of top-down biasing as a mechanism by which the frontal cortex contributes to cognitive control or controlled processing (Cohen and O'Reilly, 1996
; Miller and Cohen, 2001
). A key feature of dynamic categorization tasks such as ID/ED is that frontal representations need to be rapidly updated when the rules change to provide useful top-down activation-based support, whereas in the Stroop model the frontal representations were externally specified (e.g. through task instructions). Thus, the present model extends the Stroop model by demonstrating how dynamic gating mechanisms for activation-based working memory can provide rapidly switched, task-appropriate top-down biasing.
The Dehaene and Changeux model (Dehaene and Changeux, 1991) of the WCST has some similarities to our own, in that it is based on an error-driven search mechanism. However, a critical difference is that their model relies on weight-based learning and unlearning that is modulated by the error signals, and is not fundamentally a top-down biasing based model like the one presented here. Specifically, when an error occurs, negative weights are incremented to prevent the return to a previously unsuccessful sorting rule. Indeed, despite the presence of sustained activation memory units in their model, their primary memory manipulation involved changing the decay parameter of these negative weights. To simulate a frontal lesion, they changed the strength of weights into the error unit that is responsible for setting these negative weights, instead of damaging their sustained-firing memory units. Thus, the Dehaene and Changeux model (Dehaene and Changeux, 1991
) makes very different mechanistic assumptions, and is less closely tied to specific brain areas, than the model presented here.
Levine and Prueitt (Levine and Prueitt, 1989) also presented a WCST model that is generally similar to the Dehaene and Changeux (Dehaene and Changeux, 1991
) model, but lacks the decay function on the error-driven negative weights and some other features. Moving outside the realm of neural network models, Kimberg and Farah presented a production-system framework that accounted for a range of frontal deficits, in-cluding perseveration on the WCST (Kimberg and Farah, 1993
). The essence of the model is that frontal damage reduces the influence of specific information on production firing, such that the productions end up falling back on perseverative and noisy firing biases that operate in the absence of other specific information. Thus, they build in perseveration as the behavior that the model resorts to after a frontal lesion. In contrast, we see perseveration as a result of learning in the weight-based processing of the posterior cortex. Nonetheless, this paper makes a number of more general points that resonate well with the framework presented here. For example, Kimberg and Farah emphasize the idea that frontal cortex can be understood as performing a single function, that, when damaged, produces a range of different behavioral manifestations (Kimberg and Farah, 1993
). Furthermore, this common frontal function has something generally to do with working memory, which is consistent with our framework.
![]() |
Notes |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Appendix Implementational Details |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
The pseudocode for Leabra is given here, showing exactly how the pieces of the algorithm described in more detail in the subsequent sections fit together.
Outer loop: Iterate over events (trials) within an epoch. For each event:
Point Neuron Activation Function
Leabra uses a point neuron activation function that models the electrophysiological properties of real neurons, while simplifying their geometry to a single point. This function is nearly as simple computationally as the standard sigmoidal activation function, but the more biologically based implementation makes it considerably easier to model inhibitory competition, as described below. Further, using this function enables cognitive models to be more easily related to more physiologically detailed simulations, thereby facilitating bridge-building between biology and cognition.
The membrane potential Vm is updated as a function of ionic conductances g with reversal (driving) potentials E as follows:
![]() | (1) |
![]() | (2) |
The excitatory net input/conductance ge(t) or eta;j is computed as the proportion of open excitatory channels as a function of sending activations times the weight values:
![]() | (3) |
Activation communicated to other cells (yj) is a thresholded () sigmoidal function of the membrane potential with gain parameter
:
| (4) |
![]() | (5) |
k-Winners-Take-All Inhibition
Leabra uses a kWTA function to achieve inhibitory competition among units within a layer (area). The kWTA function computes a uniform level of inhibitory current for all units in the layer, such that the k + 1th most excited unit within a layer is below its firing threshold, while the kth is above threshold. Activation dynamics similar to those produced by the kWTA function have been shown to result from simulated inhibitory interneurons that project both feedforward and feedback inhibition (O'Reilly and Munakata, 2000). Thus, although the kWTA function is somewhat biologically implausible in its implementation (e.g. requiring global information about activation states and using sorting mechanisms), it provides a computationally effective approximation to biologically plausible inhibitory dynamics.
kWTA is computed via a uniform level of inhibitory current for all units in the layer as follows:
![]() | (6) |
![]() | (7) |
Hebbian and Error-Driven Learning
For learning, Leabra uses a combination of error-driven and Hebbian learning. The error-driven component is the symmetric midpoint version of the GeneRec algorithm (O'Reilly, 1996), which is functionally equivalent to the deterministic Boltzmann machine and contrastive Hebbian learning (CHL). The network settles in two phases: an expectation (minus) phase where the network's actual output is produced, and an outcome (plus) phase where the target output is experienced, and then computes a simple difference of a pre and postsynaptic activation product across these two phases. For Hebbian learning, Leabra uses essentially the same learning rule used in competitive learning or mixtures-of-Gaussians which can be seen as a variant of the Oja normalization. The error-driven and Hebbian learning components are combined additively at each connection to produce a net weight change.
The equation for the Hebbian weight change is:
![]() | (8) |
![]() | (9) |
![]() | (10) |
The two terms are then combined additively with a normalized mixing constant khebb:
![]() | (11) |
Temporal Differences and Adaptive Critic Gating Mechanisms
To implement the temporal differences (TD) algorithm in Leabra, the adaptive critic (AC) unit in the VTA layer has plusminus phase states that correspond to the expected reward at the previous time step (minus) and the current time step (plus). The difference between these two states is the TD error , which is essentially equivalent to the more standard kinds of error signals computed by the error-driven learning component of Leabra, except that it represents an error of prediction over time, instead of an instantaneous error in the network output.
The ACPFC relationship is formalized in the model with the following equations for the gating (multiplicative scaling) terms sin (the weight scaling of the PFC inputs) and smaint (the weight-scaling of the PFC self-maintenance connections):
![]() | (12) |
![]() | (13) |
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Christoff K, Gabrieli JDE (2000) The frontopolar cortex and human cognition: evidence for a rostrocaudal hierarchical organization within the human prefrontal cortex. Psychobiology 28:168186.[ISI]
Cohen JD, O'Reilly RC (1996) A preliminary theory of the interactions between prefrontal cortex and hippocampus that contribute to planning and prospective memory. In: Prospective memory: theory and applications (Brandimonte M, Einstein GO, McDaniel, MA, eds), pp. 267296. Mahwah, NJ: Erlbaum.
Cohen JD, Braver TS, O'Reilly RC (1996) A computational approach to prefrontal cortex, cognitive control, and schizophrenia: recent developments and current challenges. Phil Trans R Soc Lond B Sci 351:15151527.
Cohen JD, Dunbar K, McClelland JL (1990) On the control of automatic processes: a parallel distributed processing model of the Stroop effect. Psychol Rev 97:332361.[ISI][Medline]
Dehaene S, Changeux JP (1991) The Wisconsin Card Sorting Test: theoretical analysis and modeling in a neuronal network. Cereb Cortex 1:6279.[Abstract]
Desimone R, Duncan J (1995) Neural mechanisms of selective visual attention. Annu Rev Neurosci 18:193.[ISI][Medline]
Diamond A (1990) The development and neural bases of memory functions as indexed by the A-not-B task: evidence for dependence on dorsolateral prefrontal cortex. In: The development and neural bases of higher cognitive functions (Diamond A, ed.), pp. 267317. New York: New York Academy of Science Press.
Dias R, Robbins TW, Roberts AC (1997) Dissociable forms of inhibitory control within prefrontal cortex with an analog of the Wisconsin Card Sort Test: Restriction to novel situations and independence from on-line processing. J Neurosci 17:92859297.
Durstewitz D, Kelc M, Gunturkun O (1999) A neurocomputational theory of the dopaminergic modulation of working memory functions. J Neurosci 19:2807.
Felleman DJ, Van Essen DC (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:147.[Abstract]
Frank MJ, Loughry B, O'Reilly RC (2001) Interactions between the frontal cortex and basal ganglia in working memory: a computational model. Cognit Affect Behav Neurosci 1:137160.[Medline]
Freedman D, Riesenhuber M, Poggio T, Miller E (2001) Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291:312316.
Fuster JM (1989) The prefrontal cortex: anatomy, physiology and neuropsychology of the frontal lobe. New York: Raven Press.
Fuster JM, Alexander GE (1971) Neuron activity related to short-term memory. Science 173:652654.[ISI][Medline]
Gobbel JR (1997) The role of the neostriatum in the execution of action sequences. PhD thesis, University of California, San Diego, San Diego, CA.
Goldman-Rakic PS (1987) Circuitry of primate prefrontal cortex and regulation of behavior by representational memory. Handbook of physiology the nervous system, vol. 5, pp. 373417.
Kimberg DY, Farah MJ (1993) A unified account of cognitive impairments following frontal lobe damage: the role of working memory in complex, organized behavior. J Exp Psychol Gen 122:411428.[ISI][Medline]
Koechlin E, Basso G, Grafman J (1999) The role of the anterior prefrontal cortex in human cognition. Nature 399:148.[ISI][Medline]
Kubota K, Niki H (1971) Prefrontal cortical unit activity and delayed alternation performance in monkeys. J Neurophysiol 34:337347.
Levine DS, Prueitt PS (1989) Modeling some effects of frontal lobe damage novelty and perseveration. Neural Netw 2:103116.[ISI]
Miller EK, Cohen JD (2001) An integrative theory of prefrontal cortex function. Annu Rev Neurosci 24:167202.[ISI][Medline]
Mishkin M, Manning FJ (1978) Nonspatial memory after selective prefrontal lesions in monkeys. Brain Res 143:313323.[ISI][Medline]
Miyake A, Shah P (eds). (1999) Models of working memory: mechanisms of active maintenance and executive control. New York: Cambridge University Press.
Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mes-encephalic dopamine systems based on predictive Hebbian learning. J Neuroscience 16:19361947.[Abstract]
Munakata Y (1998) Infant perseveration and implications for object permanence theories: a PDP model of the AB task. Dev Sci 1:161184.
Munakata Y, Morton JB, Stedron JM (2001) The role of prefrontal cortex in perseveration: developmental and computational explorations. In: Connectionist models of development (Quinlan P, ed.). Hove: Psychology Press, in press.
O'Reilly RC (1996) Biologically plausible error-driven learning using local activation differences: the generalized recirculation algorithm. Neural Comput 8:895938.[ISI]
O'Reilly RC (1998) Six principles for biologically-based computational models of cortical cognition. Trends Cogn Sci 2:455462.[ISI]
O'Reilly RC (2001) Generalization in interactive networks: the benefits of inhibitory competition and Hebbian learning. Neural Comput 13:11991242.
O'Reilly RC, Munakata Y (2000) Computational explorations in cognitive neuroscience: understanding the mind by simulating the brain. Cambridge, MA: MIT Press.
O'Reilly RC, Rudy JW (2001) Conjunctive representations in learning and memory: principles of cortical and hippocampal function. Psychol Rev 108:311345.[ISI][Medline]
O'Reilly RC, Braver TS, Cohen JD (1999) A biologically based computational model of working memory. In: Models of working memory: mechanisms of active maintenance and executive control (Miyake A, Shah P, eds), pp. 375411. New York: Cambridge University Press.
Owen AM, Roberts AC, Hodges JR, Summers BA, Polkey CE, Robbins TW (1993) Contrasting mechanisms of impaired attentional set-shifting in patients with frontal lobe damage or Parkinson's disease. Brain 116:11591175.[Abstract]
Petrides M (1994) Frontal lobes and working memory: evidence from investigations of the effects of cortical excisions in nonhuman primates. In: Handbook of neuropsychology, vol. 9 (Boller F, Grafman J, eds), pp. 5982. Amsterdam: Elsevier.
Roberts AC, Robbins TW, Everitt BJ (1988) The effects of intradimensional and extradimensional shifts on visual discrimination learning in humans and non-human primates. Q J Exp Psychol 40:321341.[ISI]
Rogers RD, Andrews TC, Grasby PM, Brooks DJ, Robbins TW (2000) Contrasting cortical and subcortical activations produced by attentional-set shifting and reversal learning in humans. J Cogn Neurosci 12:142162.
Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13:900913.[Abstract]
Stuss DT, Levine B, Alexander MP, Hong J, Palumbo C, Hamer L, Murphy KJ, Izukawa D (2000) Wisconsin card sorting test performance in patients with focal frontal and posterior brain damage: effects of lesion location and test structure on separable cognitive processes. Neuropsychologia 38:388402.[ISI][Medline]
Sutton RS (1988) Learning to predict by the method of temporal differences. Machine Learn 3:944.
Wilson FAW, Scalaidhe SPO, Goldman-Rakic PS (1993) Dissociation of object and spatial processing domains in primate prefrontal cortex. Science 260:19551957.[ISI][Medline]