Pattern Matching Overview A Model of Addition

Notes on Methodology

COGENT embodies a number of theoretical positions on the role and proper use of computational modelling in cognitive psychology. It is necessary to understand these positions in order to make best use of COGENT. Having completed a basic introduction to the functionality of COGENT, you should now be in a position to appreciate the methodological side of the environment.

Theory and Implementation

In computational terms, psychological theory is generally abstract: rarely does it specify a unique behavioural function and never does it specify a unique implementation. Computational modelling must be sensitive to the abstract nature of psychological theory, and avoid confusing the behaviour of specific implementations with the predicted behaviour of corresponding theories.

COGENT provides some support for this theory/implementation distinction, though we acknowledge that further support is possible. Firstly, the use of box/arrow notation is a specific attempt to provide psychologists with a tool in which they may specify models at a level of abstraction commensurate with their theories. Psychological theories are typically stated at the box/arrow level, and COGENT's graphical interface is specifically designed to allow psychologists to work, where possible, at this level of abstraction.

Further support comes in the form of object parameters. Many box types have parameters or properties which determine their precise behaviour. The variation of the values of these parameters can be used to test their criticality. That is, we can test to see how critical a parameter's value is by examining the behaviour of the model when the parameter is set at different values. Behavioural regularities that hold over a range of parameter settings show that the parameter's precise value is not critical to that behaviour.

Computational Experiments

Computational models are not built in a vacuum. They are based on psychological theory and grounded in empirical data. Thus, once a model has been built, it should be tested against empirical data. How should this be done? The standard psychological experiment involves testing a number of subjects under a range of conditions. Parameters or variables of interest are identified and systematically varied. [Varying subject parameters (e.g., socio-economic status) lead to between group experimental designs. Varying environmental parameters (e.g., time of day) lead to within group experimental designs. Varying both subject and environmental parameters lead to mixed designs of various types.] One or more measures of subject behaviour (dependent variables) are recorded and later analysed using standard, well-understood, statistical procedures. We argue that computational models are best tested by using the same procedure: by conducting "computational experiments" in which parameters of interest are systematically varied and in which dependent variables are recorded and later statistically analysed.

COGENT supports computational experiments in several ways:

  1. The precise behaviour of many COGENT objects is specified by a set of properties, including (for example) buffer capacity and decay rate. In cases where there is a theoretical reason to view such parameters as reflecting between subject variation (e.g., we may be dealing with groups of subjects with high and low digit span, in which case we might hypothesise that working memory capacity is a parameter which distinguishes the groups), we may systematically explore the relationship between the parameter's value and behaviour.
  2. Certain object properties (e.g., buffer access and decay) have values that lead to constrained variation in the object's behaviour. Such variation may also lead to variation in the model's behaviour. Thus, unlike some computational models, COGENT models may contain sources of random behavioural variation (as we see in real cognition). It is therefore often not sufficient to run a COGENT model once to determine its behaviour. Behaviour of the model, like the subject, varies, and so it is appropriate (as in laboratory experiments) to collect data from many trials (via "Monte Carlo" simulations) and focus on statistical regularities that hold over these data sets.
  3. Facilities are provided so that a model can automatically be run multiple times over the same data set, so that random variation in model behaviour can be recorded. It is currently possibly to run a block of trials, with the precise number of trials being run being independently specified for each block.

The Subject and the Experimental Environment

Any psychological experiment consists of two things: the subject (or subjects) and the environment. Therefore, if we are going to build a computational model of subject performance in some experiment, we have to model both the subject and the experimental environment. The box/arrow notation and underlying language of COGENT can be used for both of these tasks, and COGENT has facilities to allow a clean separation between the two.

As the tutorial progresses, you'll see how the subject model can be "encapsulated" into a single "compound" box. A similar compound box may be used to encapsulate the experimental environment (i.e., the presentation of stimulus materials and the collection and collation of subject responses).

Experimental psychologists might note that both the subject model and the experimental environment may contain variables or parameters. Variables within the experimental environment correspond to within subject variables. (For example, we may vary presentation rate, an environmental variable, whilst holding the subject model fixed, thus modelling an experiment where a single subject performs a task under two different experimental conditions.) Variables within the subject model correspond to between subject variables. (For example, we may vary working memory capacity, a subject model variable, whilst holding the experimental environment fixed, thus modelling an experiment using subject groups of high and low memory capacity).

Pattern Matching Overview A Model of Addition