Associative Memories
A familiar voice can bring back a person, a room, and a conversation. The cue contains only a small part of what we remember. An associative memory is a system that can recover a stored representation from part of its content, rather than from an address telling it where to look. Recovering the missing parts is called pattern completion.
How can a network do this? Imagine that a memory is a pattern of activity spread across many units. Connections let the units influence one another. If the connections have learned which activities belong together, activating part of a pattern can recruit the rest. The memory is distributed: it belongs to a collective configuration, and individual units can participate in many different memories.
The Hopfield network makes this idea concrete. Each unit has two possible states, and the connections are constructed from the patterns we want to store. During retrieval, each unit responds to the input from the others. With symmetric connections and suitable update rules, the network settles into a stable configuration called an attractor. Nearby, incomplete configurations that lead to the same attractor form its basin of attraction. A memory therefore has both a pattern and a region from which that pattern can be recovered.
Here comes a difficulty. Memories share the same connections. The influence of one stored pattern can interfere with the retrieval of another; this is called crosstalk. Adding memories changes the landscape itself, and may create stable mixtures that were never intended as memories. Storage capacity asks how many patterns remain reliably retrievable under specified conditions. It is different from counting all the configurations the units could possibly take.
For a description of interacting cortical populations, two states may be too restrictive. A population can participate in different local activity patterns. In a cortical Potts network, a unit represents a patch of cortex with several possible local states, together with an inactive state. A global memory coordinates selected local states across many patches. This is coarse graining: we retain the distinctions relevant to the collective dynamics while leaving the individual neurons inside each patch undescribed. The local states are a modeling assumption, rather than a direct reading of a brain scan.
Successful retrieval creates another difficulty. A stable attractor is good at keeping a memory present, but a train of thought needs to leave it. Adaptation provides one possible mechanism: the local states that have been active gradually become harder to maintain. Inhibition also regulates how much activity a patch can sustain. The current memory can then become metastable—recognisable for a while, but no longer permanent.
Leaving a memory does not guarantee reaching another. Activity may simply die away. A transition needs a successor to become established while enough activity remains to cue it. Shared local states and learned relations can favour some successors over others. When this process repeats autonomously, the network shows latching: a sequence of transitions between stored memories. Depending on the balance of recurrence, adaptation, and inhibition, it may stop after one retrieval, make a short sequence, continue switching, or remain stuck.
This is what interests me about associative memory. Storing a representation, retrieving it from a cue, and visiting it during an ongoing sequence are three different achievements. I study how the organisation of memories and the dynamics of the network determine which of them become possible. It gives us a way to ask how a system built to remember can also generate a structured succession of new thoughts.