Statistical Physics of High Dimensions
A network can contain many thousands of units, each responding to the others. Describing every unit separately gives us a very large set of variables, but does not immediately tell us what the network does. Does it settle into a recognisable pattern? Can it leave that pattern? Does it keep exploring, or repeatedly return to the same states? Statistical physics gives us tools for relating these collective behaviours to the interactions that produce them.
Many variables, a few collective questions
Here, high dimensions refers to the space of possible configurations. If we describe a system with a thousand binary units, a configuration specifies a choice for each of them. There are two to the power of a thousand possible configurations. This space is enormous, even when the units themselves occupy an ordinary three-dimensional world. The dimension of the configuration space and the dimension of physical space describe different things.
The interactions make some configurations more likely or more stable than others. A useful picture is a landscape, where regions correspond to different collective states. A low-dimensional drawing can suggest valleys and barriers, but the actual network has many directions in which it can change. A valley in one projection may hide a route that is visible only when other variables are included.
Rather than catalogue every configuration, we look for order parameters: quantities that distinguish forms of collective organisation. In a magnet, one example is the average alignment of the spins. In a memory network, an overlap measures how closely the current activity resembles a stored pattern. Such quantities let us ask whether a memory is retrieved without following every unit separately. Which quantities we need depends on the behaviour we want to explain.
We can then study regimes and the boundaries between them. Changing noise, interaction strength, or the number of stored patterns may change the kind of behaviour the system supports. A phase diagram maps these possibilities. Statistical physics asks which properties are typical in large systems, how fluctuations matter, and whether changes become sharp transitions as the system grows. This gives a simulation a wider theoretical context.
Disorder, frustration, and free energy
Spin-glass theory offers a particularly useful example. The interactions differ across pairs of units, and their preferences can compete. A disorder is quenched when it remains fixed on the timescale over which the state variables evolve. We can place the couplings in the Hamiltonian, the function assigning an energy to each configuration, as fixed objects. Different realisations draw different couplings from a specified distribution. In a memory model, stored patterns and the connections constructed from them can play a similar role during retrieval.
Structural glasses offer a different route to disorder. Their irregular organisation arises through the configurations of the particles themselves, rather than through an externally specified set of random couplings. This is called self-generated disorder. The distinction concerns where the disorder comes from; it does not imply that a glass must relax quickly or that its disorder can simply be averaged away.
Competing interactions can create frustration: all their preferences cannot be satisfied simultaneously. Imagine three spins, with two pairs preferring equal signs and the third preferring opposite signs. Satisfying the first two makes the third impossible to satisfy. In a large network, many such conflicts can produce a rugged landscape with numerous locally stable arrangements. Barriers between them can cause slowing down and long residence in metastable states. Disorder alone, however, does not guarantee frustration or glassy behaviour.
At equilibrium, energy is only part of the story: there may be many more configurations of one kind than another. The partition function adds their statistical weights, and the free energy combines the energetic preference with this multiplicity, described by entropy. Free energy is obtained from the logarithm of the partition function. It connects the microscopic model to equilibrium properties, while the time needed to reach equilibrium remains a separate dynamical question.
Since the disorder enters the Hamiltonian, it also enters the partition function and free energy. How, then, can a theory describe a material whose samples have different microscopic couplings? An important possibility is self-averaging: sample-to-sample fluctuations of a quantity such as the free energy per component become negligible in the thermodynamic limit, as the system grows. An average over the disorder can then describe a typical large sample. This property must be established for the model and quantity concerned, rather than assumed for every observable.
The geometry of the interactions matters too. In a regular spatial lattice with short-range interactions, the boundary becomes small compared with the bulk as the system grows. This helps explain why a free energy proportional to system size can be meaningful. In a fully connected model, each unit instead interacts with a growing number of others, and the couplings need an appropriate scaling. Both systems have large configuration spaces; their interaction structure and large-system limits differ.
Here comes the tricky part.
For a quenched average, we calculate the free energy for each fixed realisation and then average those free energies. This requires averaging the logarithm of the partition function. In an annealed average, we average the partition function first and then take its logarithm. The operations generally give different results. Annealing is relevant when disorder variables can equilibrate alongside the other degrees of freedom, and it can also serve as an approximation or bound for a quenched problem.
The replica trick addresses this difficult logarithmic average. It introduces auxiliary copies of the system sharing the same disorder, calculates averages involving an integer number of copies, and formally continues the result towards zero copies. The relations between these replicas help characterise the organisation of equilibrium states. It is a powerful method, but its continuation and assumptions need justification; the trick does not by itself guarantee an exact answer.
From collective states to changing trajectories
These ideas carry over to associative memory. Learned interactions can stabilise stored patterns, while interference between patterns can distort their basins or create unintended mixtures. The relevant question is collective: how much information can be stored while reliable retrieval remains possible? Statistical physics lets us relate storage load, correlations, sparsity, and noise to the regimes a network supports.
Mean-field methods make some large interacting systems tractable by replacing the rest of the network with a self-consistent effective description of its influence on a representative unit. That influence need not be a constant average. In dynamical mean-field theory, it can include fluctuating input and dependence on past activity. Correlations describe how activity at different times is related; response functions describe how a small perturbation changes later activity. The effective process must reproduce the statistics of the network from which it was derived.
For spontaneous memory transitions, the landscape itself can change. In an adaptive Potts network, sustained activity raises thresholds that weaken the currently retrieved pattern. A sequence may then depend on both present activity and accumulated adaptation. Equilibrium free energy alone cannot describe this full process: we need a theory of the evolving variables and their history. The organisation of stable states and the organisation of trajectories become connected, but distinct, problems.
This is the connection I explore in my research. I use statistical physics to ask how a large memory system can maintain recognisable representations while also leaving them, and how changes in structure or adaptation reshape the accessible repertoire. The aim is to identify collective mechanisms and observables that remain understandable even when the full microscopic description is too large to follow.