Adaptive Systems
Modern AI provides a useful starting point for thinking about adaptation.
A neural network may contain billions of adjustable parameters. Training changes some of those parameters, and the resulting configuration changes how the model responds to future inputs.
At the finest analytical scale, a parameter can be treated as a node.
But the useful node depends on the scale of the question.
A node may instead be:
parameter → neuron / feature / attention head → layer / expert / module → AI model
The point is not that these objects are physically identical. Each can serve as an effective unit of analysis when studying how an adaptive system changes.
What is a node?
In this framework, a node is an effective unit whose state, behaviour, or relationship to other units matters at the scale being studied.
At one scale, a single parameter may be the relevant adaptive degree of freedom. At another, many parameters may be treated together as a functional module. At the highest scale used on this page, the model itself is the unit whose behaviour changes through training or other forms of adaptation.
Adaptation changes future behaviour
Suppose a model encounters new training information.
Some part of its internal state changes. Those local changes alter how information propagates through the network and therefore change future behaviour.
A simple description is:
environmental information → internal change → changed system dynamics → changed future behaviour
The scientific question is not simply whether something changed. It is:
Which parts changed, why did they change, and what consequences did those changes have for the model as a whole?
Local change and global behaviour
A change can occur locally while its effects propagate broadly because the changing part is connected to the rest of the system.
This means adaptation depends not only on what changes, but also on how the changing part is connected.
The same amount of local modification can therefore produce very different model-level consequences.
Adaptation and preservation
Learning creates two simultaneous requirements:
change what needs to change
while
preserving what should remain stable.
Improving one capability may disturb another. New training can alter previously useful behaviour. Broad updating may change more of the model than the new requirement actually demands.
This raises an architectural question:
Can adaptation be made more selective?
Scale matters
At parameter level, adaptation may involve changing numerical values.
At functional level, it may involve changing particular features, experts, or modules.
At model level, we can ask whether different architectures make useful change easier to localise or preserve.
The relevant scale is therefore not fixed in advance.
Adaptation is more than parameter updating
Gradient-based parameter updating is one implementation of adaptation, but it is not the definition of adaptation itself.
Within an AI model, future behaviour can also change through routing, activation patterns, memory, specialised modules, internal connectivity, parameter-efficient adaptation, or other persistent changes in how the model processes information.
The deeper object of study is:
history-dependent change in an organised system that alters its future behaviour.
Why this matters for AI architecture
Once adaptation is treated as an architectural problem, several questions become more precise:
- Which parts of a model should change in response to new information?
- Which existing capabilities should remain protected?
- How strongly should different functional regions be coupled?
- Can useful adaptation remain local rather than propagate unnecessarily?
- Can different parts of a model adapt at different rates?
- How can new capability be added with less disturbance to existing capability?
These are questions about the organisation of adaptive change, not model size alone.