- 24-200LLM agents used in a 2025 Science Advances study of spontaneous naming-convention formation
- 626AI agents analysed in a metadata study of emergent social structure on an autonomous agent network
- 0central coordinators required for population-wide conventions to emerge in these studies
Most agent engineering, including most of this pipeline’s own coverage of the field, treats an agent as a unit: does it complete its task, does it call its tools correctly, does it stay inside budget. That framing works fine for one agent, or even a small fixed pipeline of a handful of specialized agents handing off to each other. It stops working once you deploy hundreds or thousands of agents that interact with each other repeatedly inside a shared environment, because new behaviour appears at the population level that no single agent’s transcript would ever show you.
A 2025 study published in Science Advances is the cleanest demonstration of this so far. Researchers ran repeated coordination games with populations of 24 to 200 LLM agents, and watched the population converge on shared naming conventions through purely local, pairwise interactions — no agent was told what the eventual convention should be, and no central process assigned one. More strikingly, collective biases emerged during that convergence process even when individual agents, tested alone, showed no such bias. And in several runs, a small committed minority of agents was enough to tip the entire population toward adopting a different convention than the one that was emerging organically.
Why this doesn’t show up in single-agent evaluation
The reason this matters for anyone running agents in production is straightforward: your evaluation suite almost certainly tests agents one at a time, or in small fixed groups, against fixed scenarios. Population-level emergent behaviour requires exactly the conditions most eval frameworks are built to avoid — many instances, repeated interaction, and enough runs for a pattern to accumulate. A model card, a single-agent benchmark score, or even a well-built multi-agent orchestration test will not surface a convention or a bias that only appears once you have a few hundred agents interacting over thousands of exchanges.
| Scale | What emerges | Where it’s been measured |
|---|---|---|
| Small (2-10 agents) | Coordination gains between specific paired or grouped agents | Standard multi-agent orchestration benchmarks |
| Medium (dozens to ~200) | Population-wide naming conventions and collective bias, even absent individual bias | Controlled coordination-game studies (Science Advances, 2025) |
| Large (hundreds+) | Spontaneous social graphs — agents independently choosing peers to trust without being designed to | Metadata analysis of 626 agents on an autonomous agent network |
Trust networks nobody designed
A separate line of research looked at a population of 626 AI agents operating on a shared protocol and found that the resulting social graph — who trusted whom, who routed requests to whom — emerged from agents independently discovering shared communication infrastructure and choosing peers, rather than being specified by any designer. That is a genuinely different category of system property than what most teams are used to reasoning about. A permissions model can tell you which agents are allowed to talk to which other agents. It cannot tell you which of those permitted connections will actually get used heavily, or which agents will become de facto hubs that a disproportionate share of the population routes through.
This has an obvious operational implication: if a small number of agents become trust hubs through emergent dynamics rather than deliberate design, those agents become single points of failure and attack surface that your architecture diagram does not show, because the architecture diagram reflects what you built, not what actually happened once agents started interacting at scale.
- 1. Local interactionPairs or small groups of agents interact repeatedly on a shared task or protocol.
- 2. Convention driftNo global convention exists yet; local patterns start to differ slightly across the population.
- 3. TippingA small committed minority, or a chance early pattern, starts winning out over alternatives.
- 4. ConvergenceThe population-wide convention stabilizes — a shared naming scheme, trust pattern, or behavioural norm.
- 5. PersistenceThe convention persists and shapes how new agents entering the population behave, even though no one specified it.
How a population-level convention emerges without central coordination
This is the same basic dynamic that produces human linguistic and social conventions — it is not unique to AI agents, but it happens far faster in machine populations.
Collective intelligence versus collective failure
The research literature draws a useful distinction between coordination gains and collective intelligence. At small scale, agents interacting mostly produce coordination gains — two or three agents doing a joint task better together than any could alone, which is the property most multi-agent orchestration work already optimises for. At medium and large population scale, something qualitatively different appears: capabilities that no single agent has, arising specifically from the population’s interaction structure. That is the upside case.
The downside case is structurally identical: unwanted, unanticipated effects can emerge from agent interactions in exactly the same way desirable collective intelligence does, and current research explicitly flags that studying this empirically at realistic scale, with state-of-the-art systems, remains genuinely difficult. There is no reliable way today to predict in advance which of your deployed agent population’s emergent properties will be useful coordination and which will be an unanticipated collective failure mode, because both arise from the same underlying dynamic.
Monitor population-level metrics Do this
Track convention formation, trust-graph concentration, and behavioural drift across the whole agent population, not just individual agent success rates.
Assume single-agent eval generalises Risky default
A benchmark score for one agent tells you nothing reliable about what hundreds of instances of that agent will collectively converge on once deployed together.
Design a rigid protocol and assume behaviour stays put Doesn’t hold
Even a well-specified communication protocol leaves room for emergent trust patterns and conventions the protocol’s designer never anticipated.
Common mistakes
Common mistake
Evaluating a new agent version in isolation and assuming that clears it for population-wide deployment. A change that looks neutral or positive for one agent can shift the conventions an entire population converges on once redeployed at scale.
Common mistake
Treating your permissions graph as equivalent to your actual trust graph. Permissions describe what’s allowed; emergent usage patterns describe what actually happens, and the two reliably diverge once a population is large enough.
What worked
Teams running the largest agent populations added population-level dashboards specifically for convention drift and trust-graph concentration, alongside their existing per-agent reliability metrics, and treated a sudden shift in either as an incident worth investigating even when no individual agent’s error rate had changed.
Glossary
- Emergent behaviour
- Population-level patterns that arise from repeated local interactions between agents, not explicitly programmed into any individual agent.
- Collective intelligence
- New capabilities that appear at the population level which no single agent in that population possesses individually.
- Convention formation
- The process by which a population of interacting agents converges on a shared norm, naming scheme, or behavioural pattern without central coordination.
- Trust graph
- The actual pattern of which agents route requests to, or rely on, which other agents — which can diverge sharply from the designed permissions graph.
- Tipping point (in agent populations)
- The threshold at which a small committed minority of agents is sufficient to shift the entire population toward a different collective convention.
Key takeaways
- Populations of interacting AI agents form conventions and collective biases that no individual agent shows when tested alone, as demonstrated in controlled 2025 coordination-game research.
- A small committed minority of agents can be enough to tip an entire population toward a particular convention, mirroring known dynamics from human social systems.
- Trust networks between agents can emerge spontaneously from shared infrastructure and peer selection, producing a real social graph that diverges from the designed permissions graph.
- Single-agent evaluation and benchmarks cannot surface population-level emergent behaviour, because that behaviour requires many interacting instances over repeated exchanges to appear at all.
- The same underlying dynamic that produces useful collective intelligence can also produce unanticipated collective failure — there’s currently no reliable way to predict in advance which one you’ll get.
- Emergent trust hubs that form without deliberate design become single points of failure that don’t show up on an architecture diagram of what was actually built.
- Teams managing large agent populations need population-level monitoring — convention drift, trust concentration — in addition to standard per-agent reliability metrics.
FAQs
Do groups of AI agents really develop their own social behaviour?
Yes, in a measurable sense. Controlled 2025 research had 24 to 200 LLM agents converge on shared naming conventions purely through repeated local interactions, with no central coordinator specifying the outcome, and collective biases emerged even when individual agents tested alone showed none.
How many AI agents does it take before emergent behaviour appears?
Research suggests coordination gains between small groups appear even at just a few agents, but population-level phenomena like convention formation and spontaneous trust networks have been specifically studied and measured at scales from roughly two dozen agents up to several hundred.
Can a small number of agents change what an entire population converges on?
Yes. Studies found that a small committed minority of agents was sufficient to tip an entire population toward a different shared convention than the one that was organically emerging, a dynamic well documented in human social systems and now measured in AI agent populations too.
Do AI agents form trust relationships without being explicitly designed to?
Yes. A metadata analysis of 626 agents operating on a shared protocol found the resulting social graph — who agents chose to trust and route requests through — emerged from agents independently discovering infrastructure and choosing peers, rather than being specified by a designer.
Can standard agent evaluation catch this kind of emergent behaviour?
No, generally not. Single-agent benchmarks and even small fixed multi-agent tests don’t involve enough interacting instances or repeated exchanges for population-level conventions or biases to appear, so they simply won’t surface in that kind of evaluation.
Is emergent behaviour in agent populations always harmful?
No. The same interaction dynamics that produce genuinely useful collective intelligence — capabilities no single agent has — can also produce unanticipated, unwanted effects, and researchers currently have no reliable way to predict in advance which outcome a given deployment will produce.
What operational risk comes from emergent trust networks specifically?
Agents that become de facto trust hubs through emergent dynamics, rather than deliberate design, become single points of failure and attack surface that don’t appear anywhere on the system’s designed architecture diagram.
How should teams monitor for emergent behaviour in large agent deployments?
By adding population-level metrics — convention drift and trust-graph concentration across the whole deployed population — alongside standard per-agent reliability metrics, and treating sudden shifts in either as worth investigating even without a change in individual agent error rates.
References
- arXiv — “Emergent Social Structures in Autonomous AI Agent Networks: A Metadata Analysis of 626 Agents”
- arXiv — “Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futures”
- arXiv — “Multi-Agent Risks from Advanced AI”
- arXiv — “Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents”
Related reading on The Tech Trends: Multi-Agent Orchestration Patterns, Multi-Agent Consensus, Agent-to-Agent Protocols, Agent Observability, and Self-Improving Agents.
