| Myth | Reality |
|---|---|
| A bigger context window solves the large-codebase problem. | Even very large context windows cannot fit an enterprise monorepo, and stuffing more files in without relevance ranking degrades the model’s attention to what actually matters. |
| Vector embedding search alone is enough to find the relevant code. | Vector search captures semantic similarity but misses structural relationships like which function calls which; most working systems pair it with AST or dependency-graph analysis. |
| Indexing the whole repo once is sufficient going forward. | Codebases change constantly, so indexes need incremental updates or the assistant will reason about code that no longer exists in that form. |
| Agentic keyword search is a weaker fallback compared to a vector database. | Recent evaluations found agentic search through iterative keyword queries can reach over 90 percent of RAG-level performance without any vector database, when the underlying model is capable enough to refine its own queries. |
The Core Problem: Repositories Are Bigger Than Any Context Window
A single file rarely tells the whole story in a real codebase. A function call in one file might route through three layers of abstraction defined in other files, touch a shared configuration loaded from yet another file, and depend on a database schema defined somewhere else entirely. Large language models reason well within whatever text is actually in their context window, but an enterprise monorepo with tens of thousands of files simply cannot be pasted in wholesale, no matter how large the window grows. This is the repo-level context problem: how does an AI coding tool figure out which small slice of a much larger codebase is actually relevant to the task in front of it right now.
The naive answer — just include more files — runs into two walls. The first is a hard technical limit: even generous context windows are dwarfed by real production repositories. The second is subtler and matters even when the technical limit isn’t hit: stuffing irrelevant files into context measurably degrades a model’s ability to focus on what matters, because attention has to be spread across everything included, relevant or not. More context is not free; it has to be the right context.
How Repo-Level Context Systems Actually Work
Hybrid Indexing: Structure Plus Semantics
The approach that has emerged as the practical standard combines two complementary indexing strategies. Abstract syntax tree (AST) and code-graph analysis captures structural relationships: what calls what, how types relate to each other, which modules import which. Vector embedding search captures semantic similarity: which pieces of code are conceptually related even if they never directly reference each other, such as two different implementations of similar business logic in different services. Neither approach alone is sufficient. AST analysis without semantic search misses conceptually related code that isn’t structurally connected. Semantic search without AST analysis misses precise call-chain relationships that matter for understanding exactly how a change will propagate.
AST-aware chunking, where a codebase is broken into pieces along meaningful structural boundaries like function or class definitions rather than arbitrary character counts, has been measured to improve retrieval precision by around 40 percent over naive character-based splitting in large repositories. This matters because a chunk boundary that splits a function in half produces a retrieval result that is technically relevant but practically useless.
Layered Architecture for Context Assembly
More advanced repo-context systems are built as layered architectures rather than a single retrieval step. A core ingestion and indexing engine builds and maintains the structural and semantic indexes. A service layer assembles context for a specific query, deciding which retrieved pieces actually get included and in what order. An interface layer, often built on protocols like MCP, exposes this assembled context to agents in a standard way. Agents consume the assembled context to perform the actual coding task. Persistent storage holds the embeddings and metadata so the index does not need to be rebuilt from scratch on every query.
Some of the more ambitious systems extend indexing beyond source code entirely, incorporating commit history, issue trackers, and documentation, on the theory that a meaningful share of “why does this code work this way” lives in commit messages and old tickets rather than in the code itself. This is an attempt to capture what is sometimes called tribal knowledge — unwritten conventions and architectural rationale that never made it into a comment or a doc.
A repo-context request, traced end to end
A developer asks an assistant to modify a payment-retry function. The service layer first pulls the AST-based call graph to find every caller of that function across the repo, then runs a semantic search for conceptually related retry logic in other services that might share the same underlying bug, then checks recent commit history on the touched file for context on why a prior change was made. All three results are merged, ranked by relevance, and only the top slice is assembled into the model’s context window, leaving the rest of the repository unindexed for this particular request.
| Technique | What it captures | Typical weakness |
|---|---|---|
| AST / code-graph analysis | Call chains, type relationships, import structure | Misses conceptual similarity between unconnected code |
| Vector embedding search | Semantic similarity between code fragments | Misses precise structural dependencies |
| Agentic keyword search | Iteratively refined queries driven by the model itself | Depends heavily on the underlying model’s query-formulation skill |
| Commit and ticket history indexing | Rationale and tribal knowledge behind past changes | Noisy, requires filtering out irrelevant historical churn |
Where Agentic Search Changes the Calculus
A significant recent finding complicates the assumption that a full vector database is always necessary. Research evaluating agentic search — where the model itself issues a sequence of keyword-based queries, reads the results, and iteratively narrows its focus, much like a human developer using grep and file navigation — found this approach can reach over 90 percent of the performance of a full retrieval-augmented generation pipeline, without maintaining any vector index at all. This works when the underlying model is capable enough to formulate effective search queries and knows when to stop searching and start acting.
This matters practically because vector indexes are not free: they need to be built, kept in sync with a constantly changing codebase, and hosted somewhere. For smaller teams or codebases that change extremely fast, agentic keyword search can be a lower-maintenance alternative that gets most of the benefit. For very large, stable monorepos where query latency matters and the model’s own search skill is a bottleneck, a maintained hybrid index still tends to outperform.
Why Multi-Repo Architectures Break Standard Approaches
Retrieval systems designed around a single repository run into a specific failure mode in microservice or multi-repo architectures: embeddings computed independently per repository miss cross-service dependencies entirely. A change to a shared API contract in one service’s repository has no natural link, in a per-repo embedding index, to the three other services that depend on that contract in their own separate repositories. Solving this requires either a unified index spanning all related repositories, or an explicit dependency map maintained outside any single repository’s index, neither of which is a solved problem at the tooling level yet.
| Architecture | Context challenge | Partial mitigation |
|---|---|---|
| Single large monorepo | Sheer file volume exceeds any context window | Hybrid AST plus vector indexing with relevance ranking |
| Multi-repo microservices | Cross-service dependencies invisible to per-repo embeddings | Unified index across repos or an explicit external dependency map |
| Rapidly changing codebase | Index staleness between updates | Incremental re-indexing triggered by commits rather than full rebuilds |
| Undocumented legacy modules | No natural-language description to match against semantically | AI-generated documentation pass to create a semantic anchor |
The Practical Limits That Still Cause Missed Cross-File Implications
Even the best hybrid systems available today still miss things a senior engineer who has worked in a codebase for years would catch instantly. A few patterns show up repeatedly:
- Retrieval ranks results by relevance to the immediate query, which can systematically deprioritize a distantly related file that turns out to matter, such as a shared constant defined three layers away from the code being changed.
- Index staleness between the last indexing run and the current state of the repository means an assistant can confidently reason about a function signature that was already changed in an uncommitted or very recent edit.
- Conventions that are never written down anywhere — not in code, not in commits, not in docs — remain invisible to every indexing strategy, because there is nothing in the corpus to retrieve.
- Context assembly has a budget; even a well-ranked retrieval still has to cut off at some point, and the cutoff is where cross-file implications quietly disappear from the model’s view.
Common mistake
Assuming that because an AI assistant is connected to a repo-indexing tool, it has effectively “read the whole codebase” and will automatically surface every relevant cross-file dependency. In practice the assistant only sees whatever slice the retrieval and ranking step decided to include for that specific query, and a distantly related file that matters can be silently excluded without any indication to the user that something was left out.
What worked
Explicitly asking the assistant to enumerate every caller of a function before making a change, rather than trusting that the retrieval step already surfaced all of them. Combining this explicit call-graph query with the assistant’s normal semantic retrieval caught several cross-service callers that the default relevance ranking had deprioritized, because they used different terminology than the query itself.
Practical Guidance for Working With Repo-Level Context Tools
- Treat retrieval results as a starting hypothesis, not a complete picture, especially for changes that cross module or service boundaries.
- Explicitly request call-graph or dependency queries for anything touching a widely used function, rather than relying only on semantic similarity search.
- Keep the index fresh with incremental updates tied to commits, since a stale index causes an assistant to reason about code that no longer exists in that form.
- Invest in AI-generated documentation for undocumented modules specifically because it gives semantic search something to match against that did not previously exist.
- For multi-repo architectures, maintain an explicit cross-repo dependency map rather than assuming per-repo indexing will surface service boundaries on its own.
Frequently Overlooked Aspects of Repo-Level Context
- Relevance ranking cutoffsEvery context assembly step has a budget, and the files just below the cutoff line are invisible to the model even though they may matter to correctness.
- Index staleness windowsAn assistant working from an index that has not been refreshed since the last few commits can confidently describe code that has already changed.
- Terminology mismatch in semantic searchTwo pieces of code that do the same thing but use different variable and function names can fail to surface as related in vector search despite being conceptually identical.
- Cross-repo dependency blindnessPer-repository embedding indexes have no natural mechanism for surfacing a dependent service in a completely separate repository.
- Tribal knowledge with no written traceConventions that were never documented anywhere in code, commits, or docs cannot be retrieved by any indexing strategy, because retrieval can only surface what exists in the corpus.
- Chunking boundary qualitySplitting code along arbitrary character counts rather than structural boundaries like function definitions measurably reduces retrieval precision.
- Agentic search query skillThe effectiveness of keyword-based agentic search depends heavily on the underlying model’s ability to formulate good queries and know when to stop searching.
- Silent omissionRetrieval systems rarely signal to the user when something relevant was excluded from context, which makes the failure mode hard to notice until a regression surfaces.
Glossary
- AST (Abstract Syntax Tree)
- A structural representation of code parsed into its syntactic components, used to analyze relationships like function calls and type usage independent of exact wording.
- Vector embedding search
- A retrieval method that represents code as numerical vectors capturing semantic meaning, allowing conceptually similar code to be found even without shared keywords.
- Retrieval-augmented generation (RAG)
- An approach where relevant external content is retrieved and inserted into a model’s context before generation, rather than relying solely on the model’s trained knowledge.
- Agentic search
- A retrieval approach where the model itself issues and refines a sequence of search queries iteratively, rather than relying on a single pre-built vector index.
- Dependency graph
- A structural map of which code units depend on or call which other code units, used to trace how a change in one place might affect others.
- Context window
- The maximum amount of text a model can process at once, which fundamentally bounds how much of a codebase can be included directly in a single request.
Key Takeaways
- AI tools understand large codebases through hybrid indexing that combines structural AST or code-graph analysis with semantic vector search, not by reading everything at once.
- AST-aware chunking along structural boundaries has been measured to improve retrieval precision by around 40 percent over naive character-based splitting.
- Agentic keyword search, where the model iteratively refines its own queries, can reach over 90 percent of full RAG performance without a maintained vector database.
- Multi-repo microservice architectures break standard per-repo embedding approaches because cross-service dependencies are invisible to any single repository’s index.
- Relevance ranking cutoffs and index staleness both create silent gaps where relevant cross-file implications disappear without any signal to the user.
- Tribal knowledge that was never written down anywhere in code, commits, or docs cannot be retrieved by any indexing strategy no matter how sophisticated.
- Explicitly requesting dependency or call-graph queries for widely used functions catches cross-file implications that default semantic retrieval can miss.
FAQs
Why can’t a large context window just solve the repo-level context problem by itself?
Even generous context windows are far smaller than real enterprise repositories with tens of thousands of files. Beyond the hard size limit, including irrelevant files measurably degrades a model’s attention to what actually matters, so the goal is relevant context, not maximum context.
What is the difference between AST-based indexing and vector embedding search?
AST-based indexing captures structural relationships like which function calls which and how types relate, built from parsing the code’s syntax. Vector embedding search captures semantic similarity between pieces of code, finding conceptually related logic even when there is no direct structural connection between them.
Is a vector database always necessary for an AI coding assistant to work on a large codebase?
Not always. Research has found that agentic search, where the model issues and iteratively refines its own keyword queries, can reach over 90 percent of the performance of a full retrieval-augmented generation pipeline without maintaining any vector index, provided the underlying model is skilled at query formulation.
Why do multi-repo microservice architectures cause problems for standard retrieval approaches?
Embeddings computed independently per repository have no natural way to surface dependencies on services defined in a completely separate repository. A change to a shared API contract in one service’s repo is invisible to a per-repo index covering the dependent services elsewhere.
What causes an AI assistant to miss a relevant cross-file implication even with repo indexing in place?
Context assembly has a relevance-ranking cutoff, and files just below that cutoff are excluded even if they matter to correctness. Index staleness, terminology mismatches in semantic search, and undocumented conventions with no written trace anywhere in the corpus also contribute to missed cross-file implications.
How much does AST-aware chunking actually improve retrieval quality?
AST-aware chunking, which splits code along structural boundaries like function or class definitions rather than arbitrary character counts, has been measured to improve retrieval precision by around 40 percent over naive character-based splitting in large repositories.
Can repo-level context systems index anything beyond source code itself?
Yes. More advanced systems extend indexing to commit history, issue trackers, and documentation, on the premise that a meaningful share of the rationale behind why code works a certain way lives in commit messages and old tickets rather than in the code itself.
What practical step catches cross-file implications that default retrieval misses?
Explicitly asking the assistant to enumerate every caller of a function or trace its full dependency graph, rather than relying solely on semantic similarity search, surfaces cross-service or cross-module callers that use different terminology and would otherwise be deprioritized by relevance ranking.
For more on the tooling and practices around AI-assisted development, see how to approach building your first MCP server, a comparison of AI coding agents compared, techniques for prompt engineering for developers, where the limits of vibe coding currently sit, and why guardrails and output validation matter for agent-driven changes.
- Kilo, “AI Coding Assistants for Large Codebases: Architecture, Evaluation, and Best Practices (2026)”
- Zylos Research, “Codebase Intelligence: How AI Agents Navigate, Understand, and Reason About Large Repositories in 2026”
- Supermemory, “Large-Repo Coding Agent Memory Bottleneck”
- arXiv, “Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches”
- Augment Code, “Sourcegraph Cody vs Qodo (2026): Code Search vs Review Gates”
