Vector RAG wins for targeted lookups; GraphRAG wins when the question spans the entire corpus. Ask “what did the Q3 contract say about termination” and vector search answers faster and cheaper. Ask “what themes run through all our contracts” and vector search structurally cannot answer, because no single chunk contains the answer.
- 1MToken corpus range in Microsoft’s published GraphRAG evaluation
- 4Query modes shipped in GraphRAG, one of which is plain vector search
- 2Distinct baseline-RAG failure classes Microsoft names explicitly
- 2LLM passes needed to build the index: extraction, then summarisation
- 2018Leiden clustering paper GraphRAG’s community detection depends on
The framing that spread through developer blogs in 2024 was that GraphRAG replaced vector RAG. That was never the claim. Microsoft Research published GraphRAG as a solution to a specific failure class, not a general upgrade, and the distinction matters far more now that the first round of indexing bills has arrived.
What follows compares them across eleven dimensions, walks the indexing pipeline stage by stage, and gives you a benchmark protocol you can run on your own corpus before committing to either.
What is the difference between GraphRAG and vector RAG?
Vector RAG embeds text chunks, stores the vectors, and retrieves the top k nearest neighbours to a query. GraphRAG extracts an entity knowledge graph from the corpus, clusters it hierarchically, and pre-generates summaries of each cluster.
One retrieves passages. The other retrieves structure.
That single sentence explains almost every practical difference below, including the ones that surprise teams after deployment.
| Dimension | Vector RAG | GraphRAG |
|---|---|---|
| Index unit | Embedded chunk | Entities, relationships, claims, community summaries |
| Index cost driver | Embedding calls | LLM extraction per text unit, plus summarisation per community |
| Query cost | One vector search, one generation | Map-reduce across community summaries in global mode |
| Best question type | Specific, local, fact-lookup | Global sensemaking, thematic, multi-hop |
| Connecting the dots | Poorly | By design |
| Corpus-wide themes | Structurally cannot | Primary use case |
| Incremental updates | Cheap, per-chunk | Expensive, may require re-clustering |
| Query latency | Sub-second retrieval | Seconds to minutes in global mode |
| Explainability | Cited chunk | Traceable entity and community provenance |
| Tuning burden | Chunk size, k, reranker | Extraction prompts, community granularity, plus all of vector RAG |
| Failure mode | Retrieves plausible but irrelevant chunks | Extraction errors propagate into graph structure |
Why did Microsoft build GraphRAG in the first place?
Because baseline RAG fails at two documented things, and no amount of chunk tuning fixes either.
Microsoft Research names both failures explicitly in the GraphRAG documentation. First, baseline RAG “struggles to connect the dots” when an answer requires traversing separate pieces of information through their shared attributes. Second, it performs poorly when asked to “holistically understand summarized semantic concepts over large data collections or even singular large documents.”
Both failures share one root cause: top-k retrieval assumes the answer lives inside some retrievable chunk. When the answer is a property of the corpus rather than a passage inside it, retrieval has nothing to retrieve.
Figure 1 — Where top-k retrieval succeeds and where it structurally cannot
“What did contract 4471 say about termination?”
The answer exists verbatim inside one chunk. Embedding similarity locates that chunk. One retrieval hop, one generation.
Adding a graph layer here changes nothing except cost.
Vector RAG wins
“What themes run through all our contracts?”
No chunk contains this answer. Retrieval returns the k most similar passages, each individually irrelevant to a corpus-level question.
Raising k makes it worse, not better: more noise, same missing answer.
Retrieval has nothing to retrieve
Read it this way: the deciding test is not question difficulty. It is whether the answer physically exists inside a retrievable unit of text. That test, not corpus size, should drive your architecture choice.
The published claim is narrower than the internet reports
Read the paper carefully. From Local to Global: A Graph RAG Approach to Query-Focused Summarization (arXiv:2404.16130, submitted 24 April 2024, revised 19 February 2025) reports improvements on “a class of global sensemaking questions over datasets in the 1 million token range,” measured on comprehensiveness and diversity of generated answers.
That is not a general accuracy benchmark. It is a query-focused summarisation result, on a bounded corpus size, judged on two specific qualities that are not the same as being correct.
Most comparison posts quote GraphRAG as broadly “more accurate.” The paper does not say that. If your workload is factoid lookup, the published evidence does not support switching, and you should be suspicious of anyone who tells you otherwise.
Watch for this
Benchmarks measuring comprehensiveness and diversity reward longer, broader answers. If your product needs a short precise answer, a method that scores well on those axes may actively work against your users. Match the metric to the job before you trust the result.
How does the GraphRAG index actually get built?
Four stages, each with a distinct cost profile and a distinct way of going wrong.
Figure 2 — The GraphRAG indexing pipeline
Cost concentration: stage 2 dominates, because it makes at least one LLM call per text unit. Vector RAG’s entire equivalent pipeline is a single step — embed each chunk — with no LLM inference at all.
Stage 3 deserves more attention than it usually gets. The hierarchical clustering uses the Leiden technique (Traag, Waltman and van Eck, arXiv:1810.08473, 2018), which operates over the entire graph rather than incrementally.
That global property is the source of the update problem discussed later. It is a mathematical characteristic of the algorithm, not an implementation detail somebody will optimise away.
Which query mode should you actually use?
GraphRAG ships four, and picking wrong is the most common self-inflicted cost problem in production deployments.
| Mode | What it reads | Use when | Relative cost |
|---|---|---|---|
| Global Search | Community summaries | Holistic questions about the whole corpus | Highest |
| Local Search | Entity neighbours | Questions about one specific named entity | Moderate |
| DRIFT Search | Entity neighbours plus community context | Entity questions needing broader framing | Moderate to high |
| Basic Search | Standard top-k vectors | The question is a plain lookup | Lowest |
Microsoft ships Basic Search inside GraphRAG for a reason. Even in a fully committed graph deployment, a meaningful share of real user traffic is best served by ordinary vector search.
Treat that as the strongest available signal about how the designers expect the system to be used. The intended architecture was never graph-only.
What does the indexing cost difference look like in practice?
No vendor publishes a like-for-like figure, so here is a transparent model you can re-run with your own numbers rather than trusting a headline.
Worked example — 10M token corpus
Assume 10 million tokens sliced into 20,000 text units of roughly 500 tokens each. These are illustrative structural assumptions, not quoted vendor prices.
| Line item | Vector RAG | GraphRAG |
|---|---|---|
| Embedding calls | 20,000 | 20,000 |
| LLM extraction calls | 0 | 20,000 |
| Tokens through an LLM | 0 | ~10M in, plus extraction output |
| Community summarisation | 0 | Hundreds to thousands of calls |
| Re-index on corpus change | Proportional to changed chunks | Potentially whole-graph |
The structural point survives whatever token prices you plug in: GraphRAG moves indexing from an embedding problem to an inference problem, and inference is one to two orders of magnitude more expensive per token.
The re-index row is the one teams consistently underestimate. A vector index absorbs a changed document by re-embedding a handful of chunks. A graph index may need entities re-extracted and communities re-clustered, because the clustering is global rather than local.
Model your update frequency before your query volume. For most enterprise corpora, update cost dominates total cost of ownership within the first year.
How do they compare on the three question types that matter?
Aggregate benchmarks hide the thing you need to know. Split your traffic by question type instead.
Factoid lookup
Vector RAG wins outright. The answer sits in one chunk, similarity search finds it, and a graph traversal adds latency and cost without adding information.
In most enterprise deployments this is the majority of traffic. That fact alone should make graph-only architectures suspect.
Multi-hop questions
This is the contested middle ground, and where the honest answer is “it depends on your data.”
If the hops are shallow and the linking entities appear together somewhere in the text, hybrid search with a reranker often closes the gap at a fraction of the indexing cost. If the hops require joining entities that genuinely never co-occur in any single document, the graph earns its keep.
The diagnostic is cheap to run: sample thirty multi-hop questions and check manually whether any single document contains both endpoints. If most do, you have a retrieval-tuning problem, not an architecture problem.
Corpus-wide thematic questions
GraphRAG wins, and the win is categorical rather than incremental. Vector RAG does not perform worse here; it cannot perform at all, because the target answer is not stored in any retrievable unit.
The practical question is what share of your traffic this represents. For a legal review tool it might be 30%. For a customer support bot it is close to zero.
Choose GraphRAG if, choose vector RAG if
Figure 3 — Architecture decision tree
Vector RAG. Do not build a graph.Cost driver: embedding calls only
Continue to the next question below.Do not stop here
Vector RAG plus reranking. Revisit when churn slows.Re-clustering cost would dominate
Continue to the final question.A bounded corpus is a graph candidate
GraphRAG, with a router sending factoid traffic to Basic Search.Cost driver: LLM extraction per text unit
Consider long-context prompting and skip retrieval entirely.Cost driver: per-query tokens
Note the ordering: the questions run cheapest-to-rule-out first. Most teams reach a terminal node before ever justifying a graph build.
What does a migration actually involve?
Moving from vector RAG to GraphRAG is an addition, not a replacement. You keep your embeddings.
- Keep the existing vector index running. It becomes the Basic Search path and continues serving most traffic unchanged.
- Tune extraction prompts on a corpus sample before indexing everything. Entity extraction quality on a 500-document sample predicts quality on 50,000.
- Index a representative subset first. Validate that community summaries are coherent and that entities resolve sensibly.
- Add the router last. Classify incoming questions and dispatch to the appropriate mode.
- Instrument mode selection. If global search is handling questions Basic Search could answer, your router is the bug, not the architecture.
Where does this comparison go wrong for most teams?
Three patterns, in rough order of how much money they waste.
Building a graph before fixing retrieval
Most “RAG isn’t working” complaints trace to chunking and ranking, not to architecture. Hybrid search and a cross-encoder reranker are cheap, fast to deploy, and often close the gap entirely.
GraphRAG cannot fix a retrieval pipeline that returns the wrong chunks, because it still retrieves. Garbage extraction produces a garbage graph.
Running global search on local questions
Global search fans out across community summaries. Pointing it at “what is the renewal date on contract 4471” burns an entire map-reduce pass to answer something Basic Search resolves in one hop.
Without a router, this happens by default, and it happens to the majority of queries.
Skipping prompt tuning
The GraphRAG documentation states plainly that using it out of the box “may not yield the best possible results” and strongly recommends tuning extraction prompts to your domain.
Teams routinely skip this, get mediocre entity extraction, and conclude the method does not work. The graph is only ever as good as the extraction that built it, and generic prompts produce generic entities.
How do you benchmark this on your own corpus?
Published benchmarks will not tell you what to do, because your corpus and question mix are not theirs. Run this instead. It takes about a day.
- Sample 100 real user questions from logs. Not invented ones, which skew toward the interesting cases.
- Label each by answer location: in one passage, spread across documents that co-occur, or a property of the corpus with no single home.
- Count the third category. If it is under 10% of traffic, stop. Improve your vector pipeline and revisit in six months.
- For the second category, check co-occurrence manually. If both endpoints appear in one document, a reranker will likely handle it.
- Index 5% of your corpus with GraphRAG and run only the third-category questions against it.
- Multiply the observed indexing cost by 20 for a full-corpus estimate, then double it for the first year of re-indexing.
Step 3 ends the project for a surprising share of teams, which is exactly what a good benchmark should do.
What do experienced teams do differently?
They route, rather than choose.
A lightweight classifier sits in front of retrieval and sends factoid queries to vector search and sensemaking queries to global search. This is precisely why GraphRAG bundles Basic Search: the production answer is usually both, with a router deciding per request.
They also index selectively. Rather than graphing an entire corpus, they graph the subset carrying entity relationships worth traversing and leave the long tail in a plain vector index. A contracts archive gets a graph; the meeting notes do not.
The habit underneath both: they treat the graph as a specialised index serving a minority of queries, not as the system of record.
The case for neither
If your corpus fits inside a modern context window, retrieval may be an unnecessary layer.
Long-context prompting removes chunking, embedding, index maintenance and retrieval failure in a single move, at the cost of per-query tokens. No index to build, no index to invalidate, no retrieval to debug.
The crossover depends on how often you query the same corpus. Retrieval amortises a one-off index across many queries. Long-context pays per query and skips the index entirely. Run that arithmetic before building either pipeline, because for small corpora with low query volume, neither RAG architecture is the right answer.
A short glossary
- TextUnit
- GraphRAG’s analysable slice of source text, which also determines how precisely outputs can cite their origin.
- Community
- A cluster of closely related entities identified by hierarchical Leiden clustering over the extracted graph.
- Community summary
- A pre-generated description of a community, produced bottom-up at index time and read by global search at query time.
- Global sensemaking question
- A question whose answer is a property of the whole corpus rather than the content of any single passage.
- Baseline RAG
- Microsoft’s term for conventional retrieval using vector similarity over embedded chunks.
- Map-reduce answering
- Global search’s pattern of generating a partial response from each community summary, then summarising all partial responses into one final answer.
Key takeaways
- Vector RAG retrieves passages; GraphRAG retrieves structure built from entities, relationships and community summaries.
- Microsoft’s published GraphRAG result covers global sensemaking questions on corpora in the 1 million token range, scored on comprehensiveness and diversity, not general accuracy.
- GraphRAG indexing requires an LLM extraction call per text unit, shifting indexing from an embedding cost to an inference cost.
- Incremental updates are cheap in a vector index and potentially whole-graph in GraphRAG, because Leiden clustering operates globally.
- GraphRAG ships four query modes, one of which is ordinary top-k vector search, which reveals how its designers expect it to be used.
- Fix chunking, hybrid search and reranking before concluding you need a graph.
- Production systems usually route between both architectures rather than picking one.
Frequently asked questions
Is GraphRAG more accurate than vector RAG?
Not as a general claim. The Microsoft Research paper reports gains in comprehensiveness and diversity on global sensemaking questions over roughly 1 million token corpora. For targeted factoid retrieval, no published evidence supports GraphRAG being more accurate, and vector search is faster and cheaper.
How much does GraphRAG cost to index?
No official per-token figure exists, because it depends on your model, corpus and community granularity. The structural driver is that GraphRAG makes at least one LLM extraction call per text unit, plus summarisation calls per community, where vector RAG makes only embedding calls.
Can I use GraphRAG with an existing vector database?
Yes. GraphRAG produces entities, relationships, claims and community summaries as outputs, and those artefacts can be embedded and stored alongside your existing chunks. Many teams run both indexes over the same corpus and route queries between them.
What is DRIFT search in GraphRAG?
DRIFT search answers questions about specific entities by fanning out to their neighbours, like local search, but adds community-level context to the retrieval. It sits between local search, which is narrow, and global search, which reads community summaries across the whole corpus.
Does GraphRAG work on constantly changing data?
Poorly, relative to vector search. Because hierarchical Leiden clustering is computed over the whole graph, adding documents can change community boundaries and invalidate existing summaries. Corpora that churn daily are a weak fit; bounded reference corpora are a strong one.
Do I need GraphRAG if I have a knowledge graph already?
Not necessarily. GraphRAG’s contribution is building a graph from unstructured text with an LLM, then summarising communities within it. If you already maintain a curated graph, you can bring your own and use GraphRAG’s query layer over it.
What is the difference between global search and local search?
Global search reads pre-generated community summaries to answer questions about the corpus as a whole, using a map-reduce pass. Local search starts from specific entities and fans out to their neighbours. Global costs more and suits thematic questions; local is cheaper and suits entity-specific ones.
Should I use GraphRAG for a customer support chatbot?
Usually not. Support questions are overwhelmingly factoid lookups against a knowledge base that changes often, which is the profile vector RAG handles best and GraphRAG handles worst. Spend the budget on retrieval quality, hybrid search and reranking instead.
References
- Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., and Larson, J. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130, submitted 24 April 2024, revised 19 February 2025.
- Microsoft. GraphRAG documentation, covering indexing architecture and the Global, Local, DRIFT and Basic query modes.
- Microsoft Research. GraphRAG: Unlocking LLM discovery on narrative private data.
- Traag, V. A., Waltman, L., and van Eck, N. J. From Louvain to Leiden: guaranteeing well-connected communities. arXiv:1810.08473, 2018.
