RAG vs CAG vs KAG

A language model only knows what it was trained on. To answer from your documents, it can look a few passages up (RAG), read them all first (CAG), or follow facts linked in a graph (KAG).

Assembling the machine…

Bench

  1. RAG
    Open book: looks up 3 passages—
  2. CAG
    Reads the whole book first—
  3. KAG
    A map of facts, link by link—

Step 1 / 5

A model alone

Alone, a model never saw your documents: it cannot know who supplies P-7.

Event log · 1
  1. 0.00 msLLMnever saw the documents: “I don't know”, or a made-up supplier

The concept

A language model only knows what it was trained on. To answer from your documents, it can look a few passages up (RAG), read them all first (CAG), or follow facts linked in a graph (KAG).

RAG, CAG and KAG are three ways of giving a large language model knowledge it was not trained on. Retrieval-augmented generation (RAG) looks passages up when a question arrives; cache-augmented generation (CAG) has the model read all the knowledge ahead of time and keeps its state; knowledge-augmented generation (KAG) builds a knowledge graph and plans the question over it. This bench sends the same documents and the same question through all three.

RAG, introduced by Patrick Lewis and colleagues in 2020, splits the documents into passages, stores one vector per passage, and retrieves the passages closest to each question. In the original paper the generator reads the question with each retrieved passage and combines the results. The index can be replaced without retraining the model, which is how its knowledge is updated.

CAG, proposed by Brian Chan and colleagues in 2024, drops retrieval when the knowledge is small enough. The whole corpus is fed to the model once, and the resulting KV cache is kept and reused for every question, cut back to the corpus after each answer. It removes the retrieval step, not generation: in their test with an 8-billion-parameter model, answering took 0.85 s with a 21,000-token corpus and 2.26 s with 85,000. The authors note it becomes impractical for much larger datasets.

KAG, from Lei Liang and colleagues in 2024, has a model extract entities and relations from the documents into a graph whose nodes point back to the passages they came from. A planner rewrites each question as logical-form steps, retrieves through the graph and the passages, and checks whether the question is answered. On multi-hop benchmarks the authors report a higher F1 than a multi-step retrieval baseline, at the price of several model calls to build and to answer; errors made while building the graph lead to wrong answers.

None of the three wins everywhere. CAG needs the corpus to fit in the context window. Retrieval in one step returns passages similar to the question and can miss the one that leads to the answer. KAG's answers are only as right as the graph it extracted. Each switch here triggers one of these limits.

The run, step by step

Every scenario, in words.

What the machine plays, step by step. Beside a step, what a what-if switch plays in its place, or adds right after it.

Ask about a part

5 steps
  1. Play

    A model alone

    The analyst first asks a language model on its own: “Who supplies part P-7, the brake pads?” It was trained on public text, never on these orders, catalogues and supplier sheets. It can only say it does not know, or make up a supplier that sounds right. Each system below hands it the documents another way.

    • With Ask what takes two documents

      Play

      A model alone

      The analyst first asks a language model on its own: “Where is the supplier of part P-7, the brake pads, based?” It was trained on public text, never on these orders, catalogues and supplier sheets. It can only say it does not know, or make up a supplier that sounds right. Each system below hands it the documents another way.

  2. Play

    Three ways to hand over the documents

    Each system prepares the same 40 documents (orders, a parts catalogue, supplier sheets) before any question. RAG cuts them into chunks and stores one vector per chunk. CAG runs the model once over all of them and keeps its KV cache. KAG has a model extract entities and relations into a graph, each linked to the passage it came from.

    • With Give them 40,000 documents

      Play

      Three ways to hand over the documents

      Each system prepares the same 40,000 documents (orders, a parts catalogue, supplier sheets) before any question. RAG cuts them into chunks and stores one vector per chunk. CAG cannot: so many documents run to millions of tokens, and the whole corpus has to fit in the model's context window. KAG has a model extract entities and relations into a graph, each linked to the passage it came from.

    • With Get one fact wrong in the graph

      Play

      Three ways to hand over the documents

      Each system prepares the same 40 documents (orders, a parts catalogue, supplier sheets) before any question. RAG cuts them into chunks and stores one vector per chunk. CAG runs the model once over all of them and keeps its KV cache. KAG has a model extract entities and relations into a graph, and one relation comes out wrong: P-7 supplied by S-9.

    • With Give them 40,000 documents + Get one fact wrong in the graph

      Play

      Three ways to hand over the documents

      Each system prepares the same 40,000 documents (orders, a parts catalogue, supplier sheets) before any question. RAG cuts them into chunks and stores one vector per chunk. CAG cannot: so many documents run to millions of tokens, and the whole corpus has to fit in the model's context window. KAG has a model extract entities and relations into a graph, and one relation comes out wrong: P-7 supplied by S-9.

  3. Play

    One question, three systems

    An analyst asks: “Who supplies part P-7, the brake pads?” The same question goes to all three.

    • With Give them 40,000 documents

      Play

      One question, three systems

      An analyst asks: “Who supplies part P-7, the brake pads?” The question goes to RAG and KAG. CAG has no cache to answer from.

    • With Ask what takes two documents

      Play

      One question, three systems

      An analyst asks: “Where is the supplier of part P-7, the brake pads, based?” The same question goes to all three.

    • With Give them 40,000 documents + Ask what takes two documents

      Play

      One question, three systems

      An analyst asks: “Where is the supplier of part P-7, the brake pads, based?” The question goes to RAG and KAG. CAG has no cache to answer from.

  4. Play

    Find what to read

    RAG embeds the question and takes the 3 chunks closest to it. One says S-12 supplies P-7. CAG looks nothing up: the question is appended to the cached corpus. KAG's planner writes one step, the supplier of P-7, and runs it on the graph: S-12.

    • With Give them 40,000 documents

      Play

      Find what to read

      RAG embeds the question and takes the 3 chunks closest to it. One says S-12 supplies P-7. KAG's planner writes one step, the supplier of P-7, and runs it on the graph: S-12.

    • With Ask what takes two documents

      Play

      Find what to read

      RAG takes the 3 chunks closest to the question: all three are about P-7 and its supplier. The sheet that says where S-12 is based never mentions P-7, so it is not among them. CAG looks nothing up: the question is appended to the cached corpus. KAG's planner writes two steps, the supplier of P-7, then that supplier's city, and runs them on the graph: S-12, then Lyon.

    • With Give them 40,000 documents + Ask what takes two documents

      Play

      Find what to read

      RAG takes the 3 chunks closest to the question: all three are about P-7 and its supplier. The sheet that says where S-12 is based never mentions P-7, so it is not among them. KAG's planner writes two steps, the supplier of P-7, then that supplier's city, and runs them on the graph: S-12, then Lyon.

    • With Get one fact wrong in the graph

      Play

      Find what to read

      RAG embeds the question and takes the 3 chunks closest to it. One says S-12 supplies P-7. CAG looks nothing up: the question is appended to the cached corpus. KAG's planner writes one step, the supplier of P-7, and runs it on the graph: S-9.

    • With Give them 40,000 documents + Get one fact wrong in the graph

      Play

      Find what to read

      RAG embeds the question and takes the 3 chunks closest to it. One says S-12 supplies P-7. KAG's planner writes one step, the supplier of P-7, and runs it on the graph: S-9.

    • With Ask what takes two documents + Get one fact wrong in the graph

      Play

      Find what to read

      RAG takes the 3 chunks closest to the question: all three are about P-7 and its supplier. The sheet that says where S-12 is based never mentions P-7, so it is not among them. CAG looks nothing up: the question is appended to the cached corpus. KAG's planner writes two steps, the supplier of P-7, then that supplier's city, and runs them on the graph: S-9, then Turin.

    • With Give them 40,000 documents + Ask what takes two documents + Get one fact wrong in the graph

      Play

      Find what to read

      RAG takes the 3 chunks closest to the question: all three are about P-7 and its supplier. The sheet that says where S-12 is based never mentions P-7, so it is not among them. KAG's planner writes two steps, the supplier of P-7, then that supplier's city, and runs them on the graph: S-9, then Turin.

  5. Play

    Three answers

    RAG answers S-12, from the question and its 3 chunks. CAG answers S-12, from the whole corpus in its cache; the question is then cut off the cache, ready for the next one. KAG answers S-12, from the steps its planner ran on the graph.

    • With Give them 40,000 documents

      Play

      Two answers

      RAG answers S-12, from the question and its 3 chunks. CAG gives no answer: it never had a cache. KAG answers S-12, from the steps its planner ran on the graph.

    • With Ask what takes two documents

      Play

      Three answers

      RAG names S-12 but cannot say where it is based: no passage it was given says so. CAG answers Lyon, from the whole corpus in its cache; the question is then cut off the cache, ready for the next one. KAG answers Lyon, from the steps its planner ran on the graph.

    • With Give them 40,000 documents + Ask what takes two documents

      Play

      Two answers

      RAG names S-12 but cannot say where it is based: no passage it was given says so. CAG gives no answer: it never had a cache. KAG answers Lyon, from the steps its planner ran on the graph.

    • With Get one fact wrong in the graph

      Play

      Three answers

      RAG answers S-12, from the question and its 3 chunks. CAG answers S-12, from the whole corpus in its cache; the question is then cut off the cache, ready for the next one. KAG answers S-9, as confidently as a right answer: the mistake was made when the graph was built.

    • With Give them 40,000 documents + Get one fact wrong in the graph

      Play

      Two answers

      RAG answers S-12, from the question and its 3 chunks. CAG gives no answer: it never had a cache. KAG answers S-9, as confidently as a right answer: the mistake was made when the graph was built.

    • With Ask what takes two documents + Get one fact wrong in the graph

      Play

      Three answers

      RAG names S-12 but cannot say where it is based: no passage it was given says so. CAG answers Lyon, from the whole corpus in its cache; the question is then cut off the cache, ready for the next one. KAG answers Turin, as confidently as a right answer: the mistake was made when the graph was built.

    • With Give them 40,000 documents + Ask what takes two documents + Get one fact wrong in the graph

      Play

      Two answers

      RAG names S-12 but cannot say where it is based: no passage it was given says so. CAG gives no answer: it never had a cache. KAG answers Turin, as confidently as a right answer: the mistake was made when the graph was built.