The concept
A language model only knows what it was trained on. To answer from your documents, it can look a few passages up (RAG), read them all first (CAG), or follow facts linked in a graph (KAG).
RAG, CAG and KAG are three ways of giving a large language model knowledge it was not trained on. Retrieval-augmented generation (RAG) looks passages up when a question arrives; cache-augmented generation (CAG) has the model read all the knowledge ahead of time and keeps its state; knowledge-augmented generation (KAG) builds a knowledge graph and plans the question over it. This bench sends the same documents and the same question through all three.
RAG, introduced by Patrick Lewis and colleagues in 2020, splits the documents into passages, stores one vector per passage, and retrieves the passages closest to each question. In the original paper the generator reads the question with each retrieved passage and combines the results. The index can be replaced without retraining the model, which is how its knowledge is updated.
CAG, proposed by Brian Chan and colleagues in 2024, drops retrieval when the knowledge is small enough. The whole corpus is fed to the model once, and the resulting KV cache is kept and reused for every question, cut back to the corpus after each answer. It removes the retrieval step, not generation: in their test with an 8-billion-parameter model, answering took 0.85 s with a 21,000-token corpus and 2.26 s with 85,000. The authors note it becomes impractical for much larger datasets.
KAG, from Lei Liang and colleagues in 2024, has a model extract entities and relations from the documents into a graph whose nodes point back to the passages they came from. A planner rewrites each question as logical-form steps, retrieves through the graph and the passages, and checks whether the question is answered. On multi-hop benchmarks the authors report a higher F1 than a multi-step retrieval baseline, at the price of several model calls to build and to answer; errors made while building the graph lead to wrong answers.
None of the three wins everywhere. CAG needs the corpus to fit in the context window. Retrieval in one step returns passages similar to the question and can miss the one that leads to the answer. KAG's answers are only as right as the graph it extracted. Each switch here triggers one of these limits.