The Framework of Knowledge

🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.

What Fortune 500 companies are really spending money on AI for right now:

• format their existing documents

• search through them

• explain them using a language model

This simplistic process goes by the rather clunky moniker of Retrieval Augmented Generation but has become very attractive as the search AI (embeddings) seems to understand context much better than simple keywords.

The main issue with this methodology is the key question of how many documents get fed to the language model to explain. This paper from Stanford provides a solution that beats state of the art, namely just repeatedly summarize small chunks of documents until the main themes are stored in the top layer. Notably, it ends up clustering the smaller chunks by meaning (semantic similarity).

And then when answering, performs a tree retrieval to assemble context.

And this framework is relatively agnostic to which language model is used…after all the LLM is just an understanding engine applied to context, which is what is really important.

A truly elegant solution. Although I must admit, I’ve heard of more than one firm already using recursive summarization in production, so I doubt this is really new knowledge, rather than just proving what we already know to be true.

Become a subscriber for daily breakdowns of what’s happening in the AI world: