Just Keep Asking

🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.

This research paper from the Tencent China team tries to find out whether just organizing the existing AI models we have into teams of agents that work together on a problem improves their performance, i.e., squeezing greater intelligence from a crowd of similar intelligences.

Paper Title: More Agents Is All You Need

Who: Team from Tencent China

What did they study:

• Does increasing the number of AI agents improve their performance on a task?

• In essence, does brute force scaling of AI agents work?

How did they do it?

• Create agents that in perform language model requests in two phases

  • In query phase, the input is iteratively fed into the same LLM or multiple LLMs to get multiple outputs

  • In the voting phase, a majority vote is taken on the best response

What did they find?

• Accuracy increases with the number of agents, notably at an ensemble size of 15 (either making 15 iterative calls to the same model or having multiple agents work on the same query), a Llama2 can match GPT3.5 single query, and a GPT3.5 can match a GPT4 single query.

• Performance gains increase initially with increasing difficulty but then hit a ceiling and then decline as problem complexity overwhelms the model

• Gains increase with the number of steps or iterations

What are the implications?

• You can increase intelligence with the same AI model just by asking the same question over and over iteratively

• This technique can improve raw performance for all AI/LLMs

• Brute force scaling works to a point but hits a ceiling

Become a subscriber for daily breakdowns of what’s happening in the AI world: