- Emergent Behavior
- Posts
- Repeat the Question
Repeat the Question
Asking questions of models and Sam
đź”· Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.
Here’s today at a glance:
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/0e0158b6-b9cc-47dd-91d1-e485f8582d9b/Screenshot_2024-01-22_at_2.13.39_PM.png?t=1705950837)
đź“ť Just Keep Asking
This research paper from the Tencent China team tries to find out whether just organizing the existing AI models we have into teams of agents that work together on a problem improves their performance, ie essence squeezing greater intelligence from a crowd of similar intelligences.
Paper Title: More Agents Is All You Need
Who: Team from Tencent China
What did they study:
Does increasing the number of AI agents improve their performance on a task?
In essence, does brute force scaling of AI agents work?
How did they do it?
Create agents that in perform language model requests in two phases
In query phase, the input is iteratively fed into the same LLM or multiple LLMs to get multiple outputs
In the voting phase, a majority vote is taken on the best response
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/049fea04-a34e-4ee1-bdcd-794d2bcdbb53/Screenshot_2024-02-10_at_1.47.52_PM.png?t=1707601688)
What did they find?
Accuracy increases with the number of agents, notably at an ensemble size of 15 (either making 15 iterative calls to the same model or having multiple agents work on the same query), a Llama2 can match GPT3.5 single query, and a GPT3.5 can match a GPT4 single query.
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/66e9e9e9-b8d5-4caa-89b7-e9346e64f144/Screenshot_2024-02-10_at_1.49.24_PM.png?t=1707601803)
Adding agents to the ensemble increased accuracy in arithmetic reasoning
Performance gains increase initially with increasing difficulty but then hit a ceiling and then decline as problem complexity overwhelms the model
Gains increase with the number of steps or iterations
What are the implications?
You can increase intelligence with the same AI model just by asking the same question over and over iteratively
This technique can improve raw performance for all AI/LLMs
Brute force scaling works to a point but hits a ceiling
🌠Enjoying this edition of Emergent Behavior? Send this web link with a friend to help spread the word of technological progress and positive AI to the world!
Or send them the below subscription link:
đź’Ą Sam Altman Unleashed
One of the greatest pleasures in seeing millennials found and run companies is the final falling away of the facade of fakeness that is expected of all businesspeople. OpenAI’s Sam Altman did not disappoint recently:
Just before the Wall Street Journal report on the $7 trillion fundraising, possibly in response to a request for comment:
we believe the world needs more ai infrastructure--fab capacity, energy, datacenters, etc--than people are currently planning to build.
building massive-scale ai infrastructure, and a resilient supply chain, is crucial to economic competitiveness.
openai will try to help!
— Sam Altman (@sama)
6:17 PM • Feb 7, 2024
Shortly thereafter, in response to his brother, Jack, who had complained of being forever titled “brother of Sam Altman,”
mind your own business and let us cook
— Sam Altman (@sama)
8:25 PM • Feb 9, 2024
In response to why he’s suddenly posting
just a WFH friday and feeling a little bored
— Sam Altman (@sama)
10:54 PM • Feb 9, 2024
How many GPUs can you buy with $7 trillion?
@pbeisel probably a lot of fucking gpus idk why?
— Sam Altman (@sama)
10:58 PM • Feb 9, 2024
Was this a burn or just friendly banter? Alex Wang is the founder and CEO of ScaleAI, which does a lot of the human feedback training for large AI companies. Scale’s revenue tripled last year… off the backs of firms like OpenAI.
this is authoritative, truly no one flies in and out to more parties than you, it looks like a full time job
— Sam Altman (@sama)
9:02 PM • Feb 10, 2024
Alternate uses of trillions
obviously we need the ai to find the aliens bro
— Sam Altman (@sama)
9:29 PM • Feb 10, 2024
More banter with brother Jack
@ me next time
— Sam Altman (@sama)
8:59 PM • Feb 10, 2024
which continued
round is very tight but we have huge respect for you and i think we can look at a .002143% allocation for you if you can decide today, unfortunately we won’t be able to make the data room available
— Sam Altman (@sama)
9:39 PM • Feb 10, 2024
Of course, it was not all play
quite a lot, actually
— Sam Altman (@sama)
9:49 PM • Feb 10, 2024
The inside joke below being that Roon (@tszzl), an OpenAI employee, had disappeared from Twitter for 3 days after a series of angsty adolescent tweets, leaving his 100k+ followers lost
Mr Beast, probably the most famous and richest Youtuber of all time, hijacking the thread to make a plea for funding below
I was thinking, if you get that 7 trillion dollars you could give me like .01% of that and it’d fund my videos for a long time. Thx for considering
— MrBeast (@MrBeast)
5:31 AM • Feb 11, 2024
🗞️ Things Happen
“Researchers built a smartwatch heartbeat monitor that uses a slime mold for its operation. The slime mold has to be fed and cared for, so the users in the trial... developed an emotional attachment to it“, “she started calling me, you’re my little slime“ - so good
Attention is really all you need, was from Nobel Laureate Herbert Simon in 1973: "The information-processing systems of our contemporary world swim in an exceedingly rich soup of information. In a world of this kind, the scarce resource is not information; it is processing capacity to attend to information. Attention is the chief bottleneck in organizational activity, and the bottleneck becomes narrower and narrower as we move to the top of organizations."
🖼️ AI Artwork Of The Day
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/893ebde9-9199-470e-88d4-73f3beffdf1b/GF2MF4ZbYAAmyX7.jpeg?t=1707600962)
Finishing Blow - @henrydaubrez on X
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/0e0158b6-b9cc-47dd-91d1-e485f8582d9b/Screenshot_2024-01-22_at_2.13.39_PM.png?t=1705950837)
That’s it for today! Become a subscriber for daily breakdowns of what’s happening in the AI world:
Reply