- Emergent Behavior
- Posts
- 2024-04-01: Coding Agents Tackle Github Issues
2024-04-01: Coding Agents Tackle Github Issues
Fast Approaching Clff for Computer Science
🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.
Here’s today at a glance:
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/0e0158b6-b9cc-47dd-91d1-e485f8582d9b/Screenshot_2024-01-22_at_2.13.39_PM.png?t=1705950837)
🗃️ Coding Agents Tackle Github Issues
Who: Fudan University et al.
What: An AI agent that can automatically resolve Github issues that require code changes across an entire repository
How:
MAGIS consists of four types of AI agents that collaborate:
Manager: Plans the resolution and assembles a team of developer agents
Repository Custodian: Locates code files relevant to the issue
Developers: Modify the code based on the plan
Quality Assurance (QA): Reviews and validates the code changes
The agents work together to analyze issues, find relevant files, create a plan, implement code changes, and verify the modifications. The inspiration comes from Github Flow, a commonly used workflow paradigm.
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/c71af7b9-32a2-4710-b3f9-316ffad21c84/Screenshot_2024-04-01_at_4.41.17_PM.png?t=1712014957)
MAGIS
The research group:
Evaluated MAGIS on the SWE-bench dataset of real GitHub issues and compared it to LLMs like GPT-3.5, GPT-4, and Claude-2
MAGIS successfully resolved 13.94% of issues, an 8-fold improvement over applying GPT-4 directly, demonstrating the effectiveness of the multi-agent approach
Ablation studies showed the QA agent and ability to utilize information from comments also boosted performance
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/f72bf19c-a0e8-4783-9447-856847725c7b/Screenshot_2024-04-01_at_4.43.13_PM.png?t=1712015012)
MAGIS marks a significant step towards using AI to automatically resolve complex software issues that span multiple files and require understanding the relationships between code components.
If we are at 14% now... we will probably fully solve coding within 24 months at this rate. Ridiculous.
🌠 Enjoying this edition of Emergent Behavior? Send this web link with a friend to help spread the word of technological progress and positive AI to the world!
Or send them the below subscription link:
🗞️ Things Happen
Cryptic post from the head of Tesla’s self-driving team
It is the beginning of the end!
— Ashok Elluswamy (@aelluswamy)
5:41 AM • Apr 1, 2024
🖼️ AI Artwork Of The Day
Rooms in the style of planets of the solar system - u/requiredpaper from r/midjourney
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/0e0158b6-b9cc-47dd-91d1-e485f8582d9b/Screenshot_2024-01-22_at_2.13.39_PM.png?t=1705950837)
That’s it for today! Become a subscriber for daily breakdowns of what’s happening in the AI world:
Reply