AGI has been achieved: Prompt engineering edition

Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.

The team at codium.ai introduces what they call a “flow engineering” solution to computer code generation, which in essence, uses up to 100 calls to GPT-4 per coding problem to:

1. Reason about the goal, inputs, outputs and constraints of the problem

2. Create tests, and reason why inputs leads to outputs in each test

3. Generate 2-3 potential solutions in English and rank them in terms of correctness, simplicity and robustness

4. Iteratively pick a solution, generate code and run tests. Repeating until all tests pass

This is exactly the Test Driven Development strategy taught to a generation of coders, and AlphaCodium achieved a 44% completion rate on the CodeContests evaluation, beating Google’s well funded fine tuned AlphaCode2 at 43%. Both would outrank more than 85% of all human competitors.

To reiterate, with some heavily structured prompting, your machine can beat (most) humans at writing computer code by deploying the exact same strategies humans use to write good code.

And no, it’s still not AGI.

Become a subscriber for daily breakdowns of what’s happening in the AI world: