A Butterfly Beats its Wings Over the Pacific

🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.

The fundamental beauty of modern AI is its generality as a tool.

• Reinforcement learning: a loop where the agent observes the state of its environment and then takes an action that alters that state. Gets rewarded for it. The agent learns to maximize its expected reward.

• RL was tedious because it was very stack-dependent. Was the problem with your Atari-bot the algo or the emulation environment you jury rigged on the Linux server and speeded up 1 million fold?

• OpenAI created the Gym environment manager in 2016 and handed it off to the Farama open-source foundation in 2022.

Researchers using Gym just solved a major problem in fusion plasma instability.

• Fusion is the nuclear reaction that happens in the sun: two light atoms combine into a heavier one, with the release of large amounts of energy and no radioactive byproducts

• Tokamaks fusion reactors heat hydrogen gas to form plasma at temperatures exceeding the sun’s core. This plasma is controlled using the effect of the magnetic fields on the free-moving electrons and ions.

• Tearing instabilities caused by magnetic field irregularities within the plasma can end the fusion reaction, preventing a stable supply of power.

• It’s a nonlinear dynamic system, so far impossible to predict using traditional numerical methods.

How did they do it?

They:

• built an OODA (Observe-Orient-Decide-Act) looped agent;

• that was given a reward function that balances the conflicting objectives of maximizing plasma pressure (indicative of reactor performance) and minimizing the risk of instability;

• that could control DIII-D in San Diego, the largest US magnetic fusion tokamak

• by controlling DIII-D’s actuators in 25-millisecond intervals

What were the results?

• The AI controller successfully kept the probability of tearing instabilities below a critical threshold, ensuring a stable plasma

• It passed stress tests such as low safety factor and low torque

• Dynamic control of plasma using the AI controller was achieved

• The AI controller kept the plasma in H-mode performance, a highly efficient operational mode for tokamaks, which is often difficult to achieve and sustain with conventional, preprogrammed control systems

• Milestone for plasma control and performance which can be used in fusion projects like ITER immediately

More and more, neural networks plus massive compute look like a big civilizational unlock that can be applied everywhere.

Become a subscriber for daily breakdowns of what’s happening in the AI world: