A groundbreaking technology has been used to improve another, as researchers have demonstrated how AI could be used to control the superheated plasma inside a tokamak-type fusion reactor.
“This is one of the most challenging applications of reinforcement learning to a real-world system,” says Martin Riedmiller, a researcher at DeepMind.
Current nuclear plants use nuclear fission to harness energy, forcing larger atoms to split into two smaller atoms. Fusion, on the other hand, is the opposite process. In nuclear fusion, two or more atomic nuclei combine to form one or more larger atoms. It’s the process that powers stars, but harnessing this power and using it on Earth is extremely challenging.
If you’re essentially building a miniature star (hotter than the surface of the Sun) and then using it to harness its power, you need to be absolutely certain you can control it. Researchers use a lot of tricks to achieve this, like magnets, lasers, and clever designs, but it’s still proven to be a gargantuan challenge.
This is where AI could enter the stage.
Researchers use several designs to try and contain this superheated plasma — one of these designs is called a tokamak. A tokamak uses magnetic fields in a donut-shaped containment area to keep the superheated atoms (as plasma) under control long enough that we can extract energy from it. The main idea is to use this magnetic cage to keep the plasma from touching the reactor walls, which would damage the reactor and cool the plasma.
Controlling this plasma requires constant shifts in the magnetic field, and the researchers at DeepMind (the Google-owned company that built the AlphaGo and AlphaZero AIs that dominated Go and chess) felt like this would be a good task for an algorithm.
They trained an unnamed AI to control and change the shape of the plasma by changing the magnetic field using a technique called reinforcement learning. Reinforcement learning is one of the three main machine learning approaches (alongside supervised learning and unsupervised learning). In reinforcement learning, the AI takes certain actions to maximize the chance of earning a predefined reward.
After the algorithm was trained on a virtual reactor, it was given control of the magnets inside the Variable Configuration Tokamak (TCV), an experimental tokamak reactor in Lausanne, Switzerland.
The AI-controlled the plasma for only two seconds, but this is as much as the TCV can go without overheating — and it was a long enough period to assess the AI’s performance.
Every 0.0001 seconds, the AI took 90 different measurements describing the shape and location of the plasma, adjusting the magnetic field accordingly. To speed the process up, the AI was split into two different networks — a large network that learned via trial and error in the virtual stage, and a faster, smaller network that runs on the reactor itself.
“Our controller first shapes the plasma according to the requested shape, then shifts the plasma downward and detaches it from the walls, suspending it in the middle of the vessel on two legs. The plasma is held stationary, as would be needed to measure plasma properties. Then, finally the plasma is steered back to the top of the vessel and safely destroyed,” DeepMind explains in a blog post.
“We then created a range of plasma shapes being studied by plasma physicists for their usefulness in generating energy. For example, we made a “snowflake” shape with many “legs” that could help reduce the cost of cooling by spreading the exhaust energy to different contact points on the vessel walls. We also demonstrated a shape close to the proposal for ITER, the next-generation tokamak under construction, as EPFL was conducting experiments to predict the behaviorr of plasmas in ITER. We even did something that had never been done in TCV before by stabilizing a “droplet” where there are two plasmas inside the vessel simultaneously. Our single system was able to find controllers for all of these different conditions. We simply changed the goal we requested, and our algorithm autonomously found an appropriate controller.”
While this is still in its early stages, it’s a very promising achievement. DeepMind’s AIs seem ready to move on from complex games into the real world, and make a real difference — as they previously did with protein structure.
This doesn’t mean that we’ll have nuclear fusion tomorrow. Although we’ve seen spectacular breakthroughs in the past couple of years, and although AI seems to be a promising tool, we’re still a few steps away from realistic fusion energy. But the prospect of virtually limitless fusion energy, once thought to be technically impossible, now seems within our reach.
The study was published in Nature.
