Humans tried to teach AI how to make a little sun, and the result was -

Recently, there was a big news in the field of nuclear science. Even the scientists who achieved this achievement found it incredible - amazing!

"I've been waiting for this moment for a long time - the first demonstration of deep reinforcement learning in nuclear fusion research!" David Pfau, a scientist from DeepMind, excitedly shared his paper.
Source: Synced

I don't know how many friends have seen Iron Man in the Avengers. The reason why he is so awesome is that there is a "little sun" in the center of his armor, which continuously provides him with a lot of energy. This little sun is none other than the protagonist we are going to talk about today - controlled nuclear fusion. Although I don't know how Iron Man in the Marvel Universe achieves this controlled nuclear fusion, so far, in the human world, it is still just a dream.

"Iron Man 3" poster source: www.sfs-cn.com

01

What is controlled nuclear fusion?

Just as exothermic chemical reactions between molecules can generate energy, exothermic nuclear reactions between atomic nuclei (or neutrons) can also generate energy, and this energy is much greater than the energy generated by chemical reactions. In the current controlled nuclear fusion scheme, people use the deuterium-tritium reaction:

Schematic diagram of nuclear fusion reaction (Deuterium and Tritium react to produce Neutrons and Helium, and release energy)

Source: Reference [2]

In the fusion reaction of deuterium nuclei and tritium nuclei, a large amount of energy is released, but it is not easy to make deuterium nuclei and tritium nuclei collide with each other.

The first problem is that both deuterium and tritium are positively charged and will capture electrons to form atoms in nature. In order to make deuterium and tritium react, these electrons need to be stripped away.

Not only that, the nucleus is very small. If an atom is regarded as a football field, then the nucleus is equivalent to a ping-pong ball in the football field. It is certainly extremely difficult to make two ping-pong balls collide in such a large space.

Scientists have adopted a miraculous method to solve two problems at once - that is, heating and pressurization. Heating can first separate the electrons and nuclei in the atoms to form plasma, which solves the first problem and also increases the kinetic energy of the nuclei, which is the speed of the ping-pong ball; and pressurization is to reduce the distance between the nuclei, that is, to reduce the size of the "football field". This can greatly increase the probability of nuclei colliding. So to what extent should we heat it? Let's take the sun as a reference. The temperature at the center of the sun is 15 million degrees Celsius and the pressure reaches 300 billion times the atmospheric pressure. It is difficult for us to achieve such a pressure, so we can only use a higher temperature. The operating temperature of my country's EAST (Fully Superconducting Tokamak Nuclear Fusion Experimental Device) is as high as 100 million degrees.

But this also raises a new problem: how can such a small sun be contained? To address this problem, people have proposed various controlled nuclear fusion solutions, one of the more mainstream solutions is to use the Lorentz force for magnetic confinement. The Tokamak device is a magnetic confinement device.

Schematic diagram of a tokamak device (Coils: coils, Plasma: plasma)

Source: Reference [2]

Its main structure is a ring with a vacuum inside. There are multiple coils around the ring, which can provide a magnetic field along the tangent direction of the ring; there are also multiple coils in the center of the ring, which provide a magnetic field perpendicular to the plane of the ring and an induced electric field along the tangent direction of the ring. Under such an electromagnetic field, plasma can be confined in the ring and can be ohm-heated (materials with resistance will generate heat when current passes through them).

There are many benefits to achieving controlled nuclear fusion. On the one hand, the raw materials are abundant: the energy released by the deuterium extracted from each liter of seawater participating in the fusion reaction is equivalent to the energy released by 300 liters of gasoline; tritium can be produced by the fission of lithium atoms, and lithium exists in large quantities in the earth's crust and seawater. It can be said that once controlled nuclear fusion is achieved, humans will achieve "energy freedom" to a large extent. On the other hand, the product has little pollution. At present, the preferred solution for controlled nuclear fusion is the deuterium-tritium reaction, which only produces neutrons and helium-4 nuclei and a large amount of energy, and does not produce harmful gases and almost no radioactive pollution.

Moreover, most importantly, although both are nuclear energy, nuclear fusion is safer and more reliable than nuclear fission. In order to maintain nuclear fusion, the plasma must reach a temperature of hundreds of millions of degrees and maintain a certain density. Maintaining such a temperature and density requires extremely harsh conditions. If any of the conditions are missing, the nuclear fusion reaction will stop quickly. In addition, the plasma involved in nuclear fusion is in a vacuum, and its density is very low, which is several orders of magnitude smaller than the density of air. Therefore, there is no need to worry about the problem of these plasmas leaking and reacting with air and exploding.

However, it is not easy to get hold of this inexhaustible, clean and environmentally friendly energy. Sufficiently high temperature and pressure are necessary conditions for controlled nuclear fusion. According to previous analysis, the increase in temperature and pressure can increase the probability of atomic nuclei colliding with each other; but probability alone is not enough, we must also let such temperature and pressure last long enough, so as to increase the number of fusion reactions in an experiment, thereby increasing the total energy generated. We call this time the energy confinement time, which, together with pressure and temperature, constitutes the three elements for evaluating a controlled nuclear fusion.

02

What controlled nuclear fusion values ​​is actually...

Just as people believe that "appearance reflects the heart" and "appearance is justice", controlled nuclear fusion also attaches great importance to the "appearance" of plasma - we call it "configuration".

This is because the cross-sectional shape of plasma can affect various parameters of nuclear fusion. Scientists have found that there are three important parameters: average ion density, energy confinement time, and ion temperature. For a controlled nuclear fusion experiment, the larger the product of these three numbers, the better. Optimizing the cross-sectional shape of plasma can improve the energy confinement time among these three numbers. For 25 years, scientists have been committed to improving this product, and it has now increased by 4 orders of magnitude.

Plasma in a tokamak (left), and a schematic diagram of its cross-sectional configuration (right)

Source: Reference [1]

Although smart humans have mastered some of these rules and can infer parameters such as current and voltage based on the desired plasma configuration, it is just... a bit tiring. In simple terms, you can roughly create a control system based on these mastered rules - such as which sensor input data to read and how to respond to their changes; however, in a real nuclear fusion process, the energy configuration of the plasma always has some strange fluctuations, so there will be a process of mutual iteration of measurement and modeling, and the original system will be adjusted in real time before the final control system is generated. The controller generated in this way is not only hard to come by, but can only be used for the same plasma configuration. If you want to experiment with a completely different plasma structure in a tokamak, you will most likely have to overhaul the system.

Experimenting with a completely different plasma configuration in a tokamak, as if the client's father had made some minor changes↑

The scientist who wanted to overhaul the system put on a mask of pain↑

In fact, humans don’t want to work so hard. After all, isn’t the configuration itself more interesting than the specific method of achieving a certain configuration? So, let artificial intelligence (AI) learn such boring and brain-burning things as how to select configuration parameters!

AI: It’s time to change the world——

So, DeepMind, a star company that has challenged problems ranging from protein folding to StarCraft, teamed up with the Plasma Center of the Swiss Federal Institute of Technology in Lausanne to start the journey of taming the tokamak.

03

Reinforcement learning: I can do this question

first step

Setting goals

When learning something, of course the goal is the most important. The goal can include a variety of desired features, including but not limited to the position of the plasma, the basic stability of the current, the precise shape contour with a specified elongation and X-point position, etc. (as shown in the table below). These goals will be combined into a "reward function" to specifically punish control strategies that fail to achieve the goal (just like school rules and regulations). Although more than a dozen features are listed, the design of the reward function is indeed minimal, which can ensure that the reinforcement learning algorithm has maximum flexibility. (DeepMind: This is all the good intentions of the teacher!

The components of the reward function. Each component will have a target value and actual value, and many can change over time.

Source: Reference [1]

Step 2

Determine your strategy

The simulator generates data on the evolution of plasma states, and the reinforcement learning algorithm collects this data, continuously learns and accumulates experience, and then finds the "optimal control strategy" under the guidance of the reward function. (In fact, there is also a carefully designed "critic" algorithm specifically used for training. For details, please see the reference [1].)

Step 3

Practical Exercise

As shown in the figure, the Tokamak's "real-time control system" consists of two parts - one is the traditional controller, and the other is the control system trained by deep learning. In each experiment, after a standard plasma is generated, the first to appear is the traditional controller, which is used to maintain the position and total current of the plasma; when the plasma is stable, it is the turn of the deep learning control system to appear - it will accurately track the plasma through 92 types of measurement data 10,000 times per second, and adjust the 19 control coils of the reactor accordingly according to the previously learned strategy, and finally get the plasma shape and current we need.

The control system consists of two parts: e is a deep learning control system and f is a traditional controller. The latter first stabilizes the plasma, and the former adjusts the coil to obtain the target plasma configuration.

Source: Reference [1]

In fact, among these three steps, the first two steps are like the class and practice mode in school, and only the last step is the real test. Before the third step, the deep learning algorithm is trained with "practice questions" and "past test questions". Therefore, when the algorithm trained at the end is run in the actual physical environment in the real test, it can maintain the stability of the plasma and change the geometric structure of the plasma for different experimental conditions, and even... can simultaneously generate two independent "water drop-shaped" plasma structures in the same tokamak! (first left in the picture below)

In the eyes of scientists, each configuration means completely different parameter settings.

Source: Reference [1]

This is unimaginable in the era of traditional controllers. If "plasma configuration control" is a course, then artificial intelligence is obviously a genius student in this course! Because of one of the most complex systems - the tokamak magnetic control problem, the design of plasma controllers has always been a difficult problem. It must be said that this remarkable attempt of artificial intelligence has brought new hope and direction to the design of plasma controllers. This will undoubtedly accelerate the development of tokamak magnetic control and nuclear fusion science.

This is also the first time that artificial intelligence has demonstrated its capabilities in such an important field, following chess (Alpha Go) and games (StarCraft). As we grow, artificial intelligence is also growing. Where will it surprise us next time? Let us wait and see!

References:

Magnetic control of tokamak plasmas through deep reinforcement learning | Nature Original article~

Ariola, M., & Pironti, A. (2008). Magnetic control of tokamak plasmas (Vol. 187). London: Springer.

Controlled nuclear fusion_Baidu Encyclopedia (baidu.com)

For the first time in history, a reinforcement learning algorithm controls nuclear fusion and appears in Nature: DeepMind takes a big step forward in artificial sun (qq.com)

New Fusion Reactor Doubles Energy Record From Fusing Atoms (sciencefriday.com)

DeepMind's AI can now successfully control plasma in a fusion reactor – By Futurist and Virtual Keynote Speaker Matthew Griffin (fanaticalfuturist.com)

Controlled nuclear fusion applications, will it always be “50 years away”? (baidu.com)

"Artificial Sun" has taken another step forward, and controlled nuclear fusion technology has made another breakthrough (baidu.com)

DeepMind_Baidu Encyclopedia (baidu.com)

Why deuterium and tritium? A thorough understanding of the basic principles of fusion. - Zhihu (zhihu.com)

Controlled nuclear fusion - Zhihu (zhihu.com)

What is controlled nuclear fusion? - Zhihu (zhihu.com)

Lithium (metallic element) - Baidu Encyclopedia (baidu.com)

"Artificial Sun" is one step closer to realizing the dream - revealing the secrets of China's controlled nuclear fusion - National Energy Administration Power Reliability Management and Engineering Quality Supervision Center (nea.gov.cn)

Special thanks to: Banyan Tree

Emoticon source: Internet

Cover image: sktfaker

Editor: Banana

Source: Institute of Physics, Chinese Academy of Sciences