ACM Distinguished Member Shuiwang Ji: Deep Learning in Quantum Chemistry and Physics

This article is reproduced from Leiphone.com. If you need to reprint it, please go to Leiphone.com official website to apply for authorization.

Quantum technology and artificial intelligence are both the most advanced scientific technologies at present. The former is expected to have super computing power, and the latter has already "killed all" in various industries. What kind of sparks will collide when the two meet? In what aspects can artificial intelligence help quantum technology?

Last December, Ji Shuiwang, a professor in the Department of Computer Science and Engineering at Texas A&M University (Presidential Influential), delivered a speech entitled "Deep Learning of Quantum Chemistry and Physics" at the CNCC conference, expressing his thoughts on the two disciplines.

"Quantum breaks a lot of our common sense understanding. The operation of the quantum world is uncertain. At best, we can only predict the probability of various results."

In addition, he also said that although the research objects of quantum are objects at the atomic level or even the subatomic level, they also have something in common with macroscopic rules. For example, the structure between molecules can be regarded as a picture and processed.

Shuiwang Ji received his Ph.D. in Computer Science from Arizona State University in 2010, with Professor Jieping Ye as his advisor. His research interests include machine learning, deep learning, data mining, and computational biology. He received a CAREER Award from the National Science Foundation in 2014. He was elected as an ACM 2020 Distinguished Member in 2020.

The following is the full text of the speech, which AI Technology Review has edited without changing the original meaning.

Hello everyone, today I will focus on how to use AI and graphics computing technology to solve quantum physics and quantum chemistry problems.

First, let's get some background: In classical physics, we discuss objects or phenomena in the macroscopic world. For example, if you kick a ball, if you know the specific mass, speed, and current time of the ball, you can predict the position of the ball in five seconds. However, in the quantum realm, since the objects of study are objects at the atomic or even subatomic level, such as molecules composed of atoms and chemical bonds, it is impossible to think about the rules of the field according to traditional logic.

In recent years, we have been working with experts in various fields, hoping to gain research breakthroughs from quantum physicists, quantum chemists, quantum materials scientists, etc. Scholars from these different fields have some common topics that need to be studied, and these topics are related to images, AI, and especially deep learning. Now I will report the latest progress.

1 AI meets quantum chemistry

Molecules are composed of atoms and chemical bonds between atoms. For example, in molecules, atoms are represented by points, while molecules are represented by lines. Therefore, molecules can be presented in the form of 2D graphics. In the fields of machine learning and data mining, graphic computing is a commonplace topic. However, when migrating to the molecular field, new challenges are also faced: the form of 2D graphics cannot fully explore the properties of molecules. After all, a molecule is not actually a 2D plane, but has three-dimensional spatial properties. Its structure is not only determined by the properties of points and lines, but also by spatial coordinates, chemical bond angles, and so on. Therefore, when exploring the function of a molecule, it is necessary to pay attention to its three-dimensional structure.

How to efficiently use the spatial information of molecules to make predictions and generate models? Message Passing Neural Network (MPNN) is a commonly used graph neural network framework. We can find that this type of method can be summarized into two equations: aggregation function and node update function. The aggregation function can aggregate the information of neighbor nodes.

When we try to calculate the information of a node, we basically consider the properties of the node itself at the last time point, the properties of the intermediate nodes, and the information of the boundaries. After calculating the aggregation function, we need to use the node update function. This step requires the ability to use the information and properties of the previous step to update the current node information. However, this simply considers the characteristics of the node and the boundary. Therefore, our recent work is to try to build a three-dimensional graph computing network so that we can obtain complete three-dimensional information.

Once three-dimensional information needs to be included, the computational network will become very complex and information transmission will become inefficient. So we hope that the network will be efficient while making the computation equivalent and stable. If there is a molecule, when you rotate it, many of its 2D properties may not change, but the 3D information may not; therefore, we hope that in the prediction and generation model, when a node of the molecule rotates, its quantum properties will also remain stable.

The predictive function of the model refers to predicting the properties of a given molecule, for example, we can predict whether a molecule has the potential to be an antibiotic. The generative model refers to generating/synthesizing corresponding new molecules based on given properties.

Currently, many researchers have taken 3D properties into consideration. One of the earliest works is called SchNet, which takes distance as a three-dimensional attribute. Using SchNet means taking into account the boundaries and the length of the boundaries. Recently, there is also a work called DimeNet. DimeNet goes a step further on the basis of SchNet because it takes angles into consideration. For example, if you have information from j to i, and you need to calculate mi, j, then you need to take into account not only the node information, but also the angle between the two chemical bonds.

But in chemistry, we find that it is not enough to just consider distance and geometry. As shown in the figure above, the red part represents a plane that does not really exist, and the same is true for the blue part. Molecules have geometric shapes, but knowing only the distances of three chemical bonds and two bond angles cannot fully determine the geometric shape of a molecule.

Let's think about it. There is an angle φ between the plane determined by d1 and d2 and the plane determined by d2 and d3. It is this angle that becomes the uncertainty factor in the above model. Because even if the two bond angles are the same, the φ angle keeps changing, which causes the geometric shape of the molecule to change.

What we are trying to build is a complete geometric framework that can solve all cases, which we call spherical information transfer.

To solve the above problem, the angle φ is taken into consideration, which is the angle between X and its projection.

One consideration is that the molecules presented must be stable and invariant. For example, when the molecules are rotated, their properties, such as all the angles, should not change. In spherical information transfer, we construct a spherical coordinate system, including reference points, distances, and torsion angles. But this model is not 100% perfect.

Because only one control is considered, there will be uncertainty when considering other nodes. So our work is imperfect but very efficient.

Recently, there is a system called GemNet. Their idea is that our system only uses the information of the neighbor nodes of node A, and does not use the 2-hop field information. GemNet scientists believe that when you use the 2-hop field information, the system will be close to perfection. Indeed, when you use the 2-hop field information, the angle information will be better merged and achieve a nearly perfect effect, but the problem is that once you use the 2-hop information, the information will be updated to include a large number of neighbor nodes, and the entire information update step will become extremely complicated.

In comparison, although our system is not 100% perfect, it is more efficient and can intuitively see the complexity: n represents the number of nodes, and K is the average degree of freedom of all nodes. In actual effect, our model is very similar to the more complex GemNet.

The following figure can clearly show what our model can or cannot represent. Figures a and b show a phenomenon known as chirality in chemistry.

In fact, the two molecules are like mirror images. The network we designed can distinguish between these two situations, while many previous methods cannot do so. Because in our method, we use the torsion angle as the relative angle, and in the case of chirality, the angles of q1 in the two are 60° and 90° respectively. However, in the second case, the torsion angle of q1 is 90°, so our method cannot distinguish it. The second situation is also the one that was pointed out by colleagues in the community as "hope to be corrected", but from a chemical point of view, the probability of this happening is very, very low, because q2 and q3 are different atoms, and it is almost impossible for them to have the same torsion angle as q1. So we believe that although our model does not cover all situations 100%, the situations that cannot be covered are unlikely to occur in nature.

2 AI meets quantum mechanics

When we start looking at quantum mechanics, the Schrödinger equation provides us with a solution. If you know the values ​​of the distance angle and the torsion angle, you can use different functions in the equation, such as spherical harmonics and spherical Bessel functions, or other basic functions to collect the values ​​of θ and finally get an eigenvector. This is a physically meaningful eigenvector that can be used in actual information transmission.

The following figure shows the system construction process. There is an input module, an interactive module that uses the torsion angle and distance information as input. This interactive module may be re-evaluated many times, and the number of repetitions will depend on the amount of data you have. Finally, there is an output module, which enables information transmission for some events, such as the public catalyst challenge.

The Open Catalyst Challenge is a competition initiated by Facebook AI and CMU. The purpose of the competition is to use new large-scale molecular data to predict thermodynamic data. In the field of catalyst discovery, these target molecules are usually relatively large, with an average of 80 atoms per molecule in structure.

So they divided the data set into four groups based on the relationship between training and testing, and scored based on the average of each absolute error to evaluate the best quality that the system can measure. Each row represents a model, CGCNN comes from a company that uses models to study molecules, and there are SchNet, DimeNet and GemNet. You can see that among all the systems, SphereNet can occupy a very competitive position.

The above figure is the result from another data set QM9. This is a relatively small data set. Each column represents a quantum attribute, and each row represents a prediction method. From the table, we can see the average error of each method on different attributes.

Our system is also successful on other datasets, such as MD17, which is a smaller dataset. As we mentioned, GemNet uses 2-hop data, which requires more computational power, so it can only be used in smaller datasets.

As can be seen in the figure above, even in smaller datasets, our system performs slightly better than DimeNet and is comparable to GemNet, which is more computationally expensive.

The following figure shows the comparison of computational consumption. So far, our computational consumption is much smaller than that of two generations of GemNet computation.

The following figure shows the filter of the system. As you can see, each row represents a molecule with a different torsion angle, and in many cases our filter shows very different results at different torsion angles, which also confirms that the torsion angle parameter is very important in capturing different molecular modes.

To summarize briefly, our idea is to try to fully display the three-dimensional information of molecules, so we built the SphereNet framework. And the framework is theoretically almost perfect and very efficient. From the actual effect, our framework may have 100% coverage, and we have made a lot of progress in this direction.

At present, the related work has been open sourced and designed into a "dive into graphs" library.

In particular, for molecular research applications, we have a dedicated library called "molecule X". If you follow the KDD competition, you will know that we are one of the leaders in graph neural network computing. We also participated in the AI ​​Cures open challenge for Covid-19. Currently, we are ranked first in both AUCROC and AUPRC.

Therefore, our work is mainly to develop new image processing technologies to solve problems in basic science, especially quantum chemistry, quantum physics, and materials science. My team has developed computational methods, open sourced software libraries, and published our results in conferences and journals. At the same time, we have also participated in many open challenges, such as the KDD Cup.

Our research is at the intersection of artificial intelligence and quantum physics. In quantum physics, the Schrödinger equation is the basis. The corresponding research is very expensive. If you build a particle system, you need huge computing power to solve the eigenvalue problem. But combining quantum physics with AI computing will be a very hot field. At present, this field is still in the exploratory stage.