Regarding chaos, what did Ulam, the father of the hydrogen bomb, do?

Before the definition of "chaos" in a definite sense came out, Ulam conducted in-depth research on its early exploration - modern ergodic theory - and proposed far-reaching questions, gave algorithms, and guessed convergence, opening up a new field of computational ergodic theory. Based on York and his collaborators' mathematical arguments on "existence", Chinese mathematician Li Tianyan independently gave a numerical method for calculating invariant density functions and proved convergence, and he is also a pioneer in computational ergodic theory.

Written by Ding Jiu (Professor of Mathematics at the University of Southern Mississippi)

Great people are often humorists. Stanislaw Ulam (1909-1984), the father of the hydrogen bomb, once quipped: "Calling the study of chaos 'nonlinear analysis' is like calling zoology 'the study of animals other than elephants'."

In fact, although the definition of "chaos" in the deterministic sense was not introduced until the mid-1970s, the exploration of its "ergodicity" began 45 years earlier: the two classical ergodic theorems, von Neumann's average ergodic theorem and Birkhoff's pointwise ergodic theorem in the early 1930s, were the main representatives, and the research on modern ergodic theory can be said to be led by Ulam.

The “Invariant Density Function” Method for Exploring “Chaos”

What does the word "ergodic" mean in mathematics? Let me first use an example to illustrate the meaning of "non-ergodic" to help you imagine what "ergodic" means. Define a mapping S that maps [0, 1] to itself: when 0 ≤ x < 1/4, S(x) = 2x; when 1/4 ≤ x < 3/4, S(x) = 2x – 1/2; when 3/4 ≤ x ≤ 1, S(x) = 2x – 1. Here is the graph of S:

The definition of an "ergodic map" can be figured out from the example of a non-ergodic map above. For any map that maps a domain onto itself, there are always two trivial "invariant sets", the empty set and the domain itself, since their inverse images under the map are equal to themselves. A map is ergodic if it essentially has no "non-trivial" invariant sets, i.e., there is no non-trivial subset of the domain such that it is the same set as its inverse image under the map. In short, a map is ergodic if it has only trivial invariant sets.

When does an "invariant density function" exist?

In the book Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics co-authored by Andrzej Lasota (1932-2006) and Michael Mackey (1942-), in Section 2 of Chapter 6, the authors give a proof that the above operator has only the zero function as its unique fixed point.

Note the difference between the two examples above: the derivative of the first mapping is always equal to 1/2, which is strictly less than 1; but the derivative of the second mapping is strictly greater than 1 everywhere on (0, 1], but equal to 1 at x = 0. This raises a question for us: if a mapping has a derivative that exists everywhere on the domain interval except for a few exceptions, and the absolute value of the derivative is strictly greater than 1, does the corresponding Frobenius-Perron operator have a non-zero fixed point? A mapping that satisfies the above derivative conditions is called

Readers who have studied elementary calculus should know that a piecewise-stretched mapping cannot be differentiable everywhere on the entire domain interval. Otherwise, according to Lagrange's middle value theorem, which is known as the "fundamental theorem of differential calculus", the range interval of the mapping will be longer than the domain interval, which contradicts the basic assumption that the range is contained in the domain. The fact that the derivative of the familiar piecewise-stretched tent mapping T does not exist at the point x = 1/2 also shows this. Therefore, the piecewise-stretched interval mapping can only be "piecewise-stretched" through "piecewise differentiability".

The Legend of Ulam

Ulam was a pioneer in the comprehensive field of "nonlinear analysis". In fact, he and his lifelong friend John von Neumann (1903-1957) and physicist Enrico Fermi (1901-1954) and other wise men created this discipline while working on the development of the atomic bomb. Many years ago, I read Ulam's collection of essays Science, Computers, and People. The first paragraph of the preface written by mathematics popularizer Martin Gardner (1914-2010) is: "Ulam, or Stan as his friends called him, was one of those great creative mathematicians who was interested not only in all areas of mathematics, but also in physical and biological sciences. Like his good friend von Neumann and unlike many of his peers, Ulam could not be classified as a pure or applied mathematician. He never stopped finding the same amount of beauty and excitement in pure areas that had nothing to do with applied problems, and in the applications of mathematics."

Ulam was a Jew. His birthplace, Lviv, was originally part of Poland under the Austro-Hungarian Empire, and is now part of Ukraine, located in the west of the country. Therefore, he is a Polish/Ukrainian mathematician, and is listed as the first of the ten most legendary Ukrainian mathematicians in history on the "Pantheon" website (pantheon.world). At the beginning of his popular autobiography Adventures of a Mathematician, which I have read three times, Ulam told readers that when he was four years old, he was fascinated by the geometric patterns on the Persian carpet in his living room. When his father, a lawyer, laughed at this, he said to himself: "He laughed because he thought I was naive, but I knew these were curious patterns. I know something that my father doesn't know." This may be the first manifestation of his lifelong talent for exploring new things. Another proof that he liked to raise and solve questions is that in the 1930s, when the Polish school of mathematics was famous all over the world, the Polish mathematical elite led by Stefan Banach (1892-1945), the master of functional analysis, discussed mathematics in Scottish cafes and wrote down the mathematical problems they encountered in real time. The "Scottish Book" is now famous in the international mathematics community. Ulam, who was in his twenties, contributed the most questions!

Because he loved discussing with people and asking questions, Ulam's "feeling for the key points" that sprouted in his brain later became the beginning of several mathematical fields. For example, he first proposed the "cell automaton theory" to von Neumann; the idea of ​​the "Monte Carlo method" came from his thinking about the thorny problems of number theory and integration; and the "nonlinear analysis" that later set off a wave of soliton and chaos research began to flow from his fingers playing with the computer keyboard. What is relevant to this article is that a numerical method he proposed announced the birth of the "computational ergodic theory", a discipline that combines pure mathematics, applied mathematics, and computational mathematics.

At the age of 20, Ulam published a mathematical paper on set theory. On the eve of World War II, he went to the United States. Since then, his genius brain has contributed many brilliant ideas to the country, one of which is so important to the US government that it can "change the course of history." Because of this great contribution, he is hailed as the "father of the hydrogen bomb" by most scientists, and many people think that the Hungarian-American theoretical physicist Edward Teller (1908-2003) played this role because the latter's social reputation was higher than the former in the entire process from the conception to the successful development of the hydrogen bomb. At noon on January 23, 1951, Ulam's wife found her husband at home staring at the garden outside the window with a strange expression, and said: "I found a way to make it work." "What work?" his wife asked him. "Hydrogen bomb," he replied, "This is a completely different plan, it will change the course of history." Even Ulam, a "mathematical scientist" with extraordinary ability, was often amazed to see that "some scribbles on the blackboard or draft paper will change the course of human development."

As for who between Teller and Ulam was the "biological father" of the hydrogen bomb, their German superior at Los Alamos National Laboratory, director of the theoretical department and winner of the 1967 Nobel Prize in Physics, Hans Bethe (1906-2005), once made a wonderful comment: "After the hydrogen bomb was made, reporters began to call Teller the father of the hydrogen bomb. For the sake of history, I think it is more accurate to say that Ulam is the father because he provided the seeds, and Teller is the mother because he stayed with the child. As for me, I think I am the midwife." Here Bethe specifically wrote the name "Ulam" in bold for emphasis.

Ulam's questions, methods and conjectures

In 1960, Ulam published a small book of only 150 pages, "A Collection of Mathematical Problems". This thin book is full of mathematical ideas and has made many mathematicians successful. James Yorke (1941-) and his collaborator Losuda are two of them. Ulam asked in the fourth section of Chapter 6 of the book: "If the mapping S that maps the unit interval to itself is defined by a sufficiently 'simple' function (for example, a piecewise linear function or a polynomial), and its graph does not cross the straight line y = x in a way that the absolute value of the slope is less than 1, then does the corresponding Frobenius-Perron operator have a non-trivial invariant density function?" Ulam then gave an example of a family of piecewise linear mappings with parameters, and the answer to the above question had not yet been given.

Thirteen years later, as the next generation of Ulam's compatriots and the successor of "nonlinear analysis", Losuda and his North American collaborator York published the answer to Ulam's question in the Transactions of the American Mathematical Society. This is an important paper in modern ergodic theory, titled "On the existence of invariant measures for piecewise monotonic transformations". The abstract is only one sentence, which succinctly summarizes the contribution of the article: "This paper proves that a class of piecewise continuous and piecewise quadratically differentiable transformations on the interval [0, 1] has absolutely continuous invariant measures." In the paper, they proved the following results:

Losunda-York theorem

If the map S that maps an interval onto itself is piecewise quadratically continuously differentiable, and the absolute value of its derivative on the interval is never less than a constant greater than 1, then the Frobenius–Perron operator corresponding to S has at least one invariant density function.

In particular, Losuda and Yorke gave a positive answer to the family of piecewise linear mappings defined in Ulam's book, that is, for every parameter a satisfying 0 < a < 1/2, the mapping Sa has an absolutely continuous invariant measure. Furthermore, their pure mathematical work gave birth to a creative article in computational mathematics, written by Yorke's doctoral student Li Tianyan (1945-2020), which has become a classic in computational ergodic theory.

In his book Mathematical Problems, Ulam not only proposed the existence of an invariant density function, but also proposed a computational method based on probability to numerically approximate the invariant density function that is assumed to exist. For a given mapping S: [0, 1] → [0, 1], Ulam first divides the domain interval [0, 1] into n equal parts. For i = 1, 2, …, n, the i-th subinterval is

"We conjecture that if the Frobenius-Perron operator has an invariant density function, then when the number n of partitions of [0, 1] becomes infinite, the invariant step function will converge to the invariant density function." This is now called the "Ulam conjecture", and the numerical format he constructed for calculating the invariant density function was later called the "Ulam method" - the modern research field of "computational ergodic theory" started with pages 74-75 of his book, and this is the earliest and most famous algorithm among them.

Li Tianyan, who was ill, proved the "Ulam conjecture" for the first time

After Losuda and York proved and published their theorem, the existence of an absolutely continuous invariant measure of a class of piecewise elongated mappings was strictly guaranteed. When Li Tianyan, who had just obtained his doctorate, read their paper, he, who had already developed a great interest in computational mathematics, began to seriously consider how to effectively calculate the invariant density function that was guaranteed to exist in theory. However, God was not too kind to his body. Just six weeks after obtaining his doctorate in 1974, his blood pressure rose to 220/160 millibars because his kidneys were really broken. Just as on June 25, 2020, the day after Professor Li Tianyan passed away due to illness, at the memorial service held for him by our disciples, his mentor Professor York recalled that after he went from Taiwan to the University of Maryland to study for a doctorate in 1969, his kidneys began to have problems the following year. He was only 25 years old at that time. By 1976, his kidney function was only 10%, so he began kidney dialysis for five and a half years, three times a week, each time lasting five hours, not including the time spent on the road. He then went to Europe to receive a kidney transplant, but it failed due to rejection. Finally, in 1981, a kidney from his sister matched his and was successfully transplanted. This selfless kidney worked for him for 39 years and also helped him write 26 doctoral dissertations.

Li Tianyan showed a steely will in the face of illness and never gave up. In the mid-1970s, many of his research ideas were conceived on his sickbed in the hospital. In order to design a utility for calculating absolutely continuous invariant measures for the class of interval mappings that satisfy the conditions of the Losuda-York theorem, following the general principles of numerical analysis, he first discretized the Frobenius-Perron operator. First, like Ulam, he divided the [0, 1] interval into n equal parts. Then he defined a projection operator with a range of finite-dimensional subspaces, which projects each integrable function on [0, 1] into a piecewise constant function corresponding to the above interval partitioning. Its constant value on each relevant subinterval is the average value of the integrable function on this subinterval, that is, the integral of the function on the subinterval divided by the length of the subinterval. It is obvious that this projection operator projects a non-negative integrable function into a non-negative step function and keeps the integral of the function on [0, 1] unchanged.

The "unit operator" that maps each integrable function to itself is also a projection operator, and it is the projection operator with the largest range, so the above projection operator becomes a finite-dimensional approximation of the unit operator. Li Tianyan combined it with the Frobenius-Perron operator, which constitutes a finite-dimensional approximation of the Frobenius-Perron operator. If it is restricted to the n-dimensional subspace composed of all piecewise constant functions determined by interval decomposition, then its domain and range are the same subspace, and under the standard density function basis of the subspace, its matrix representation is a random matrix. The density function in this basis is the characteristic function of all n subintervals divided by the length of the subinterval. At that time, Li Tianyan did not know that the matrix was exactly the matrix constructed by Ulam using the probability method in the book published fifteen years ago.

At this time, Li Tianyan was particularly fond of Brouwer's fixed point theorem, because he had previously created a modern homotopy extension method based on differential topology to numerically approximate such fixed points. Therefore, he did not use the standard Perron-Frobenius non-negative matrix theory, but directly borrowed Brouwer's fixed point theorem to prove that the finite-dimensional approximation operator of the Frobenius-Perron operator he constructed in this way must have a non-zero fixed point, which is a piecewise constant density function. This conclusion is valid for any finite partition of the interval [0, 1], so the "piecewise constant function projection method" he proposed is a well-posed numerical method, that is, for any natural number n, the algorithm can calculate the approximate invariant density function of the Frobenius-Perron operator with a step image based on n subintervals.

Now there is only one problem left to solve, but it is the most important problem for computational mathematicians, and often the most difficult problem: does the well-posed algorithm we have constructed converge? In other words, does the approximate invariant density function that is guaranteed to exist for any natural number n converge as n goes to infinity? Moreover, if it converges, does it converge to the exact invariant density function of the Frobenius-Perron operator, as we hope?

This requires looking for key clues to help prove convergence in the reasoning details of the Losuda-York theorem, and also requires digging out the essence of analytical mathematics stored in the mind of the algorithm proposer. In Losuda-York's paper, York contributed a useful inequality, which uses an inequality sign to build a bridge between the variation of a bounded variation function on the domain and the variation of the product of the function and the characteristic function of a subinterval. This bridge enabled the two collaborators to obtain the key "Losuda-York variation inequality", which ultimately led to the theorem conclusion of the existence of invariant density functions. Variation is an important concept in mathematical analysis, but I will omit its definition here and only point out one point: it often plays a leading role in "convergence analysis". Li Tianyan keenly observed that the projection operator he defined corresponding to the interval decomposition would not increase the variation of the integrable function. Thus, with the help of the Losuda-York variation inequality, he proved that the variation of the approximate invariant density function is uniformly bounded for all natural numbers n for the piecewise elongated mapping that satisfies the Losuda-York theorem. Then, using the classic Hurley selection theorem in analysis, the obtained piecewise constant density function sequence contains a subsequence that converges to an invariant density function of the Frobenius-Perron operator in the sense of the "norm" of the integrable function space. In particular, if this invariant density function is unique, then the piecewise constant approximate invariant density function sequence will converge to it.

This is the first time that someone has conducted numerical analysis on a specific type of interval mapping for Frobenius-Perron operators since Ulam proposed the "Ulam conjecture" on the convergence of invariant step function sequences. It is also one of Li Tianyan's three major mathematical contributions in his life, and he completed it independently and published it in the American Journal of Approximation Theory in 1976. His other two contributions can be referred to as "chaos definition" and "homotopy algorithm", which are his masterpieces before the age of 30. However, until the article was written, he did not know that the algorithm he invented was essentially the same as the matrix written by Ulam on a page of "Mathematical Problems".

After submitting the article, someone told the author: The method you proposed is the Ulam method that came out fifteen years ago. The difference is that Ulam did not prove the convergence of his method for any mapping family. He just guessed that as long as the invariant density function exists in theory, the algorithm converges. From this point of view, Li Tianyan "accidentally" proved the Ulam conjecture for the Losuda-York interval mapping family. Ulam is a world-renowned mathematician, and putting his name in the title of the article should attract more potential readers. So Li Tianyan lengthened the original title of his article "Finite-dimensional approximation of Frobenius-Perron operator" to "Finite-dimensional approximation of Frobenius-Perron operator: a solution to Ulam's conjecture". In a sense, Li Tianyan's proof of Ulam's conjecture for a certain type of one-dimensional interval mapping revived the Ulam method that had been basically dormant for fifteen years. He can be regarded as the main pioneer of computational ergodic theory together with Ulam.

Over the past half century, the computation of invariant measures of chaotic maps has become an active branch of ergodic theory in mathematics and nonlinear analysis in engineering technology. In the research and application of Ulam's method and its higher-order generalizations, Ulam's original work and Li Tianyan's innovative paper have become almost indispensable classics.

However, if the interval mapping is not piecewise elongated, even if the corresponding Frobenius-Perron operator has an invariant density function, the convergence of Ulam's method has not been truly theoretically guaranteed in some cases, or although convergence is observed in numerical experiments, it cannot be rigorously proved. However, over the past few decades, Ulam's conjecture has been proved for many other categories of interval mappings and high-dimensional transformations. Inspired by Professor Li Tianyan's proof method, the author of this article and his collaborator Zhou Aihui from the Institute of Computational Mathematics and Scientific and Engineering Computing of the Chinese Academy of Sciences used the concept of variation of multivariable functions to prove the Ulam conjecture in the 1990s for a class of piecewise extension transformations that map high-dimensional bounded regions to themselves, and the existence of absolutely continuous invariant measures for such transformations was proved by two Canadian mathematicians in the late 1980s.

Professor Li Tianyan once recalled to me the inner sigh caused by his inadvertent success in solving Ulam's conjecture: "If I had known that this was an unresolved problem proposed by Ulam, a mathematician at the same level as von Neumann, I might have been scared and dared not touch it." It seems that it is not necessarily a loss that Professor Li did not read Ulam's famous booklet "Collection of Mathematical Problems" in advance, otherwise he might really not dare to touch it. However, if readers have browsed my book "Out of Chaos: My Mathematical Love with Li Tianyan" published in 2021, and learned about his lifelong experience and experience in reading and studying, they will probably believe like me that he is by no means a person who blindly believes in authority, but directly faces new problems, thinks independently, and finds ways to solve them. As he said to his disciples, including me, in a famous saying: "Just because a big man can't solve a problem doesn't mean that a small man can't solve it either."

Two Views on Chaos: Deterministic and Statistical

At the end of the 19th century, French mathematician Henri Poincaré (1854-1912), known as the "ancestor of chaos", first discovered the chaotic phenomenon in the three-body problem in natural science; seventy years later, in the middle of the 20th century, the "father of chaos" Edward Lorenz (1917-2008) solved the mystery of the "butterfly effect" in weather forecasting; fifteen years later, due to the insight of mathematicians, a beautiful chaos theorem came out, and a mathematical definition of chaos broke out. Since then, the study of chaos has emerged like the tide of the sea. "Chaos" is considered by many scientists to be the third major scientific discovery in the 20th century after "relativity" and "quantum mechanics". Its concepts, ideas, theories and methods have long been blossoming in many fields of physical science, life science and engineering science.

Chaos in a deterministic sense reveals the complexity and diversity of nature. Because of this, it has no unified definition and can only be described from a unique perspective, such as "dynamics freed from the shackles of order and predictability", "irregular and unpredictable behavior of deterministic nonlinear dynamic systems", "complex and non-periodic attractive orbits of certain dynamic systems", etc. The late Chinese theoretical physicist Hao Bolin (1934-2018) regarded chaos as "order without periodicity". No matter how chaos is described, "sensitive dependence on initial conditions" is the basic characteristic of chaos, which leads to the unpredictability of the final state of the dynamic system.

However, the chaotic behavior that seems to be disorderly in the sense of determinism has returned to "regularity" in the sense of probability statistics, which not only provides new ideas and new ways to study chaos, but also reflects the unity of opposites between disorder and order. Yan Jia'an (1941-), a Chinese probabilist, used a five-character poem, "Randomness is not random, probability breaks the mystery. Disorder hides order, statistics solves confusion." to concisely describe the fruitful results of the nutrients of probability statistics sown on the land of deterministic systems. The two perspectives of "determinism and statistics" on chaos reflect the two complementary and reasonable ways of modern mathematics to describe the laws of nature. It is an organic combination of necessity and contingency, and it is also a strong proof of the philosophical concept of "deterministic mathematics and random mathematics go hand in hand and coexist and prosper together"! Modern ergodic theory and its calculation methods are powerful weapons for studying the statistical properties of deterministic systems.

Produced by: Science Popularization China

Special Tips

1. Go to the "Featured Column" at the bottom of the menu of the "Fanpu" WeChat public account to read a series of popular science articles on different topics.

2. Fanpu provides a function to search articles by month. Follow the official account and reply with the four-digit year + month, such as "1903", to get the article index for March 2019, and so on.

Copyright statement: Personal forwarding is welcome. Any form of media or organization is not allowed to reprint or excerpt without authorization. For reprint authorization, please contact the backstage of the "Fanpu" WeChat public account.