Is the weather forecast always a lie? AI makes the weather more predictable

We are probably accustomed to the real-time weather conditions on our phones and desktops. With one click, we can get the weather conditions for the next few days. The weather forecast can be accurate to every hour, showing the changes in weather, wind direction, and humidity.

How is such a detailed weather forecast given?

Weather forecast released by China Weather Network, source: China Weather Network

Recently, Nature magazine published two articles on weather forecasting, introducing the Huawei Cloud Pangu meteorological model and Tsinghua University's NowcastNet model. What new opportunities do these two models bring to weather forecasting? Is it possible for us to achieve "predictable weather" and obtain more accurate weather forecasts? What role do AI, big data, and supercomputing play in this?

On July 6, Huawei Cloud Pangu Meteorological Model was published in Nature magazine. Image source: Nature magazine official website

No.01

Unexpected weather

Weather affects people's daily lives. About 10,000 years ago, humans entered an agricultural society. The germination and harvest of seeds surprised people by discovering that they no longer had to travel across mountains and rivers, and that they could feed their families by cultivating a piece of land. This led to a stronger expectation for "good weather and good harvests."

Due to the limitations of the cognitive level and production and living methods at that time, various weather phenomena such as wind, clouds, thunder and lightning, and snow were shrouded in mystery. The ancients believed that the changes in wind and clouds were the will of heaven, and they obtained weather forecasts by divination. More than 3,000 years ago, the King of Shang wanted to go out hunting, so he asked if there would be heavy rain. The divination result showed that there would be no rain, so the King of Shang and his party set out happily, but they were soaked during the hunt. I wonder if the King of Shang expressed his feeling that "there are unpredictable winds and clouds in the sky".

Image source network

Although the ancients did not have accurate weather forecasts, they wisely discovered the regularity of the sun's movement and natural phenomena, and creatively summarized the 24 solar terms, 72 phenological terms, and various meteorological proverbs.

For example, the last solar term in spring is Guyu. "On the day of Guyu, duckweed begins to grow. Five days later, doves begin to flutter their feathers. Five days later, hoopoes descend on mulberry trees." Guyu means the end of the cold wave and the increase in precipitation, reminding people to sow seeds. The rice seedlings and crops in the fields have just been planted and need rain the most. The ancients believed that during this period, "rain brings forth all kinds of grains," so it was called "Guyu."

National spring sowing map during the Grain Rain season, source: China Weather

No.02

God-dropping artifact assistance

Obviously, the low-precision empirical weather forecasts such as solar terms and phenology cannot meet people's living and production requirements. So, how did the real modern weather forecast come onto the historical stage?

In November 1854, during the Crimean War, the British and French allied forces were preparing for a decisive battle with the Russian army in the Black Sea. However, before the battle began, a strong storm caused heavy losses to the allied forces.

Later, Le Verrier, director of the Paris Observatory, studied the storm. After collecting a large amount of meteorological data, he drew a weather map at the time and found that the movement of the storm had a certain pattern. If this phenomenon had been noticed in advance, the failure could have been avoided. So Le Verrier suggested setting up a meteorological observation network to collect observation data and analyze and produce weather maps. In 1856, France established the world's first formal weather forecast information service system.

Storm hits ships at sea, source: Internet

From then on, mankind entered the era of weather maps. Modern weather forecasting began with these weather maps, which plotted the weather conditions of different places at the same time on a map. Forecasters then analyzed and described the weather development and changes and made forecasts for the future weather.

Weather map forecasting is a common method used by meteorological stations to forecast weather, and it is still used today. These weather maps provide a large amount of data for the discovery of weather patterns, making it possible to use physical equations and mathematics to calculate weather phenomena and processes.

French mathematician, astronomer, and director of the Paris Observatory, Le Verrier, picture source network

However, the method of drawing weather maps and making weather forecasts is greatly influenced by the subjective influence of forecasters and often leads to errors. How can we make more rational weather forecasts?

Scientists believe that with the two sharp weapons of mathematics and physics in hand, everything can be calculated.

But how to calculate it? This question stumped many people until a father and son, William and Jacob Piekness, came on the scene. They introduced the methods of thermodynamics and fluid mechanics into the study of meteorology and proposed to use complex differential equations to describe weather changes.

Portrait of Vilhelm Bjerknes (1862-1951). Image source: Internet

But differential equations are difficult to solve, and it takes several days to solve them. Due to limited computing power, weather forecasts are like buying an out-of-date newspaper, which is not practical.

Later, British Louis Richardson proposed a bold hypothesis to build a "weather forecast factory". This factory was built in a circular building, with each seat distributed according to the longitude and latitude of the earth. Each person was responsible for calculating the differential equation of his own longitude and latitude, and finally sent his own calculation results to the people in the center for summary, and obtained the weather forecast for all parts of the world. However, this dream design was obviously impossible to implement, because in order to give the forecast results in advance, at least 64,000 calculators were needed.

Richardson's conception of a "weather forecasting factory" (1922), source: Internet

Does the concept of a "weather factory" look familiar to you? Does it look like a computer?

ENIAC, which was used to calculate ballistics in World War I, attracted the attention of meteorologists. After continuously simplifying the algorithm and converting a series of mathematical and physical equations describing atmospheric motion into computer language, in 1950, Charney, Firtov, and von Neumann used ENIAC to complete the feasibility experiment of numerical weather forecasting. They used this computer to complete the weather forecast calculation 24 hours in advance in about 24 hours. In 1954, the British BBC radio broadcast this numerical weather forecast to the world, and weather forecasting truly entered an era of scientific forecasting.

Von Neumann and the first electronic computer ENIAC, source: China Meteorological Science Network

NO.03

Is the era of AI forecasting coming?

The essence of numerical weather forecasting is to solve complex differential equations, which is highly dependent on initial conditions. The atmosphere is changing in real time, and a small change may cause a long-term huge chain reaction in the entire system, which is the well-known "butterfly effect".

Even the most complex physical models find it difficult to fully simulate the overall picture of atmospheric movement, and the detailed movement of the atmosphere is also difficult to capture. Therefore, the simulation of extreme weather and climate events has become a global problem.

The classic graphic of a chaotic system resembles a butterfly. Source: Internet

Numerical weather forecasting is the core technology for the development of modern meteorological services. It can be called the "chip" and "national treasure" of the meteorological field. my country began research on numerical weather forecasting in 1954, and is one of the earliest countries in the world to carry out numerical weather forecasting.

In the early days, China mainly imported data from abroad, but later developed its own core technologies. In December 2020, China's first-generation global atmosphere/land surface reanalysis system and products independently developed were put into commercial use, with the ability to independently produce reanalysis products, greatly reducing the dependence of meteorological services on foreign data products. Now, with the development of the next generation model targeting the Earth system model, my country's numerical forecasting business has achieved a leapfrog development from "zero" to "one" and then to "excellence".

EarthLab, my country's Earth System Numerical Simulation Device with Independent Intellectual Property Rights

Recently, the Huawei Cloud team and Tsinghua University published two articles on the Cloud Pangu meteorological model and the NowcastNet model in the journal Nature. How is this different from traditional numerical forecasting?

Both models use AI models to predict the weather. However, the two models have different uses. The former is a model that can predict global weather patterns one week in advance, while the latter is a short-term weather forecast for extreme precipitation events.

AI joins the weather forecasting track, bringing new possibilities. AI is good at handling repetitive tasks and fitting unknown data relationships. Through deep learning to understand the relationship between various meteorological data, it shows superiority in both prediction accuracy and prediction speed over traditional numerical forecasts.

First, let's talk about Huawei's Pangu Meteorological Model. Improvements in AI algorithms have made Pangu Meteorological's forecast accuracy comparable to the world's best numerical weather forecast system, the European Centre for Medium-Range Weather Forecasts' integrated forecast system. Not only can it handle more complex data, Pangu Meteorological can process three-dimensional meteorological data and capture the relationship between atmospheric states at different pressure layers. And it's faster. At the same spatial resolution, Pangu Meteorological is more than 10,000 times faster than the numerical weather forecast system.

Huawei Developer Conference released the Pangu series of ultra-large-scale pre-trained models from the Internet

The NowcastNet model is a large-scale extreme precipitation nowcasting model completed by Tsinghua University, the National Meteorological Center, and the National Meteorological Information Center after three years of joint research. It can predict extreme precipitation within 0 to 3 hours at a kilometer scale.

At the World Meteorological Organization summit on May 27 this year, precipitation nowcasting within three hours was listed as one of the important unresolved scientific problems. Due to the short duration and small spatial range of extreme precipitation processes, the original numerical calculation method can only predict within one hour.

Imagine that we can know that a heavy rain is coming from 1 hour in advance to 3 hours in advance. In the former case, we could only find a place to hide nearby, but now we can return home more calmly. The response strategy has changed significantly. For some actual production and life scenarios, this forecast will also have a more impressive disaster reduction performance, which is undoubtedly a huge leap forward.

Precipitation nowcast based on radar observations, from the Internet

So will AI weather forecasting replace traditional numerical forecasting? Will weather forecasters lose their jobs?

There is no need to worry about this. Researchers say that the fusion calculation of the mechanism and data for the widespread application of AI in the field of meteorology is very challenging and there is still a long way to go. At present, artificial intelligence will not replace humans in this field. Suman Ravuri, a research scientist at DeepMind, said that weather forecasting requires the participation of experts and humans to ensure that the understanding of the forecast is reasonable and then communicate it to the public.

From divination to empirical judgment, from weather maps to numerical weather forecasts, and then to the addition of AI, this is the result of countless researchers working hard and diligently to jointly achieve the "predictable changes in the weather."

END

Author: Institute of History of Science and Technology and Meteorological Civilization, School of Law and Politics, Nanjing University of Information Science and Technology

Associate Professor Li Beibei

Gong XinyuMaster

Editor: Guru