Let's take pictures of the sun's poles! AI neural network reconstructs the sun in 3D

Author: Add Zero

Editor: Li Baozhu, Sanyang

Researchers at the National Center for Atmospheric Research (NCAR) in Colorado used the NeRFs neural network to convert two-dimensional images of the sun into three-dimensional reconstructions, revealing the sun's poles for the first time.

Extreme ultraviolet radiation (EUV) refers to solar radiation with a wavelength in the range of 10 to 120 nm. While EUV affects the atmospheric drag of low-Earth orbit satellites, it also poses a threat to human health. Excessive exposure to EUV radiation can lead to vision loss, sunburn, and even serious diseases such as skin cancer.

EUV prediction is inseparable from a complete solar image. However, current EUV imaging satellites can only image around the solar equator (ecliptic) and cannot directly observe some non-ecliptic viewpoints. In addition, due to the influence of the atmosphere, two-dimensional graphics cannot obtain accurate position mapping, and it is also difficult to process a large number of images in a limited time. These obstacles make it challenging to reconstruct the three-dimensional geometric structure of the sun.

To solve this problem, Benoit Tremblay, a solar physicist at the National Center for Atmospheric Research (NCAR) in Colorado, and his colleagues used the NeRFs neural network to convert a two-dimensional image of the sun into a three-dimensional reconstruction, revealing the sun's poles for the first time. For non-ecliptic observation points, the model has a peak signal-to-noise ratio of 43.3 dB and an average absolute relative error of 0.3%, providing a consistent three-dimensional reconstruction of the sun.

Artificial intelligence reconstructs one of the sun's polar regions

This area has never been observed in real life.

Paper address: https://arxiv.org/abs/2211.14879

Experimental process: 3D reconstruction of the sun

Dataset: Sun forward model images

The researchers used the magnetohydrodynamic (MHD) simulation of the solar corona by Predictive Science Inc. (PSI) to estimate the global three-dimensional distribution of plasma parameters and magnetic fields in the solar atmosphere. 256 forward model images of the Sun taken from evenly spaced observation points over 193 years were selected, of which 32 observation points on the ecliptic were used as the training set, and observation points at latitudes outside the ecliptic were used as the test set.

Sun images for training

a: 2019-07-02 20:41:08 (UT) Satellite image of the 193Å Sun taken from the ecliptic;

b: Simulated image from the satellite viewpoint extracted from the 3D model of the sun;

c: Locations of the 256 observation points extracted from the 3D model, with color coding indicating which viewpoints were used for the training set (pink) and the test set (green).

Algorithm structure: SuNeRF model

Purpose: To reconstruct the 3D geometry of the Sun from a set of training images.

Methods: Each coordinate point (x, y, z) is mapped to emission and absorption coefficients (ε, κ) using a neural network designed to simulate the volume.

Function: For each pixel, sample a ray from the population.

Radiative transfer principle: Total intensity calculation based on the radiation transfer principle.

SuNeRF model architecture

Training process: 3D reconstruction of 2D images

The NeRFs neural network is modified and the SuNeRFs neural network is constructed for algorithm training.

Modify NeRF model: Adjust NeRFs model to fit the physical reality of the Sun, replacing NeRF model predictions of density and color with emission and absorption coefficients.

Emission and absorption calculation: For each pixel, the total emission is calculated by sampling points along the ray path. At each point (x, y, z), the emission and absorption coefficients (ϵ, κ) are predicted. Emission (I) is calculated by multiplying κ by the sampling ray distance (ds). Absorption (A) is defined as exp(κ * ds), scaled between 0 and 1 at each point.

Calculate the total observed intensity: Integrate over all sample points, taking into account the absorption on the ray path from the origin to the observer, and use the integrated intensity value to calculate the total observed intensity (I_total).

Pixel value optimization: Apply asinh stretching to optimize the value range for training.

NeRF ray sampling adapted to solar geometry: rays are sampled from within [-1.3, 1.3] solar radii from the sun.

AI transforms a 2D satellite image of the sun (left) into a 3D reconstruction (center)

And calculated the process of the solar polar region (right) that has never been seen before

The training took about 19 hours on an NVIDIA A100 GPU using about 30 epochs with a batch size of 8096 rays. The back-propagation algorithm proposed by Rumelhart et al. in 1986 was used, the adaptive moment estimation (Adam) optimizer (Kingma and Ba, 2015) was used, the learning rate lr = 5 × 10^-4, and the mean square error (MSE) was used as the loss function.

Experimental results: High-accuracy 3D reconstruction

The uncertainty of the model is estimated by fitting an ensemble of five SuNeRFs with different initializations and calculating the standard deviation of the output.

Quality Assessment: Figure (a) shows the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) for each viewpoint in the simulation. SuNeRFs provides high-quality results with a minimum SSIM of 0.97. Points close to the ecliptic plane show the smallest error, while the error gradually increases with increasing latitude, which is in line with the expectation of the training-test split.

Model comparison: Figure (b) compares the model with the baseline method. At higher latitudes, the simple reprojection shows artifacts and large deviations from the ground truth, while the SuNeRF model renders almost the same image. The difference map shows that the main errors occur near the edge of the sun and in the edge region, which is also reflected in the uncertainty map. It is important to note that the reprojection method cannot handle the edge region.

Evaluating SuNeRF

a) PSNR and SSIM evaluated at 256 viewpoints, represented by points at the corresponding latitude and longitude. The color indicates the quality of the reconstruction, with larger values ​​indicating better consistency with the ground truth. The red dashed line at ± 7 degrees of latitude marks the separation between training and testing viewpoints;
(b) Qualitative comparison of the baseline method (spherical reprojection; first row), simulated data (ground truth; second row), and SuNeRF reconstructions (third row) at different latitudes. The difference maps (fourth row) identify the regions where our method deviates from the ground truth. The uncertainty estimates (fifth row) are consistent with the errors.

The table below summarizes the quantitative evaluation results on the entire test set. The SuNeRF model outperforms the baseline method by a large margin and shows no signs of over- or underestimation, achieving high accuracy in 3D reconstruction of the Sun.

Kuafu: The Chinese Dream of Chasing the Sun

The sun is the star that is most closely related to us and the only star that can be studied in detail. It brings us light and warmth, and also has a significant impact on the earth. Therefore, people have never stopped exploring the mysteries of the sun for many years, and "chasing the wind and the sun" has always been the dream of Chinese researchers.

The Xihe satellite launched in 2021 can be called the pathfinder of my country's solar exploration project, while the Kuafu-1 (ASO-S) is a versatile observer of the sun, which can observe the sun from ultraviolet, visible light and X-ray bands. The two solar exploration satellites launched by my country have their own focuses, and will jointly enhance my country's influence in the field of solar physics research in the world and become the strongest partner for Chinese scientists to "chase the sun".

As my country's comprehensive solar exploration satellite, Kuafu-1 has achieved three firsts:

1. For the first time, "one magnetic field and two storms" were selected as scientific targets and the corresponding payload combination was configured

2. For the first time, the full solar disk vector magnetic field, non-thermal radiation imaging of solar flares, solar disk formation of coronal mass ejections, and coronal propagation were simultaneously observed on a single satellite platform.

3. First simultaneous observation of the entire solar disk and the corona in the Lyman-alpha band

If the increasingly mature AI technology is a bow that is fully drawn, then human beings’ rich desire for knowledge is an arrow that flies into the mysterious deep space.

References:

https://ml4physicalsciences.github.io/2022/files/NeurIPS_ML4PS_2022_170.pdf

https://www.cas.cn/cm/202103/t20210316_4781101.shtml