The blue sky and white clouds were all gone. It’s not our fault. It’s because the camera is not powerful enough.丨World Photography Day

Produced by: Science Popularization China

Author: Salty Fish in the Sea (Master of Optics from Changchun Institute of Optics and Fine Mechanics, Chinese Academy of Sciences)

Producer: China Science Expo

August 19 is World Photography Day, a festival for photographers and photography enthusiasts around the world. We point our lenses at the sky, at the clouds, at the shutter, and at the passion in our hearts.

Have you ever noticed that on a sunny day, the abundant sunshine makes the whole world bright and gorgeous? You took out your phone to record this moment, but found that the sky you took was pale, the blue sky was gone, and the white clouds were also submerged in the whiteness. Why is this?

What the human eye sees

Captured by camera

Image source: self-made by the author

This problem is actually caused by the fact that the lighting conditions in the natural environment are too complex and the cameras we currently use are not powerful enough.

The lighting conditions in the natural environment are very complex, and the brightness changes greatly. However, the human eye can automatically adapt to environments of different brightness , so in our view, the brightness changes of the external environment do not seem too drastic. If we use a quantifiable unit to describe the size of the ambient light, we can feel the difference in brightness more intuitively from the numbers.

Lux (symbol: lx ) is the international unit of illumination, which reflects the intensity of light on the surface of an object . It is used to indicate the brightness of the environment and is an important indicator for measuring the shooting environment. On a cloudy day, the outdoor light illumination can reach about 20,000 lx, and on a sunny day, it can reach 50,000 lx. The illumination of indoor places such as offices or supermarkets is between 300 and 1,000 lx; ​​on a moonlit night, the illumination is only about 0.08 lx. Therefore, from a numerical point of view, the difference in illumination values ​​in different environments can jump between hundreds and tens of thousands of times.

How does the human eye observe environments of different brightness?

When observing environments of different brightness, the human eye can adjust the amount of light entering the retina according to the ambient brightness in real time, allowing the human eye to see clearly regardless of whether it is in a bright or dim environment.

The human eye is constantly moving. When we adjust our gaze position, we are also re-sensing the lighting conditions of the external environment, and then adjusting the size of the pupil to control the amount of light entering. When encountering a very bright environment, the human pupil will shrink appropriately to reduce the amount of light entering, and in a dim environment, the pupil will expand to increase the amount of light entering.

Human eye pupil

(Photo source: VEER Photo Library)

The human eye can not only use the pupil to adjust the amount of light entering, but also adapt to extremely dark environments by switching different visual cells. There are two main types of visual cells in the human eye: cones and rods. During the day, we mainly rely on cones, which can perceive light intensity and color at the same time, but in extremely dark environments at night, we will switch to rods to receive light because they are more sensitive to light, but the disadvantage is that they cannot distinguish colors.

How does a mobile phone camera take photos and record environments of different brightness?

But unlike the human eye, the phone lens is not that flexible. The size of the phone lens determines the amount of light that enters, and the sensor determines the camera's photosensitivity. When taking pictures, the phone algorithm automatically determines the exposure time and sensor sensitivity based on the external light brightness data detected. When taking pictures, the phone camera records the external environment at one time, but because the camera can only record a limited range of illumination of the external environment, areas that are too dark or too bright in the environment will become pure black or pure white in the photo, losing a lot of details. Therefore, the blue sky and white clouds mentioned at the beginning of the article turned into a pale white after hundreds of photos - the sky was too bright and exceeded the camera's photosensitivity limit.

In contrast, in reality, when faced with a scene with a large contrast between light and dark, the human eye actually changes the area of ​​focus. When focusing on a bright area, the human eye will shrink the pupil, and when focusing on a dark area, the human eye will expand the pupil. This is a process of dynamic adaptation and acceptance of information, and the human eye will naturally retain more details than photos.

There is a special term that describes the difference in brightness between the external environment, which allows us to more intuitively understand the difference in capabilities between the human eye and the camera: dynamic range.

Dynamic range is the ratio of the maximum and minimum values ​​of a variable signal (such as sound or light). In an environment, the ratio of the illumination at the brightest point to the illumination at the darkest point is called the dynamic range of the environment . The human eye also has a dynamic range for perceiving the environment, which is the ratio of the maximum brightness to the minimum brightness that the human eye can detect; similarly, the camera also has its own dynamic range . This is one of the important factors that everyone pays attention to when buying a camera. So, what is the dynamic range of the natural environment, the human eye, and the camera? See the figure below:

The dynamic range of the camera is several orders of magnitude lower than that of the natural environment.

(Image source: self-made by the author)

Thanks to the human eye's strong ability to adapt, we can adapt to almost all lighting conditions in the natural environment (except looking directly at the sun, which is not recommended as it will burn your eyes). The dynamic range of mobile phone cameras is somewhat limited compared to the dynamic range in the natural environment.

How can we capture as many details of real scenes as possible?

Increasing the diameter of the lens is the most direct way, as it can increase the amount of light entering the camera, which helps record dim scenes. But what about bright scenes? Too much light entering is not good either, and you need to add an aperture that can adjust the amount of light entering. Another option is to use a sensor with a higher sensitivity and a larger photosensitivity range.

Schematic diagram of mobile phone camera structure

(Photo source: VEER Photo Library)

In fact, in order to improve the dynamic range performance of the camera, the most common method is to use image processing technology to process the image in the later stage. Almost every brand of mobile phone is equipped with an image processing technology called "HDR" (high dynamic range imaging). If the mobile phone turns on the HDR mode when taking pictures, every time we click the camera button, the camera will use different sensitivities to quickly take multiple photos of the same scene with different brightness and darkness. In the later stage, the algorithm will be used to combine multiple photos into one photo, so as to achieve the purpose of improving the dynamic range.

HDR Extended Dynamic Range Principle

(Image source: self-made by the author)

HDR Examples

(Image source: self-made by the author)

However, the actual effect of HDR is not always ideal. Some pictures processed by HDR algorithm may look unnatural.

Photos with poor HDR processing

(Image source: self-made by the author)

Therefore, in practice, when photography enthusiasts use cameras to take pictures, once they encounter a scene with a large change in brightness, most of them will adjust the camera's exposure to make the scene slightly darker, so as to retain the details in the bright areas and obtain a larger dynamic range as much as possible through post-adjustment of the image. Similarly, when taking pictures with a mobile phone, you can first click the screen and then slide it up and down to adjust the exposure. When taking pictures in an environment with a large change in brightness, take a slightly darker picture to leave more room for post-adjustment.

References:

[1] Xu R .REAL-TIME REALISTIC RENDERING AND HIGH DYNAMIC RANGE IMAGE DISPLAY AND COMPRESSION[J].Dissertation Abstracts International, 2005.DOI:http://dx.doi.org/.

[2] Wang Hawk HDR - High Dynamic Range Imaging

[3] Fattal R , Lischinski D , Werman M .Gradient Domain High Dynamic Range Compression[C]//Conference on Computer Graphics & Interactive Techniques.ACM, 2002.DOI:10.1145/566570.566573.

[4] Zhao Kaihua. New Concept Physics Course. Optics[M]. Higher Education Press, 2004.

[5] Hao Yunxiang, Chen Xiaju, Zhang Baozhou. Photometry[M]. China Metrology Press, 2010.