This picture has deceived countless people. I can’t believe my eyes after seeing it!

First of all, let's look at the picture below. Is there a can of red Coca-Cola in it? Some people will say, I believe my eyes, it is absolutely true, there must be one.

But wait, zoom in on this picture on your phone or computer screen, and then look at it carefully. Do you feel something is wrong? There seem to be only some black, white and dark blue lines, but no real red. But when you shrink the picture back to its original size, the red Coke can appears again.

The "red Coke can" picture widely forwarded on social media (the right side is a partial enlargement) Source: jimhejl (twitter)

How can we explain this strange phenomenon that is both real and illusory? Is it true that if we have good in our hearts, we see only light, and if we have evil in our hearts, we see only darkness, and if we have Coca-Cola in our hearts, we see only red cans?

If your vision deceives you, don't be sad or anxious, optics can answer your questions. It is not uncommon for a photo to show colors that appear out of nowhere. This is related to the way the photo itself represents color information .

Whether it is a camera, display, printer, projector, or any other digital imaging device, when representing a picture, it is in the form of a pixel array . To put it simply, the entire picture is composed of many small squares as the basic unit, and each small square is called a pixel and has its own color.

Twenty or thirty years ago, the image quality of early video game consoles was rough, the resolution was low, and the size of each pixel was large. The image always had a "grid feel", like a mosaic. Later, with the development of display technology, the number of pixels in a picture increased, and the size of each pixel became smaller, so the whole picture looked more and more realistic, and the individual pixels seemed to be invisible. However, no matter how things change, the pixel array form has remained the same.

Of course, driven by the retro trend, the popular NFT avatars represented by CryptoPunk will still give you a clear "pixel sense" (the professional name of NFT is Non-Fungible Token, non-fungible token).

Screenshot of early video game consoles Source: Light Science/VEER

NFT avatar image source: Light Science Forum/VEER

We often talk about black and white photos and color photos, black and white movies and color movies. In fact, "black and white" should refer to "grayscale" . Although there is no color in it, it has a continuous transition from full black, dark gray, medium gray, light gray to full white (specifically, it can be expressed by grayscale values ​​0 to 255), which can also bring a considerable degree of image effect and expressiveness.

For example, the director of the famous film Schindler's List in the 1990s intentionally used grayscale effects without color in most of the images. But sometimes, due to technical limitations, true grayscale is impossible. For example, a simple printer or display can only display full black (0) and full white (255) in each pixel, which is equivalent to only two brushes, pure black and pure white, which can be called a true "black and white" binary display. Even a good cook cannot cook without rice. If you want to draw a photo with different shades of gray in different positions, you need to use your brain.

Grayscale image of a guy's photo (left), black and white binary image obtained by rough approximation (middle), black and white binary image obtained by pixel optimization (right) Image source: Light Science / VEER

For example, if we want to display this grayscale photo of a foreign guy, if we directly approximate the grayscale values ​​of different pixels to pure black or pure white, the light gray will be "rounded" to white, and the dark gray will be "rounded" to black. The final result will be as horrible as the second picture, a complete "clown face".

The third picture looks much more comfortable. It seems that the grayscale depth is variable, not just black and white. But if you look closely, there are still only these two kinds of pixels. It is just that an algorithm called " error diffusion " [1] is used to reasonably control the ratio of black and white pixels in different areas. The areas with relatively dense black dots represent dark areas, and the areas with relatively dense white dots represent bright areas. The spatial distribution is exchanged for the accuracy of color depth representation. It is like magic, bringing a complete grayscale viewing experience.

The same can be done by using a limited number of brushes to create a myriad of colors. In color display, red, green and blue are the three primary colors. Red light and green light can be mixed to produce yellow light, green light and blue light can be mixed to produce cyan light, and red light and blue light can be mixed to produce purple light. The result of mixing all three primary colors together is white light.

Ideally, a pixel can produce red, green and blue (RGB) light at the same time , and the intensity of the three channels can be freely adjusted, so that various colors can be easily mixed. But ideals are beautiful, and reality is skinny. If a pixel can only display red or blue alone, how can it produce purple?

In the checkerboard patterns below, each square is red or blue, with a 1:1 ratio. From left to right, as the grid gets smaller, the pixels get smaller and denser, and it seems that the red and blue are gradually disappearing, and eventually the whole pattern turns purple.

This is actually wrong!

All pixels in all patterns (including the one on the far right) contain only red and blue. Purple is an illusion created by the human eye fusing red and blue pixels , or in other words, a color that does not exist is perceived out of thin air.

The red and blue blocks gradually decrease in size and merge into purple. Image source: Jiao Shuming

In the left picture below, each pixel has only 256 possible colors, which makes it difficult to express richer color levels. However, after the pixels are distributed more reasonably, the picture on the right becomes obviously more pleasing to the eye.

The original 256-color image (left) and the optimized image (right) Image source: Wikimedia Commons

We can create new colors “out of thin air” through the spatial distribution of a finite number of color pixel locations, and the time dimension is another way.

To give a simple example, many people must have played such a game when they were young. Make a small disc with cardboard, paint the small disc alternately with red and yellow, and then put a small pillar through the middle of the small disc, so that the small disc can spin quickly like a top. What will you see?

That's right, the small disc is neither red nor yellow, but red and yellow flash in front of your eyes in rapid "shifts" and are "adjusted" to the middle orange. When the display screen plays a video, it is actually equivalent to dozens of static images flashing quickly in sequence every second, and the human eye happens to have a visual persistence effect, which overlaps them together. In this way, the two primary colors are displayed rapidly and alternately, which can naturally deceive your eyes to see the new color after fusion.

In fact, the image processing tricks mentioned above have a special name: dithering . This story dates back to the Second World War decades ago.

At that time, electronic computers had not yet been invented, and the US military's bombers could only use mechanical computers to calculate flight directions and bombing curves. A large box contained a large number of gears and lever parts. Although the calculation accuracy was not very good, the computers were also quite fragile and afraid of being bumped and dropped. Going up and down in the bomber cockpit was not as comfortable as sitting on the sofa at home. People were most worried that the violent shaking would cause some parts to fall off, causing the mechanical computer to stop working, the plane to lose its course, and the bomb would be dropped on its own position by mistake, which would cause big trouble.

However, this computer worked very well and was not damaged by the shaking on the plane. Surprisingly, the calculation results were more accurate than when it was used safely on the ground, which was puzzling. Since it had this "strange temper", engineers once specially designed a vibrator to simulate the flight state, so that the mechanical computer could also experience the feeling of flying on the ground to improve the calculation accuracy.

Why do these guys work better after being shaken up on a plane?

This is mainly because early mechanical parts cannot be accurate to many decimal places like today's computers, but only have a limited number of digits. Assuming that it can only be accurate to ten places (100, 110, 120, 130, 140, ...), the units place has to be rounded off. For example, 123 can only be expressed as 120, and 128 can only be expressed as 130. This is obviously not reliable enough. After step-by-step calculations, the errors will accumulate more and more.

Instead of expressing 123 with the "shortweight" 120 every time, a smarter way is to express 123 sometimes with 120 and sometimes with 130, but because it is closer to 120, the proportion represented by 120 should be slightly larger, just like mixing a cup of boiling water and a cup of zero-degree ice water in appropriate proportions can make warm water with a certain degree of hotness or coldness in between.

Shaking the mechanical computer is equivalent to introducing errors, which causes the calculation that originally rounded off normally to fail sometimes, and the number cannot be rounded off or rounded off. This " numerical blending " effect is achieved by accident. This idea of ​​adding some random small errors to eliminate the large errors when rounding off during calculation is a brilliant idea of ​​"fighting poison with poison", which is the earliest source of dithering technology [2].

In 1946, after the end of World War II, the world's first modern electronic digital computer ENIAC was born at the University of Pennsylvania . Mechanical computers gradually faded out of history, but dithering technology shines on another stage of color display technology.

From the early stages of development and even until today, one of the thorny problems encountered by various cameras and displays when faithfully recording and reproducing the colorful real world is the insufficient variety of colors. This was particularly evident in newspaper and magazine printing and television and computer displays decades ago.

Similar to the dithering method used to solve the problem of insufficient bits for representing numerical values ​​in early computers, just like a painter uses a palette, we can use multiple existing colors to blend non-existent colors. After all, human "eyesight" is not that good, so we can merge tiny pixels that are very close in position or pixels that change rapidly at different time points together for virtual color adjustment .

With the development of technology, the most common mobile phone displays can now directly support a wide range of colors, and there is no need to create new colors out of thin air. However, many professional display devices used in specific scenarios in laboratories still rely heavily on related technologies due to performance limitations, such as liquid crystal spatial light modulators and digital micromirror devices for holographic three-dimensional displays [3][4][5].

When it comes to color perception, human vision has many other incredible and wonderful phenomena. For example, when you first look at the picture below, do you think it’s a color photo?

Color assimilation grid optical illusion Image source: stuarthumphryes (twitter)

The truth is that most of the pixels in this image are grayscale, and it is impossible to distinguish colors . Only the pixels in the grid lines overlaid on the grayscale photo are colored.

Of course, the colors of these pixels are not set randomly, but are oversaturated compared to the original correct colors of the corresponding pixels in the photo. It can be simply understood that the colors of the pixels on the grid are much brighter than normal and have a high saturation. In comparison, the grayscale pixels in the background have no color at all and have a saturation of 0. The fusion of the two "extreme" pixels in the human eye makes the entire photo look normal in color.

This effect is known as the color assimilation grid illusion [6], and of course we can make the grid denser, down to the scale of individual pixels, making the fact that most pixels in the “false color” photo are grayscale even more difficult to detect.

Finally, let’s go back to the red Coca-Cola can at the beginning of the article. This is another misjudgment of color by the human eye. The strawberry picture below has a similar effect.

To produce this image effect, the specific steps are:

Decompose the original normal color photo into many tiny dots or fine lines. Take lines as an example, they are divided into two interlaced groups. In one group, the red component of the line pixels is set to 0, so the parts of these lines that originally had red appear black or the parts that originally had no red appear cyan. In the other group, the red component is kept unchanged, but the intensity values ​​of the green and blue components are set to the maximum (oversaturation), and these lines will appear white. Both interlaced lines are equivalent to increasing the intensity of green and blue and reducing the intensity of red, which is equivalent to viewing the original photo through a filter that removes red light. The whole picture looks like a cyan veil.

In this case, even objects that are actually red will appear black. Conversely, for objects that appear to be black on the surface, the human eye and brain instinctively tend to restore them to red. This is how Coke cans and strawberries are "dyed red". This is called the color constancy illusion [7].

Original color image Light Science Center/VEER

The illusion of red is created by two sets of staggered lines (the right side is a magnification of the lines: where is the red?)

Image source: Light Science Center/VEER

The illusion of red is created by two sets of staggered squares (a zoomed-in version of the grid is on the right: where is the red?)

Image source: Light Science Center/VEER

The above “scams” ​​are enough to prove that human vision is extremely weak in distinguishing colors at certain times, but an even more outrageous example[8] is below.

We instinctively think that the colors of the two objects above and below are obviously different, one is dark gray and the other is white. But if you use a finger to cover the intersection between the two, you will find that the colors of the two objects are exactly the same! This result will definitely drive you crazy.

Figure 11: A picture that subverts your ability to judge color depth

Image source: American Scientist.org

After reading this article, do you still believe your own eyes?

They do often make all sorts of inexplicable mistakes, and each time they make a mistake they correct it and make it again. However, it is precisely this kind of well-intentioned deception that prompts you to "fill in the blanks" with colors that do not exist in real objects, making the world you perceive more colorful.

Author: Jiao Shuming (Pengcheng Laboratory)

Review｜Cao Liangcai (Tsinghua University)

References:

[1] RW Floyd, L. Steinberg, An adaptive algorithm for spatial gray scale. Proceedings of the Society of Information Display 17, 75–77 (1976).

[2] Ken C. Pohlmann (2005). Principles of Digital Audio. McGraw-Hill Professional.

[3] S. Jiao, D. Zhang, C. Zhang, Y. Gao, T. Lei, and X. Yuan, "Complex-amplitude holographic projection with a digital micromirror device (DMD) and error diffusion algorithm", IEEE Journal of Selected Topics in Quantum Electronics, 26(5), 2800108 (2020)

[4] X. Yang, S. Jiao, Q. Song, G.-B. Ma, and W. Cai, "Phase-only color rainbow holographic near-eye display", Optics Letters 46(21), 5445-5448 (2021)

[5] K. Liu, Z. He, and L. Cao, “Pattern-adaptive error diffusion algorithm for improved phase-only hologram generation,” Chinese Optics Letters 19(5), 050501 (2021).

[6] S. Jiao and J. Feng, "Image steganography with visual illusion," Opt. Express 29(10), 14282-14292 (2021)

[7]https://www.wired.com/story/remember-the-dress-heres-why-we-all-see-colors-differently/

[8] Purves D, Lotto RB, Nundy S. Why we see what we do: A probabilistic strategy based on past experience explains the remarkable difference between what we see and physical reality. American Scientist. 2002 May 1;90(3):236-43.

Source: Light Science Forum

The cover image of this article is provided by Light Science and Technology