Is 100 million enough? Comparing chips and configurations, do mobile phone manufacturers still have a "pixel" card?

• Jobs said, "The focus on high pixels in mobile phone photography is a manifestation of mobile phone manufacturers taking advantage of consumers' lack of knowledge about the principles of photo imaging."

Xiaomi has released a concept phone with 100 million pixels. Yes, it is a "concept phone". Of course, this is not the point. The point is that Xiaomi has 100 million pixels, while this year's new iPhone camera has a maximum pixel of only 12 million. The contrast is quite strong.

Many people say, "People who dare to take pictures of their faces with iPhone are those who are very confident in their looks." In other words, at a time when major mobile phone manufacturers are clamoring that their mobile phones' camera performance can rival that of SLRs, the iPhone is not competitive at all. After all, it has been stagnant at the 12-megapixel mark for several years.

But is the iPhone's camera really that bad? Is pixel really the only factor that determines camera performance? Before figuring out these questions, let's talk about the importance of pixels for photography.

Things about pixels

• What is a pixel?

The description of "pixel" in an encyclopedia is as follows:

"Pixels are small squares of an image. These small squares have a clear position and assigned color values. The color and position of the small squares determine what the image looks like."

This may be a bit obscure, let's take a look at the following picture:

Taking a square as a unit, it is obvious that the left image has much fewer pixels than the right one, which is reflected in the final image. The image on the right has higher clarity, and even the dew on the petals is clearly visible. In short, in an ideal state, the higher the pixel, the finer the image, which is also the main reason why major mobile phone manufacturers are scrambling to pursue high pixels.

But increasing the number of pixels is not a simple matter. The first hurdle is to ensure that the image sensor has sufficient image resolution capabilities. The key technology in this area has long been monopolized by Samsung and Sony, or Sony alone.

• Sony and Samsung's battle, from CCD to CMOS

As the basic hardware of photographic equipment, the most common image sensor today is CMOS. The "XXXX million" in the XXXX megapixel camera we usually refer to the number of photoelectric sensing elements on the CMOS.

But in fact, at the beginning, when photography was still limited to the field of cameras, CCD was more widely used, and Sony was the absolute leader in this field. As early as 1969, when CCD was invented at Bell Labs, Sony quickly entered the market.

Whether it is the original positive hole accumulation layer, the introduction of HAD sensors and their application in variable speed electronic shutter products, or the innovative installation of tiny lenses in front of each photodiode, or the introduction of SUPERHAD CCD technology, which laid the foundation for CCD technology, Sony's related products are at the forefront of the industry in terms of pixel integration and cost control, relying on the application of ultra-large-scale integrated circuit process technology, and are also far ahead in the market. The situation did not change until around 2007, when the demand for image sensors in mobile phones increased significantly.

Samsung has been trying hard to make a big splash in the photography field, but it has never had the ability to mass-produce CCDs. With the popularity of smartphones and the increasing demand for photography, CMOS, which can output digital signals, has simple processes, lower power consumption and lower costs, has become the first choice.

Compared with CCD, although CMOS is slightly weaker in photosensitivity and image analysis and processing, mobile phones are not professional cameras after all, and the requirements are not that high. Samsung also took notice of this point and quickly made up for the previous process and production capacity issues, mass-produced CMOS. In addition, its own smartphones performed well in the market. The combination of the two has made it a latecomer and a leader.

Of course, Sony was not late to the party. As early as 2007, it released its first Exmor sensor, which not only had a built-in ADC (analog-to-digital converter), but also achieved this in a small sensor size, successfully winning the "entry certificate" to the CMOS market. In 2008, the company launched the Exmor R sensor using BSI (back-illuminated) technology, successfully entering the mobile phone market and winning orders for the iPhone 4s.

Although Samsung has a weak foundation and entered the market late, it has relied on its advantages in the semiconductor industry to launch the ISOCELL image sensor targeting Sony Exmor before the CMOS market broke out in 2017. In 2018, it also launched the 16-megapixel ISOCELL Slim 3P9, which is not only plug-and-play, but also supports four-pixel integration. This year, it launched the 64-megapixel sensor GW1 for the first time, as well as a 108-megapixel image sensor. The latest 43.7-megapixel sensor ISOCELL Slim GH1 has a unit pixel of only 0.7μm.

It seems that Samsung is ahead of Sony in technology. But you can only know whether a product is good by using it.

Are mobile phone manufacturers eager for quick success and instant benefits behind the soaring pixel count?

As we can see, as camera performance is becoming more and more important, high pixels have become an important selection criterion for consumers when buying mobile phones, and almost all mobile phone manufacturers have put all their efforts in this regard. But in fact, as early as 2010, Steve Jobs said, "Many mobile phone manufacturers take advantage of consumers' lack of knowledge of photo imaging principles and focus on high pixels in mobile phone photography. However, pixels are only one of the factors that affect the performance of mobile phone photography."

Here is a comparison:

• When Xiaomi released its first mobile phone in 2011, it had an 8-megapixel rear camera. However, the Xiaomi Mi 9 released earlier this year has a 48-megapixel rear camera.
• The iPhone 4s released by Apple in 2011 also had an 8-megapixel rear camera, but the newly released iPhone 11 only has a 12-megapixel rear camera. And since the iPhone 6s adopted a 12-megapixel camera in 2015, the iPhone has not been upgraded in this regard for 5 years.
• In the same 8 years, Xiaomi has increased its pixel by 6 times, while iPhone has only doubled it. Does that mean Apple is not as good as Xiaomi? Actually, that's not entirely true.

There is a saying in the field of photography, "a bigger sensor is better than a bigger one", which means "when the pixels are the same, a larger sensor will receive more light per pixel, and the photo effect will be better". When it comes to mobile phone cameras, the larger the CMOS area, the better the photo performance. The iPhone camera has not changed its pixels for 5 years, but its photo performance is getting better and better, because it has made multiple upgrades in the CMOS area. When it comes to the external features of the camera, the most intuitive thing is that the aperture is getting bigger and bigger.

Taking the previous generations of iPhone as an example, iPhone 4/5 all used Sony IMX145, with a CMOS area of ​​1/3.2 and a unit pixel of 1.4μm; iPhone 6s used Sony's newly customized IMX315, with a CMOS area of ​​1/3, a single pixel area increased to 1.22μm, and an aperture of f/2.2; iPhone X's CMOS is the smallest at 1/2.6, and the wide-angle lens has an aperture of ƒ/1.8.

That is to say, Apple has been working on improving the quality of individual pixels, such as the amount of light intake of each pixel and the coordination of color depth.

As for Xiaomi, the number of camera pixels increased directly from 12 million to 48 million from Xiaomi 8 to Xiaomi 9. However, some Zhihu netizens said that the actual photo-taking effect of Xiaomi 9 is seriously smeared and the sharpness is not as good as Xiaomi 8. In the latest DxOMark ranking, it is only seventh in the world.

The 108-megapixel CMOS equipped in the MIX Alpha released today has a size of 1/1.33, but the large pixel is only 1.6μm. In other words, the quality of a single pixel does not actually reach the level of 100 million pixels.

In fact, even for full-frame cameras, the CMOS size is at least 10 times that of a mobile phone, not to mention that companies including Fuji and Hasselblad have launched products with only 50 million pixels. It is indeed incredible that mobile phone cameras can suddenly have hundreds of millions of pixels.

Of course, like pixels, CMOS area is only one of the important factors affecting photo effects. There are many other ways to improve photo performance. For example, Huawei, which has been stuck at the "40 million mark", has put a lot of effort into filters.

What else can we do besides pixels?

In March this year, Huawei launched the P30, which is said to be able to "take photos of the moon at any time". It has a rear camera with a 40-megapixel main camera, a 16-megapixel ultra-wide-angle camera, and an 8-megapixel telephoto camera. The biggest highlight is the innovative RYYB sensor design, which replaces the green pixels in the three primary colors with yellow pixels, increasing the amount of light by 40%, ensuring the image quality in low-light environments. The newly released Mate 30 Pro also uses this design.

What is the power of RYYB? Here we have to talk about the Bayer array. The Bayer array is an important sensor design for modern color digital photography and one of the main technologies for realizing color image capture with CCD or CMOS sensors. In simple terms, it is a 4×4 array consisting of 8 green, 4 blue and 4 red pixels, which is what we usually call "RGGB".

However, as the market's requirements for camera performance increase, more and more people have found that the insufficient grayscale resolution of RGGB has been magnified in ultra-high-definition or high-definition shooting. The main reason is that the three primary colors of RGB are additive and have limited light absorption. For example, when shooting high-definition textiles, due to the small color difference between fibers, cross-color is prone to occur.

The RYYB filter launched by Huawei reshapes the three primary colors of red, green and blue, replacing green with yellow, that is, by sacrificing the amount of green light entering, increasing the amount of yellow light entering, and in disguise increasing the amount of red light entering (yellow light is a mixture of red light and green light), thereby improving the camera's performance in low-light environments. In terms of actual effect, the amount of light entering RYYB is 40% higher than that of RGGB, which is why the P30 can clearly shoot the moon at night.

However, all innovative designs require strong analytical technology support, which involves processor performance. Currently, there are three main types of high-end mobile SoCs:

• Qualcomm Snapdragon 800 series has been upgraded to 855 Plus, which has limited improvement over 855. The large core speed has been increased to 2.96GHz, and it is considered to be a 5G transition chip.
• Apple's A series has now been upgraded to A13, using the second-generation 7nm process technology. A single chip has 8.5 billion transistors and can run 1 trillion times per second. The CPU uses a combination of "2 high-performance cores + 4 high-performance cores", the GPU has 4 cores, and the power consumption is reduced by 40%. The NPU has 8 cores;
• Huawei's Kirin 900 series, the latest one is the Kirin 990, which integrates 10.3 billion transistors, uses TSMC's 7nm+EUV process, and has modified the architecture to "dual large cores + dual medium cores + four small cores".

At present, the latter two have received more attention, which is reflected in mobile phones. For example, taking video shooting, which is more difficult to analyze, the iPhone 11Pro can shoot at 4K/60fps and multiple cameras at the same time; while the Mate 30 can achieve 4K HDR and 4K time-lapse shooting, video ISO up to 51200 frames, and supports 7680 fps ultra-high-speed video shooting. In terms of photography, anti-shake, AI blur, depth of field, dark night, macro, telephoto, wide angle, etc. can all be achieved.

In addition, Huawei has also made hardware upgrades to ISP. For example, the latest Kirin 990 uses ISP 5.0, which can achieve real-time multi-instance segmentation. In the face detection scenario, the micro-core energy efficiency is improved by 40%, and it supports mobile phone-side BM3D SLR-level image noise reduction technology and dual-domain joint video noise reduction technology.

Of course, in addition to technical means, directly stacking cameras is also one of the main ways. However, nowadays, the maximum number of rear cameras on mobile phones has been upgraded to 5. Even the iPhone, which insisted on only having a single camera in the past, has launched a rear triple camera. But if mobile phones become like this in the future, "trypophobic patients" may find it really difficult to accept it.