iOS development record of "send original image" function problem

This article mainly records the process of a bug from discovery, location to delayed resolution. The pitfalls that have been stepped on are added at the end of the article

[[208835]]

Recently, when working on the "send original image" function, I encountered a bug: when sending the original image on Android, Windows, or Mac clients, and receiving and saving it on iOS clients, the physical size of the original image remained unchanged, the storage space became smaller, and the corresponding Exif information such as location was lost. At the same time, there was no problem sending the original image between iOS clients. To address this issue, I conducted the following test investigations and now record them:

Send original image style in WeChat

1. First, let me introduce the process of sending an original image:

• For example, when sending an original image on the Android side, it is first uploaded to the IM server, and then the message body is sent after the upload is successful; (after the upload is successful, the server will assign three URLs corresponding to the thumbnail, large image, and original image respectively)
• The receiver receives the message body and downloads the thumbnail;
• Click on the thumbnail to download the full-size image;
• Then click the "View original image" button to download the original image;
• After successful download, long press the image to save the original image.

2. The problem is located in the last step, saving the picture:

• The size of the downloaded image is exactly the same as the size stored on the server. After saving, the size changes (currently, the "jpg" format becomes smaller, and the "png" format image becomes larger);
• The download methods tried AFNetwoking download, SDWebImage normal download and advanced download (because the product requirements require the display of the download progress of the original image)
• Conclusion The downloaded image size is exactly the same as the server storage size because there is a downloaded bit value in the completionBlock of the SDWebImage advanced download method
• I tried every way I could find to save the image, but none of them worked.

Verification test: Download the pictures generated by the Android end (including photos "jpg" and screenshots "png") from the browser to the computer, the size remains unchanged, drag the picture file to the project, execute the method to save the picture, and the size also changes.

Supplementary test: The Android client takes a picture (size is 5M), sends it to the iOS client (download size is 5M), saves it (size is 3M), then sends the saved picture to the Android client (save size is 3M), and the Android client sends it to the iOS client (save size is 3M). Conclusion: The compression is only performed once

After saving the image, the Exif information of the image is lost, but the size of the Exif information is much smaller than the size of the file loss. The physical size of the image has not changed.

3. Current status of this function of competing products:

• Android, Windows, Mac send original images, iOS client receives
• There is no problem sending and receiving original images on all terminals of Tencent (QQ, WeChat)
• DingTalk images are smaller and Exif information is lost
• The original image sent by BearyChat is smaller and the Exif information is lost
• Slack only supports sending pictures, not original pictures

IV. Analysis based on the current situation

• When iOS saves an image, it will encode and decode the image, and may optimize the bitmap.
• The storage size of an image is calculated as follows: horizontal pixels vertical pixels 1 color black and white or 3 primary colors * color depth bits = MB. This may be caused by different bases in different systems.
• If you want to solve this problem, you need to first study the very low-level things such as image encoding and decoding. At present, the cost performance is very low.
• If you have encountered a similar problem before and found an effective solution, please let me know in a private message if it is convenient.

5. Add some useful information

1. About the new ".heic" format images added in iOS11

What is a ".heic" format image?

Previously called "live" pictures, you can turn on this mode by clicking the button in the red box in the picture below. After taking a photo, a clip of about two seconds before and after the photo is captured. Unlike "Gif" pictures, this format also includes sound (currently only)

[[208836]]

What kind of mobile phone can take pictures in ".heic" format?

Only when using iOS 11 and the CPU is A10 or above (*** also requires iPhone 7), in other cases the pictures taken are ordinary "live" pictures, that is, they will be automatically converted to ".jpg/.jpeg" format when the format needs to be converted.

How to determine whether a picture is in ".heic" format?

SDWebImage-NSData+ImageContentType.m has been updated. When I first encountered this problem, I raised an issue and asked me to provide the corresponding URL.

 + (SDImageFormat)sd_imageFormatForImageData:(nullable NSData *)data {
 if (!data) {
 return SDImageFormatUndefined;
 } 
      
    // File signatures table : http://www.garykessler.net/library/file_sigs.html
 uint8_t c;
    [data getBytes:&c length:1];
 switch (c) {
 case 0xFF:
 return SDImageFormatJPEG;
 case 0x89:
 return SDImageFormatPNG;
 case 0x47:
 return SDImageFormatGIF;
 case 0x49:
 case 0x4D:
 return SDImageFormatTIFF;
 case 0x52: {
            if (data.length >= 12) {
                //RIFF....WEBP
                NSString *testString = [[NSString alloc] initWithData:[data subdataWithRange:NSMakeRange(0, 12)] encoding:NSASCIIStringEncoding];
                if ([testString hasPrefix:@ "RIFF" ] && [testString hasSuffix:@ "WEBP" ]) {
 return SDImageFormatWebP;
 }
 }
 break;
 }
 case 0x00: {
            if (data.length >= 12) {
                //....ftypheic ....ftypheix ....ftyphevc ....ftyphevx
                NSString *testString = [[NSString alloc] initWithData:[data subdataWithRange:NSMakeRange(4, 8)] encoding:NSASCIIStringEncoding];
                if ([testString isEqualToString:@ "ftypheic" ]
                    || [testString isEqualToString:@ "ftypheix" ]
                    || [testString isEqualToString:@ "ftyphevc" ]
                    || [testString isEqualToString:@ "ftyphevx" ]) {
 return SDImageFormatHEIC;
 }
 }
 break;
 }
 }
 return SDImageFormatUndefined;
 }
 + (nonnull CFStringRef)sd_UTTypeFromSDImageFormat:(SDImageFormat)format {
 CFStringRef UTType;
 switch (format) {
 case SDImageFormatJPEG:
            UTType = kUTTypeJPEG;
 break;
 case SDImageFormatPNG:
            UTType = kUTTypePNG;
 break;
 case SDImageFormatGIF:
            UTType = kUTTypeGIF;
 break;
 case SDImageFormatTIFF:
            UTType = kUTTypeTIFF;
 break;
 case SDImageFormatWebP:
            UTType = kSDUTTypeWebP;
 break;
 case SDImageFormatHEIC:
            UTType = kSDUTTypeHEIC;
 break;
 default :
 // default   is kUTTypePNG
            UTType = kUTTypePNG;
 break;
 }
 return UTType;
 }

How should we deal with ".heic" format pictures?

• First of all, it is definitely not the server that supports this type, because Windows and Android do not support the normal display of this type of pictures, especially Windows has clearly stated that it will not support it in the future.
• WeChat currently converts it into jpg format, so just use UIImageJPEGRepresentation(originalImage, 0.82); to convert it to jpg
• However, after many tests, it was found that the compression ratio must be set to 0.82 so that the converted size is as close to the original image size as possible.