APK Slimming Notes: How to Achieve a 53% Compression Effect

[[164861]]

1. How do I think about this matter?

APK is the file format of the Android system installation package. This topic is actually a commonplace topic. Whether it is within the company or on the external network, predecessors have summarized many methods and rules. However, with the rapid development of mobile technology in the past two years, some new ways of thinking and optimization methods have gradually emerged and matured. The author has stepped on some pits in the process of practice and gained some experience. Here is a reflection and summary, so I write it down for everyone, hoping that it will be helpful and reference for everyone when they are engaged in related work. At the same time, it is also a starting point. I hope everyone can discuss this open topic together.

I won't elaborate on why APKs need to be slimmed down. I'll just talk about three aspects. For users (or customers), the larger the APK, the more traffic they will consume during the download and installation process, and the longer the installation waiting time will be. For the product itself, it means a lower download conversion rate (because among competing products, users have more opportunities to choose the one with the best experience, the most features, the best performance, and the smallest package). For R&D, it is an opportunity to optimize and improve technology.

If we want to lose weight, we should first find out the reasons and problems of obesity. According to the goal-path-resource thinking mode, there are several ways to find the reasons and problems. The first is to use your own experience and judgment, or even subjective imagination; the second is to search for keywords on search engines, browse various technical forums to listen to what technical experts say, and read various technical articles to extract and refine; the third is to use a measurable tool or method to find the problem.

I won't go into details about the first two methods. I will talk about the third method. Use a measurable tool or method to analyze. As the saying goes, if you want to do your job well, you must first sharpen your tools. You can forge this tool yourself or use a ready-made one. Here I recommend an online apk analysis tool. Because it is an external tool, please do not upload unreleased products during use. For data security, I will use an open source Android project on github as a slimming example.
2. Find the problem

NimbleDroid is a system for analyzing Android app performance indicators developed by a team of doctoral entrepreneurs from Columbia University in the United States. There are two ways of analysis: static and dynamic. Static analysis can analyze the ranking of large files in APK installation packages, the size of various well-known SDKs and the proportion of the overall code, the size of various types of files and their ranking, the number of methods of various well-known SDKs and the proportion of the number of methods in all dex. Without further ado, let's take a look at the high-definition, uncensored pictures.

If you want to use the analysis function to analyze your own products, please log in and upload the apk package of your own product. All functions are currently free to use. If you want to analyze products that have been released on Google Play, you can directly click "Play Apps" to view them. You can also use the search function to view results based on the application name and package name. Once again, please do not upload any unreleased products.

Log in

Upload apk file

In the summary of the analysis results, you can see some overview information, apk file size, and the total number of methods.

File size analysis details page, large file list, here is the ranking of files in the apk file that are larger than 100k, the file size here refers to the size of the apk file

The size of various well-known SDKs and their proportion to the overall code. Currently, Android Support, Jackson JSON parser, Google Play Services, Paypal, Glide, OkHttp, Facebook SDK, Fabric, Gson, etc. can be identified here. Application refers to the part of the code written by yourself in the App

The size and ranking of various types of files

The proportion of various well-known SDKs in the number of methods in all dex

Ranking of methods of various well-known SDKs

Do you feel refreshed after reading this apk cross-section? I compare this analysis tool to the smart scale we bought at home, which can measure weight, fat content, bone weight, bone density, muscle content, etc. Then, have we found some problems, and then can we use logic to connect these problems with the reasons we relied on experience and intuition before.

Then, we can analyze the data to sort out our optimization goals.

• In the list of large files, 11 PNG files are over 100k in size. Remember, these are compressed files.
• In the large file ranking, the size of resources.arsc is close to 2M, which is also an optimization point;
• In the large file rankings, classes.dex is close to 3M. classes.dex is the carrier of code, and the optimization of this part needs to be subdivided. Then look at the rankings of segmented SDKs;
• In the component ratio chart, Android Support, Jackson JSON Parser, and Google Play Services are the top three third-party libraries;
• In the file type rankings, png, dex and arsc are the top three;

3. Sort out optimization goals

So our goal is to have no cavities, no, it is the following goal:

1. PNG image optimization;

2. Optimization of resources.arsc file;

3. Code Optimization
3.1 Attempts at Image Optimization

First of all, the first goal is to optimize the pictures. Slow down, let's see why these pictures are so big first. To be precise, why are these pictures so big in the apk (actually the zip file). Okay, let's use the tool to analyze.

This time I used a simple combination of tools, and the cmd that comes with the system is fine.

The results of the command execution are as follows

Well, all png files are stored in apk in STORE mode. For details about STORE and DEFLATE in zip, see)

In layman's terms, when a file is stored in zip in STORED mode, it means that the file is not compressed. If it is in Defl:N mode, it means that it is compressed and stored in zip in DEFLATED normal mode.

This seems a bit unreasonable. If the png file is put into the zip file intact, the resulting apk file will be larger. So, how to solve this problem? I first tried to use the android gradle plugin, but found that neither aaptOptions nor packagingOptions could solve the problem. I found an open source project AndResGuard on github and tried to integrate it into the project. The results are as follows:

Before optimization:

10536027 bytes

After optimization:

Normal zip compression: 8786265 bytes (compressed by nearly 17%)

Compressed with 7zip: 8567150 bytes (compressed by nearly 19%)

Let's take a look at what this tool does, and compare it before and after turning on resource obfuscation

Before optimization

After optimization

1. Resource (png, xml, jpg, etc.) name obfuscation, resource path name obfuscation and name length compression;
2. The png files originally stored in zip in STORED format have been changed to DEFLATED (normal compression storage);
3. I unexpectedly found that resources.arsc, META-INF/*.SF and META-INF/*.MF became smaller, and the file size after decompression also became smaller.

Use apk decompilation tool jadx to spy on apk and find the truth

It turns out that the relative paths of resources (png, xml, and properties files) in the apk are stored in META-INF/*.SF and META-INF/*.MF, and the SHA1 value is calculated for each resource file and stored in these two files. As for why this is done and the difference and function of these two SHA1s, please refer to the articles on this knowledge on the Internet. It is beyond the topic of this article, so I will not go into details here.

For resources.arsc file

It is easy to see that it is the resource file index table, so after reading this, you should understand why these three files have become smaller.
3.2 An unexpected discovery

Looking down resources.arsc, I found something interesting.

This will become an optimization point. Remove those useless translation resources and introduce some third-party SDKs. These SDKs often come with a lot of translation resources, such as the Android support library. Let's see the effect after removing them.

Assume that we only keep English. Of course, this is just an experiment. It depends on the specific situation in reality.

Compressed with 7zip: 8220738 bytes (compressed by nearly 22%, an increase of 3 points)

Of course, this is impossible in a real project, but every little bit counts!

In fact, what I want to say is that this provides an optimization idea, which is to use the gradle configuration to get rid of useless resources. The same can be used on the so local library and resolution (gradle configuration has been deprecated).

The gradle configuration example is as follows:

Remember to wrap it in android{}. Then, someone may ask, why is there no x86 in abi? It is said that Intel has provided a solution called houdini, which is a middleware running on x86 devices. It can transcode arm into x86 instructions, but the efficiency is very low. Some operations, such as calculating MD5 and SHA1, are even worse than java. The author has done a test comparison, which is another topic. I will not go into details here. Interested readers can move on.

So far, we have been moving towards the first goal. We accidentally discovered the relationship between the first goal and the second goal, so we used the resource obfuscation tool to achieve the second goal.

Using 7zip compression, we compressed the entire package by 2 points, which was a result beyond our expectations.
3.3 Image Optimization Methods

Regarding the first goal, our path is not over yet. The path I came up with is to compress PNG, convert non-alpha images to jpg, and what else? So I went to various technical forums and consulted various technical experts. The path I sorted out is as follows:

1. Manual lint check, manually delete resources that are not referenced in the code, the actual effect varies.

In Android Studio, open "Analyze" and select "Inspect Code...", select the entire project, and click "OK"

The configuration is as follows

2. Enable shrinkResources in gradle script

The script reference is as follows

shrinkResources works better with minifyEnabled. For details, see shrinkResources usage and notes

Compressed with 7zip: 8115283 bytes (compressed by nearly 23%, plus 1 point)

3. Use image compression tools to compress the size of PNG images and convert non-alpha images into jpg format. My colleagues and experts on the Internet have already sorted out this in detail. I will make a brief summary here. For details, please see the reference in the appendix.

Using tinypng, I just want to say that we are making products in the company, this solution should be used with caution, uploading any unreleased product content to the external network may cause data leakage, so use this solution with caution. The following is an alternative solution.
WASTED
pngquant
ImageAlpha
ImageOptim
The above tools are too scattered. Is there any integrated tool? The answer is "yes", imagemin developed by @心伦童鞋
MSImageResourcesHelper developed by @Brother-in-law
The specific effects of converting png to jpg format vary.

4. ***The killer, convert png to webp. For more details about webp, please refer to Google's official documentation and Android Developer Online Reference

First, the effect picture:

Compressed with 7zip: 4926912 bytes (compressed by nearly 53%, an increase of 30 points)

You read it right, it is 30 points. The current apk size is less than half of the original apk size, and I did it without changing a single line of code, I just used some tools!

To be honest, I didn’t take any diet pills or go on a hunger strike, but I lost half my weight!!!

However, it has not been used in the project yet because of two pitfalls.

On some Samsung models, some images with alpha backgrounds will have a very obvious black line. I won't show the image here. This problem is currently solved by whitelisting and not converting it into webp images.
On Xiaomi 2 phones updated to 4.xx, the webp image described in the xml file was not correctly recognized, resulting in the xml layout file loading after the interface appeared. The file failed to load the webp file and reported an error saying resource file not found, causing the app to crash. After tracking, it was found that the Xiaomi machine proxied the class Resource as MIUIResource, but this MIUIResource failed to correctly recognize webp, so the resource file failed to load. Preliminary judgment, there is no solution for the time being, so we can only reluctantly give up this optimization plan.

That’s all I have to say about the first goal, the optimization of image resources.
3.4 Code Optimization

The second goal has been achieved, and the third goal remains, code optimization, and the following optimization paths are sorted out:

1. Enable proguard code optimization

Will proguardFiles getDefaultProguardFile('proguard-android.txt'),'proguard-project.txt'

Change to proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'),'proguard-project.txt'

Please see the appendix for notes on enabling code optimization.

2. Remove unused libraries

If the latest version supported by the apk is API 14, and the code does not use any API higher than API 14, you can consider removing the entire android support library.

3. Use smaller library alternatives

If you only use Google Analytics, don't integrate the entire Google Play Services, just the required parts.

4. Clean up abandoned code regularly

Regularly delete useless logic and outdated business function modules, as well as abandoned A/B test codes.

5. The business module adopts a plug-in framework, and the code is dynamically pulled from the cloud

Pluginization is another topic and will not be discussed here.

The final result of apk weight loss record

10536027 bytes compressed to 4926912 bytes, a compression of nearly 53%
Summarize

• Enable resource obfuscation and resource compression in the script
• Use 7zip instead of zip
• Enable code obfuscation optimization and useless resource deletion in gradle script
• Use a smaller image and use compression tools to reduce the image size
• Remove useless resources, languages, local so libraries, second-party and third-party libraries, and resolutions
• Use smaller libraries
• Try to completely kick the android support library out of your project
• Clean up your code regularly
• Try to use H5 to write the interface and get pictures from the cloud
• Try plug-in business modules
• Find all files stored in the zip folder in the STORE format (not limited to the raw directory), try to compress them, and load these resources with alternative solutions
• Try webp image loading solution to seek breakthroughs

***, continue to learn and try new optimization solutions

This article is dedicated to the technicians who believe that “only slimming and products are worth living up to”! ! !