Understanding Code Obfuscation in Android

This article is a summary of my practice, introducing some knowledge and precautions of obfuscation. I hope it can help you learn and use code obfuscation better.

What is obfuscation

Regarding confusion, the explanation of this entry on Wikipedia is

Obfuscated code, also known as junk instructions, is the act of converting the code of a computer program into a form that is functionally equivalent but difficult to read and understand.

The elements affected by code obfuscation are

• Class Name
• Variable Name
• Method Name
• Package Name
• Other Elements

Purpose of obfuscation

The purpose of obfuscation is to increase the cost of decompilation, but it cannot completely prevent decompilation.

How to turn on obfuscation

• Usually we need to find the build.gradle file in the app directory under the project path
• Find the minifyEnabled configuration and set it to true.

A simple example is as follows

 buildTypes { 
 
 release { 
 
 minifyEnabled true   
 
 proguardFiles getDefaultProguardFile( 'proguard-android.txt' ), 'proguard-rules.pro'  

What is proguard

Java official website's definition of Proguard

ProGuard is a free Java class file shrinker, optimizer, obfuscator, and preverifier. It detects and removes unused classes, fields, methods, and attributes. It optimizes bytecode and removes unused instructions. It renames the remaining classes, fields, and methods using short meaningless names. Finally, it preverifies the processed code for Java 6 or higher, or for Java Micro Edition.

• Proguard is a tool that combines file compression, optimization, obfuscation and verification functions
• It detects and removes unused classes, variables, methods and properties
• It optimizes the bytecode and removes useless instructions.
• It obfuscates class names, variable names, and method names by renaming them to meaningless names.
• Finally, it also verifies the processed code

Common configurations for obfuscation

-keep

Keep is used to keep Java elements from being obfuscated. There are many variants of keep, and they are generally

• -keep
• -keepclassmembers
• -keepclasseswithmembers

Some examples

Keep the classes and subpackages under a package

 -keep public class com.droidyue.com.widget.**

Keep all public methods that use Otto in the class

 # Otto 
 
 -keepclassmembers class ** { 
 
 @com.squareup.otto.Subscribe public *; 
 
 @com.squareup.otto.Produce public *; 
 
 }

Keep the BOOK_NAME property of the Contants class

 -keepclassmembers class com.example.admin.proguardsample.Constants { 
 
 public   static java.lang.String BOOK_NAME; 
 
 }

For more information about Proguard keep, please refer to the official documentation

-dontwarn

dontwarn is inseparable from keep, especially when dealing with imported libraries.

The introduced library may have some references that cannot be found and other problems, which may cause warnings during build. If we do not handle them, it will usually cause the build to terminate. Therefore, in order to ensure that the build continues, we need to use dontwarn to handle the warnings of these libraries that we cannot solve.

For example, to turn off the Twitter SDK warning, we can do this

 -dontwarn com.twitter.sdk.**

Other obfuscation-related introductions can be obtained by visiting the official documentation.

What should not be confused

Elements used in reflection

If some obfuscated elements (attributes, methods, classes, package names, etc.) are obfuscated, problems may occur, such as NoSuchFiledException or NoSuchMethodException.

For example, the following example source code

 //Constants.java 
 
 public class Constants { 
 
 public   static String BOOK_NAME = "book_name" ; 
 
 } 
 
 //MainActivity.java 
 
 Field bookNameField = null ; 
 
 try { 
 
 String fieldName = "BOOK_NAME" ; 
 
 bookNameField = Constants.class.getField(fieldName); 
 
 Log.i(LOGTAG, "bookNameField=" + bookNameField); 
 
 } catch (NoSuchFieldException e) { 
 
 e.printStackTrace(); 
 
 }

If the Constants class above is obfuscated, then the above statement may throw a NoSuchFieldException.

To verify, we need to take a look at the obfuscated mapping file, named mapping.txt, which stores the mapping relationship before and after obfuscation.

 com.example.admin.proguardsample.Constants -> com.example.admin.proguardsample.a: 
 
 java.lang.String BOOK_NAME -> a 
 
 void <init>() -> <init> 
 
 void <clinit>() -> <clinit> 
 
 com.example.admin.proguardsample.MainActivity -> com.example.admin.proguardsample.MainActivity: 
 
 void <init>() -> <init> 
 
 void onCreate(android.os.Bundle) -> onCreate

From the mapping file, we can see

• The Constants class was renamed to a.
• BOOK_NAME of the Constants class renamed a

Then, we decompile the APK file to find out. I recommend this online decompilation tool decompile Android .apk ✓ ONLINE ✓

Note that after using jadx decompiler, it will be renamed, as shown in the comment /* renamed from: com.example.admin.proguardsample.a */ below.

 package com.example.admin.proguardsample; 
 
 /* renamed from : com.example.admin.proguardsample.a */ 
 
 public class C0314a { 
 
 public   static String f1712a; 
 
 static { 
 
 f1712a = "book_name" ; 
 
 } 
 
 }

The translated source code of MainActivity is

 try { 
 
 Log.i( "MainActivity" , "bookNameField=" + C0314a.class.getField( "BOOK_NAME" )); 
 
 } catch (NoSuchFieldException e) { 
 
 e.printStackTrace(); 
 
 }

The attribute name obtained by reflection in MainActivity is still BOOK_NAME, but the corresponding class no longer has this attribute name, so a NoSuchFieldException is thrown.

Note that if the above filedName uses a literal or string constant, there will be no NoSuchFieldException even if obfuscation is used. This is because in these two cases, obfuscation can sense the reference to filed from the outside world and replace it with the obfuscated name when calling it.

GSON serialization and deserialization

GSON is a great tool that allows us to easily serialize and deserialize data. But when it encounters confusion, we need to pay attention.

A simple class Item to handle serialization and deserialization

 public class Item { 
 
 public String name ; 
 
 public   int id; 
 
 }

Serialized code

 Item toSerializeItem = new Item(); 
 
 toSerializeItem.id = 2; 
 
 toSerializeItem. name = "Apple" ; 
 
 String serializedText = gson.toJson(toSerializeItem); 
 
 Log.i(LOGTAG, "testGson serializedText=" + serializedText);

Log output after enabling obfuscation

 I/MainActivity: testGson serializedText={ "a" : "Apple" , "b" :2}

The attribute name has been changed to a meaningless name, which is very troublesome for some of our subsequent processing.

Deserialization code

 Gson gson = new Gson(); 
 
 Item item = gson.fromJson( "{\"id\":1, \"name\":\"Orange\"}" , Item.class); 
 
 Log.i(LOGTAG, "testGson item.id=" + item.id + ";item.name=" + item. name );

The corresponding log result is

 I/MainActivity: testGson item.id=0; item.name = null  

It can be seen that after obfuscation, the deserialized attribute value setting failed.

Why?

Because deserialization creates objects by reflection, the key of the json string is used as the attribute name, and the value corresponds to the attribute value.

How to solve it

• Eliminate confusion between serialization and deserialization classes
• Annotate fields with @SerializedName

@SerializedName(parameter) is implemented through annotation attributes

• In the serialized result, specify the attribute key as the value of parameter.
• In the deserialized object, it is used to match key and parameter and assign attribute values.

A simple usage is

 public class Item {
 @SerializedName( "name" )
 public String name ;
 @SerializedName( "id" )
 public   int id;

Don't confuse enumerations

Enumeration is a very convenient feature introduced in Java 5, which can be a good replacement for the previous constant form.

Enumeration is very simple to use, as follows

 public enum Day {
 MONDAY,
 TUESDAY,
 WEDNESDAY,
 THURSDAY,
 FRIDAY,
 SATURDAY,
 SUNDAY
 }

Here we use enumeration like this

 Day   day = Day .valueOf( "monday" );
 Log.i(LOGTAG, "testEnum day=" + day );

Running the above code usually works fine. Does this mean that enumerations can be obfuscated?

Actually not.

Why is there no problem? Because the default Proguard configuration has already handled the enumeration-related keep operations.

 # For enumeration classes, see http://proguard.sourceforge.net/manual/examples.html#enumerations 
 
 -keepclassmembers enum * { 
 
 public   static **[] values ​​(); 
 
 public   static ** valueOf(java.lang.String); 
 
 }

If we manually remove this keep configuration and run again, an exception like this will appear.

 E AndroidRuntime: Process: com.example.admin.proguardsample, PID: 17246 
 
 E AndroidRuntime: java.lang.AssertionError: impossible 
 
 E AndroidRuntime: at java.lang.Enum$1. create (Enum.java:45) 
 
 E AndroidRuntime: at java.lang.Enum$1. create (Enum.java:36) 
 
 E AndroidRuntime: at libcore.util.BasicLruCache.get(BasicLruCache.java:54) 
 
 E AndroidRuntime: at java.lang.Enum.getSharedConstants(Enum.java:211) 
 
 E AndroidRuntime: at java.lang.Enum.valueOf(Enum.java:191) 
 
 E AndroidRuntime: at com.example.admin.proguardsample.aa(Unknown Source) 
 
 E AndroidRuntime: at com.example.admin.proguardsample.MainActivity.j(Unknown Source) 
 
 E AndroidRuntime: at com.example.admin.proguardsample.MainActivity.onCreate(Unknown Source) 
 
 E AndroidRuntime: at android.app.Activity.performCreate(Activity.java:6237) 
 
 E AndroidRuntime: at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1107) 
 
 E AndroidRuntime: at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2369) 
 
 E AndroidRuntime: at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2476) 
 
 E AndroidRuntime: at android.app.ActivityThread.-wrap11(ActivityThread.java) 
 
 E AndroidRuntime: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1344) 
 
 E AndroidRuntime: at android.os.Handler.dispatchMessage(Handler.java:102) 
 
 E AndroidRuntime: at android.os.Looper.loop(Looper.java:148) 
 
 E AndroidRuntime: at android.app.ActivityThread.main(ActivityThread.java:5417) 
 
 E AndroidRuntime: at java.lang.reflect.Method.invoke(Native Method) 
 
 E AndroidRuntime: at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726) 
 
 E AndroidRuntime: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616) 
 
 E AndroidRuntime: Caused by : java.lang.NoSuchMethodException: values ​​[] 
 
 E AndroidRuntime: at java.lang.Class.getMethod(Class.java:624) 
 
 E AndroidRuntime: at java.lang.Class.getDeclaredMethod(Class.java:586) 
 
 E AndroidRuntime: at java.lang.Enum$1. create (Enum.java:41) 
 
 E AndroidRuntime: ... 19 more

Now comes the fun part. Let's see why this exception is thrown.

1. First, an enumeration class will generate a corresponding class file, here is Day.class. What does this class contain? Let's take a look at the decompiled results.

 proguardsample javap Day   
 
 Warning: Binary file Day   contains com.example.admin.proguardsample. Day   
 
 Compiled from   "Day.java"   
 
 public final class com.example.admin.proguardsample. Day extends java.lang.Enum<com.example.admin.proguardsample. Day > { 
 
 public   static final com.example.admin.proguardsample. Day MONDAY; 
 
 public   static final com.example.admin.proguardsample. Day TUESDAY; 
 
 public   static final com.example.admin.proguardsample. Day WEDNESDAY; 
 
 public   static final com.example.admin.proguardsample. Day THURSDAY; 
 
 public   static final com.example.admin.proguardsample. Day FRIDAY; 
 
 public   static final com.example.admin.proguardsample. Day SATURDAY; 
 
 public   static final com.example.admin.proguardsample. Day SUNDAY; 
 
 public   static com.example.admin.proguardsample.Day [] values ​​() ; 
 
 public   static com.example.admin.proguardsample.Day valueOf (java.lang.String); 
 
 static {}; 
 
 }

• Enumeration actually creates a class that inherits from java.lang.Enum
• The enumeration type in the Java code is finally converted into a static final attribute in the class
• There are two more methods, values() and valueOf().
• The values ​​method returns an array collection of the defined enumeration types, that is, the seven types from MONDAY to SUNDAY.

2. Find the crash track where Day.valueOf(String) calls Enum.valueOf(Class,String) method

 public   static com.example.admin.proguardsample.Day valueOf (java.lang.String); 
 
 Code: 
 
 0: ldc #4 // class com/example/admin/proguardsample/ Day   
 
 2: aload_0 
 
 3: invokestatic #5 // Method java/lang/Enum.valueOf:(Ljava/lang/Class;Ljava/lang/String;)Ljava/lang/Enum; 
 
 6: checkcast #4 // class com/example/admin/proguardsample/ Day   
 
 9: areturn

The valueOf method of Enum will indirectly call the Day.values() method. The specific steps are:

• Enum.value calls the Class.enumConstantDirectory method to get the mapping from String to enumeration
• The Class.enumConstantDirectory method calls Class.getEnumConstantsShared to get the current enumeration type
• The Class.getEnumConstantsShared method uses reflection to call values ​​to get a collection of enumeration types.

After obfuscation, the values ​​are renamed, so a NoSuchMethodException occurs.

Regarding the call trace, if you are interested, you can study the source code yourself, it is not difficult.

It is not recommended to confuse the four major components

We all use the four major components in Android. The reason why these components cannot be confused is:

• The four major component declarations must be registered in the manifest. If the obfuscated class name is changed, but the obfuscated class name is not registered in the manifest, it does not comply with the Android component registration mechanism.
• External programs may use the string class name of the component. If the class name is obfuscated, it may cause an exception.

Annotations cannot be confused

Annotations are increasingly used on the Android platform, with ButterKnife and Otto being the most commonly used. In many scenarios, annotations are used to reflect and determine the characteristics of some elements at runtime.

In order to ensure that annotations work properly, we should not confuse annotations. The default obfuscation configuration of the Android project already includes the following configuration to retain annotations

 -keepattributes *Annotation*

For more information about annotations, read this article. Detailed explanation of annotations in Java

Other things that should not be confused

• Java method called by jni
• Java native methods
• How to call java from js
• It is not recommended to confuse third-party libraries
• Other reflection-related situations

Recovering the stacktrace

Proguard obfuscation brings many benefits, but it also makes the crash stacktrace we collect more difficult to read. Fortunately, there are remedial measures. Here we introduce a tool, retrace, which is used to restore the obfuscated stacktrace to the information before obfuscation.

retrace script

The Android development environment comes with a retrace script by default. The path is usually ./tools/proguard/bin/retrace.sh

mapping table

After Proguard is obfuscated, a mapping table will be generated. The file name is mapping.txt. We can use the find tool to find it under Project.

 find . - name mapping.txt 
 
 ./app/build/outputs/mapping/release/mapping.txt

A crash stacktrace

A raw crash message looks like this.

 E/AndroidRuntime(24006): Caused by : java.lang.NullPointerException: Attempt to invoke virtual method 'int android.graphics.Bitmap.getWidth()'   on a null object reference 
 
 E/AndroidRuntime(24006): at com.example.admin.proguardsample.aa(Utils.java:10) 
 
 E/AndroidRuntime(24006): at com.example.admin.proguardsample.MainActivity.onCreate(MainActivity.java:22) 
 
 E/AndroidRuntime(24006): at android.app.Activity.performCreate(Activity.java:6106) 
 
 E/AndroidRuntime(24006): at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1123) 
 
 E/AndroidRuntime(24006): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2566) 
 
 E/AndroidRuntime(24006): ... 10 more

Process the above information and remove E/AndroidRuntime(24006): These strings retrace can work properly. The resulting string is

 Caused by : java.lang.NullPointerException: Attempt to invoke virtual method 'int android.graphics.Bitmap.getWidth()'   on a null object reference
 at com.example.admin.proguardsample.aa(Utils.java:10)
 at com.example.admin.proguardsample.MainActivity.onCreate(MainActivity.java:22)
 at android.app.Activity.performCreate(Activity.java:6106)
 at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1123)
 at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2566)
 ... 10 more

Save the above stacktrace into a text file, such as npe_stacktrace.txt.

Get started

 ./tools/proguard/bin/retrace.sh /Users/admin/Downloads/ProguardSample/app/build/outputs/mapping/release/mapping.txt /tmp/npe_stacktrace.txt

The resulting readable stacktrace is

 Caused by : java.lang.NullPointerException: Attempt to invoke virtual method 'int android.graphics.Bitmap.getWidth()'   on a null object reference 
 
 at com.example.admin.proguardsample.Utils. int getBitmapWidth(android.graphics.Bitmap)(Utils.java:10) 
 
 at com.example.admin.proguardsample.MainActivity.void onCreate(android.os.Bundle)(MainActivity.java:22) 
 
 at android.app.Activity.performCreate(Activity.java:6106) 
 
 at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1123) 
 
 at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2566) 
 
 ... 10 more

Note: To make stacktrace analysis easier and more efficient, it is recommended to keep the SourceFile and LineNumber properties

 -keepattributes SourceFile,LineNumberTable

These are some of my personal experience summaries about confusion. I hope it can be helpful to everyone.