Android compatibility | NDK toolset update notes

Influenced by other improvements to the Android platform, the dynamic linker in Android M and N has stricter requirements for writing clean and cross-platform compatible native code; native code that meets these requirements can be successfully loaded. To ensure a smooth transition to newer Android versions, your app's native code must follow these rules and recommendations.

Below, we'll reiterate and detail each of the changes related to native code loading, their impact, and what you can do to avoid issues.

Required tools: In the NDK, there is an <arch>-linux-android-readelf binary per architecture (e.g. arm-linux-androideabi-readelf or i686-linux-android-readelf, under toolchains/), but you can use readelf for any architecture since we will only be doing basic checks. On Linux, you will need to install the "binutils" package for readelf and the "pax-utils" package for scanelf.

Private API (implemented from API 24)

Native libraries may only use public APIs and may not link to non-NDK platform libraries. This rule is enforced starting with API 24, after which apps can no longer load non-NDK platform libraries. This rule is enforced by the dynamic linker, so non-public libraries cannot be accessed no matter how the code loads them: System.loadLibrary(...), DT_NEEDED entries, and direct calls to dlopen(...) all fail equally.

Users should have a consistent app experience across updates, and developers should not have to make emergency app updates to accommodate platform changes. Therefore, we recommend against using private C/C++ symbols. The Compatibility Test Suite (CTS), which all Android devices must pass, does not test private symbols. Such symbols may not exist or may behave differently. This may cause apps that use private symbols to break on certain devices or in future releases of the system. Many developers discovered this problem when Android 6.0 Marshmallow switched from OpenSSL to BoringSSL.

To reduce the impact of this transition on users, we've identified a few libraries that are commonly used by the most installed apps on Google Play and that we can still support in the short term (including libandroid_runtime.so, libcutils.so, libcrypto.so, and libssl.so). To give you more time to transition, we're temporarily supporting these libraries; so if you see a warning that your code will break in a future release, fix it now!

 $ readelf --dynamic libBroken.so | grep NEEDED  
 0x00000001 (NEEDED) Shared library: [libnativehelper.so]
 0x00000001 (NEEDED) Shared library: [libutils.so]
 0x00000001 (NEEDED) Shared library: [libstagefright_foundation.so]
 0x00000001 (NEEDED) Shared library: [libmedia_jni.so]
 0x00000001 (NEEDED) Shared library: [liblog.so]
 0x00000001 (NEEDED) Shared library: [libdl.so]
 0x00000001 (NEEDED) Shared library: [libz.so]
 0x00000001 (NEEDED) Shared library: [libstdc++.so]
 0x00000001 (NEEDED) Shared library: [libm.so]
 0x00000001 (NEEDED) Shared library: [libc.so]

Potential issues: Starting from API 24, the dynamic linker will not be able to load private libraries, causing the app to fail to load.

Solution: Rewrite the native code to rely only on public APIs. A short-term solution is to copy the platform library (libcutils.so) without complex dependencies to the project; a long-term solution is to copy the relevant code to the project tree. SSL/Media/JNI internal/binder APIs must not be accessed from native code. Native code should call appropriate public Java API methods when necessary.

All public libraries are listed under platforms/android-API/usr/lib of NDK.

Note: SSL/crypto is a special case and applications must not use the platform libcrypto and libssl libraries directly, even on older platforms. All applications should use the GMS security provider to ensure that applications are protected from known vulnerabilities.

Missing section header (implemented since API 24)

Each ELF file contains additional information in its section headers. Now, these section headers must be present in the file because the dynamic linker uses them for sanity checking. Some developers try to obfuscate their binaries to prevent reverse engineering by removing these section headers. (This doesn't really work because there are tools available to reconstruct the removed information.)

 $ readelf --header libBroken.so | grep 'section headers'  
 Start of   section headers: 0 (bytes into file)
 Size   of   section headers: 0 (bytes)
 Number of   section headers: 0
 $

Solution: Remove the extra step for removing section headers from your build.

Text repositioning (implemented since API 23)

Starting with API 23, shared objects must not contain text relocations. That is, the code must be loaded as is and must not be modified. This approach improves loading time and improves security.

A common cause of text relocations is the use of a hand-written compiler that is not position-independent. This is not a common situation. Please use the scanelf tool as described in our documentation to diagnose further:

 $ scanelf -qT libTextRel.so 
 
 libTextRel.so: (memory/data?) [0x15E0E2] in (optimized out : previous simd_broken_op1) [0x15E0E0] 
 
 libTextRel.so: (memory/data?) [0x15E3B2] in (optimized out : previous simd_broken_op2) [0x15E3B0] 
 
 [skipped the rest]

If you don't have the scanelf tool available, you can use readelf instead to do a basic check, looking for either a TEXTREL entry or a TEXTREL tag. Looking for one or the other is sufficient. (The value corresponding to the TEXTREL entry doesn't matter and is usually 0; its presence indicates that the .so contains text relocations.) In the following example, both indicators are present:

Note: It is technically possible to have a shared object with the TEXTREL entry/tag that does not contain any actual text relocations. This will not happen in the NDK, but if you are generating ELF files yourself, make sure not to generate an ELF file that declares it contains text relocations, as the Android dynamic linker trusts this entry/tag.

Potential problems: Relocations force code pages to be writable and increase the number of dirty pages in memory, which is wasteful. Starting with Android K (API 19), the dynamic linker issues warnings about text relocations, and in API 23 and higher, it refuses to load code containing text relocations.

Solution: Rewrite the compiler to be position-independent to ensure that no text relocations are needed. See the Gentoo documentation for details.

Invalid DT_NEEDED entry (implemented since API 23)

While library dependencies (DT_NEEDED entries in ELF headers) can be absolute paths, this is meaningless on the Android platform because you have no control over where the system will install the library. The DT_NEEDED entry should be the same as the SONAME of the required library, leaving the task of finding the library at runtime to the dynamic linker.

Prior to API 23, Android's dynamic linker ignored the full path when looking for required libraries and only used the base name (the part after the last "/"). Starting with API 23, the runtime linker will fully respect DT_NEEDED, so if the library does not exist in a specific location on the device, the linker will fail to load the corresponding library.

Even worse, some build systems have bugs that cause them to insert DT_NEEDED entries pointing to files on the build host that cannot be found on the device.

 $ readelf --dynamic libSample.so | grep NEEDED  
 0x00000001 (NEEDED) Shared library: [libm.so]
 0x00000001 (NEEDED) Shared library: [libc.so]
 0x00000001 (NEEDED) Shared library: [libdl.so]
 0x00000001 (NEEDED) Shared library:
 [C:\Users\build\Android\ci\jni\libBroken.so]
 $

Potential issues: Prior to API 23 the basename of the DT_NEEDED entry was used, but starting with API 23 the Android runtime will try to load the library using the specified path, which does not exist on the device. Some broken third-party toolchains/build systems use a path on the build host instead of SONAME.

Solution: Make sure all required libraries are referenced by SONAME only. It is better to let the runtime linker find and load these libraries, because the location of the libraries may vary on different devices.

Missing SONAME (used since API 23)

Every ELF shared object ("native library") must have a SONAME (Shared Object Name) attribute. The NDK toolchain adds this attribute by default, and its absence indicates that the alternative toolchain is incorrectly configured or there is a misconfiguration in the build system. Missing SONAME can cause runtime problems, such as loading the wrong library: when this attribute is missing, the filename is used instead.

 $ readelf --dynamic libWithSoName.so | grep SONAME   
 
 0x0000000e (SONAME) Library soname: [libWithSoName.so] 
 
 $

Potential problems: Namespace conflicts can cause the wrong library to be loaded at runtime, leading to crashes if a required symbol is not found or if you try to use an unexpected and ABI-incompatible library.

Solution: The latest NDK generates the correct SONAME by default. Make sure you are using the latest NDK and that your build system is not configured to generate incorrect SONAME entries (using the -soname linker option).

Note that clean cross-platform code built with the latest NDK should work fine on Android N. We recommend that you modify your native code build configuration so that correct binaries are generated.

Android compatibility has always been a concern for many developers. We will continue to pay attention to changes in Android compatibility and publish a series of related articles to help everyone keep up to date. If you find an Android compatibility issue that we have not yet included while using the NDK toolset, please leave a message and we will try our best to find the answer and provide the answer in a new article.