Divergence or coexistence? A detailed explanation of Android kernel security

1. Knowledge Background

In October 2003, Andy Rubin and several friends founded Android, a smart device operating system company that later influenced many people (acquired by Google in 2005). Nowadays, more and more smart terminals in the world, including mobile phones, TVs, SmartBoxes, IoT, cars, multimedia devices, etc., use the Android system in depth, and the bottom layer of Android is the Linux kernel, which also makes the security of the Linux kernel have a significant impact on Android.

However, because Android wanted to bypass the commercial licensing issue, it developed similar solutions such as replacing Glibc with bionic and replacing Cairo with Skia, which did not use the standard kernel and GNU/Linux. For these reasons, Google's ideas on the issue of Android kernel open source were not very compatible with the Linux kernel community, which led to Android making a lot of targeted modifications to the kernel, but they could not be merged into Upstream. This also led to the Android kernel being somewhat different from the Linux kernel in terms of security, and the focus was also different.

At the operating system level, the Android platform not only provides the security features of the Linux kernel, but also provides a secure inter-process communication (IPC) mechanism for secure communication between applications running in different processes. These operating system-level security features are designed to ensure that even native code is restricted by the application sandbox. Whether the corresponding code is the result of native application behavior or the result of exploiting application vulnerabilities, the system can prevent illegal applications from harming other applications, the Android system, or the device itself.

The following configuration settings are used as the basis for the Android kernel configuration. The settings organize the android-base and android- recommended.cfg files, both of which are located in the android-common kernel repo: https://android.googlesource.com/kernel/common/.

• android-base These options enable core Android functionality and should be enabled on all devices.
• android-recommendedThese options enable advanced Android features and can be selectively enabled on devices.

The upstream Linux kernel 4.8 release specifies a new location for kernel configuration snippets (kernel/configs). For branches based on version 4.8 or higher, Android base and recommended configuration snippets are located in this directory. For branches based on kernels prior to version 4.8, configuration snippets are located in the android/ directory.

2. Generate kernel configuration

For devices with minimal defconfig, you can use the following command to enable the option, generate a .config file, use that file to save the new defconfig or compile a new kernel with Android features enabled:

ARCH=arch scripts/kconfig/merge_config.sh path/device_defconfig android/configs/android-base.cfg android/configs/android-recommended.cfg

Seccomp-BPF and TSYNC

Seccomp-BPF is a kernel security technology that enables the creation of sandboxes to restrict the system calls a process can make. The TSYNC feature enables the use of Seccomp-BPF from multithreaded programs. This capability is limited to architectures that have seccomp support upstream: ARM, ARM64, x86, and x86_64. Kernel 3.10 backports for ARM-32, X86, X86_64, ensure CONFIG_SECCOMP_FILTER=y is enabled in Kconfig (verified as of Android 5.0 CTS), then cherry-pick the following changes from the AOSP kernel/common:android-3.10 repository: 9499cd23f9d05ba159fac6d55dc35a7f49f9ce76…a9ba4285aa5722a3b4d84888e78ba8adc0046b28​

1. cfc7e99e9 arm64: Add _NR* definitions for compat syscalls

(arm64: add _NR* defines for compatibility syscalls) by JP Abgrall

2. bf11863 arm64: Add audit support

(arm64: add audit support) by AKASHI Takahiro

3.3e21c0b arm64: audit: Add audit hook in syscall_trace_enter/exit()

(arm64: audit: add audit hooks in syscall_trace_enter/exit()) by JP Abgrall

4.9499cd2 syscall_get_arch: remove useless function arguments

(syscall_get_arch: remove useless function arguments) by Eric Paris

5．2a30a43 seccomp: create internal mode-setting function

(seccomp: create internal mode-setting functions) by Kees Cook

6. b8a9cff seccomp: extract check/assign mode helpers

(seccomp: extraction check/allocation mode helpers) by Kees Cook

7．8908dde seccomp: split mode setting routines

(seccomp: Split mode setup routines) by Kees Cook

8． e985fd4 seccomp: add “seccomp” syscall

(seccomp: add "seccomp" system call) by Kees Cook

9．9d0ff69 sched: move no_new_privs into new atomic flags

(sched: move no_new_privs to new atomic flags) by Kees Cook

10． b6a12bf seccomp: split filter prep from check and apply

(seccomp: Separating filter preparation from inspection and application processes) by Kees Cook

11．61b6b88 seccomp: introduce writer locking

(seccomp: Introducing the writer lock) by Kees Cook

12. c852ef7 seccomp: allow mode setting across threads

(seccomp: allow cross-thread mode settings) by Kees Cook

13. f14a5db seccomp: implement SECCOMP_FILTER_FLAG_TSYNC

(seccomp: implement SECCOMP_FILTER_FLAG_TSYNC) by Kees Cook

14. 9ac8600 seccomp: Replace BUG(!spin_is_locked()) with assert_spin_lock

(seccomp: replace BUG(!spin_is_locked()) with assert_spin_lock) by Guenter Roeck

15．900e9fd seccomp: fix syscall numbers for x86 and x86_64

(seccomp: fix syscall numbers for x86 and x86_64) by Lee Campbell

16. a9ba428 ARM: add seccomp syscall

(ARM: Add seccomp system calls) by Kees Cook

17．4190090 ARM: 8087/1: ptrace: reload syscall number after secure_computing() check

(ARM: 8087/1: ptrace: reload syscall number after secure_computing() check) by Will Deacon

18. abbfed9 arm64: ptrace: add PTRACE_SET_SYSCALL

(arm64: ptrace: add PTRACE_SET_SYSCALL) by AKASHI Takahiro

19. feb2843 arm64: ptrace: allow tracer to skip a system call

(arm64: ptrace: allow traced process to skip system calls) by AKASHI Takahiro

20． dab1073 asm-generic: add generic seccomp.h for secure computing mode 1

(asm-generic: add generic seccomp.h for secure computing mode 1) by AKASHI Takahiro

21．4f12b53 add seccomp syscall for compat task

(Add seccomp system calls for compatibility tasks) by AKASHI Takahiro

22．7722723 arm64: add SIGSYS siginfo for compat task

(arm64: add SIGSYS siginfo for compatibility tasks) by AKASHI Takahiro

23．210957c arm64: add seccomp support

(arm64: add seccomp support) by AKASHI Takahiro​

4. HWAddressSanitizer

Hardware-assisted AddressSanitizer (HWASan) is a memory error detection tool similar to AddressSanitizer. Compared with ASan, HWASan uses much less memory, making it more suitable for system-wide cleanup. HWASan is only available for Android 10 and higher, and can only be used on AArch64 hardware. Specifically, the following anomalies can be detected:

• Stack and heap buffer overflow/underflow
• Heap usage after release
• Out of scope stack usage
• Double free/error free
• Stack usage after return

HWASan is based on a memory tagging approach where a small random tag value is associated with both a pointer and a memory address range. For memory access to be valid, the pointer and memory tags must match. HWASan relies on the ARMv8 feature Top-Byte-Ignore (TBI, also known as virtual address tagging) to store pointer tags in the highest bit of the address.

HWASan requires the Linux kernel to accept tagged pointers in system call arguments. Support for this requirement is implemented in the following upstream patch sets:

• arm64 tagged address ABI
• arm64: unsign user pointers passed to kernel
• mm: avoid creating virtual address aliases in brk()/mmap()/mremap()
• arm64: Verify tagged address in access_ok() called from kernel thread

The common Android kernel in android-4.14 and higher branches provides these patches as backports, but Android 10-specific branches (such as android-4.14-q) do not provide these patches as backports.

5. KASAN

Android includes the Kernel Address Sanitizer (KASAN). KASAN is a combination of kernel and compile-time modifications that form an instrumentation system that enables simpler bug discovery and root cause analysis. KASAN can detect many types of memory violations in the kernel. It can also detect out-of-bounds reads and writes in the stack, heap, and global variables, and can detect use-after-free and double-free errors. KASAN combines compile-time memory function instrumentation with shadow memory to track memory accesses at runtime, with one-eighth of the kernel memory space dedicated to shadow memory to determine if a memory access is valid. It is currently supported on x86_64 and arm64 architectures. It has been part of the upstream kernel since 4.0 and has been backported to kernels based on Android 3.18. KASAN has been tested on an Android kernel compiled with gcc based on kernel 4.9.2. In addition to KASAN, kcov is another kernel modification that is very useful for testing. kcov is designed to allow coverage-guided fuzzing in the kernel. It measures coverage on system call inputs and is very useful for fuzzing systems such as syzkaller. To compile the kernel with KASAN and kcov enabled, add the following build flags to your kernel build configuration:

CONFIG_KASANCONFIG_KASAN_INLINECONFIG_TEST_KASANCONFIG_KCOVCONFIG_SLUBCONFIG_SLUB_DEBUGCONFIG_CC_OPTIMIZE_FOR_SIZE and remove the following: CONFIG_SLUB_DEBUG_ONCONFIG_SLUB_DEBUG_PANIC_ONCONFIG_KASAN_OUTLINECONFIG_KERNEL_LZ4 Then build and flash the kernel as usual. KASAN kernels are much larger than the original kernel. With this in mind, modify any boot parameters and bootloader settings if applicable. After flashing the kernel, check the kernel boot log to see if KASAN is enabled and running. The kernel will boot and display KASAN's memory map information, for example:

 ...
 [ 0.000000 ] c0 0 Virtual kernel memory layout :
 [ 0.000000 ] c0 0 kasan : 0xffffff8000000000 - 0xffffff9000000000 ( 64 GB )
 [ 0.000000 ] c0 0 vmalloc : 0xffffff9000010000 - 0xffffffbdbfff0000 ( 182 GB )
 [ 0.000000 ] c0 0 vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000 ( 8 GB maximum )
 [ 0.000000 ] c0 0 0xffffffbdc0000000 - 0xffffffbdc3f95400 ( 63 MB actual )
 [ 0.000000 ] c0 0 PCI I / O : 0xffffffbffa000000 - 0xffffffbffb000000 ( 16 MB )
 [ 0.000000 ] c0 0 fixed : 0xffffffbffbdfd000 - 0xffffffbffbdff000 ( 8 KB )
 [ 0.000000 ] c0 0 modules : 0xffffffbffc000000 - 0xffffffc000000000 ( 64 MB )
 [ 0.000000 ] c0 0 memory : 0xffffffc000000000 - 0xffffffc0fe550000 ( 4069 MB )
 [ 0.000000 ] c0 0 .init : 0xffffffc001d33000 - 0xffffffc001dce000 ( 620 KB )
 [ 0.000000 ] c0 0 .text : 0xffffffc000080000 - 0xffffffc001d32284 ( 29385 KB )
 ... 

 The error will look like this: 

 [ 18.539668 ] c3 1 ====================================================================
 [ 18.547662 ] c3 1 BUG : KASAN : null - ptr - deref on address 0000000000000008
 [ 18.554689 ] c3 1 Read of size 8 by task swapper / 0 / 1
 [ 18.559988 ] c3 1 CPU : 3 PID : 1 Comm : swapper / 0 Tainted : GW 3.18 .24 - xxx #1
 [ 18.569275 ] c3 1 Hardware name : Android Device
 [ 18.577433 ] c3 1 Call trace :
 [ 18.580739 ] c3 1 [ < ffffffc00008b32c > ] dump_backtrace + 0x0 / 0x2c4
 [ 18.586985 ] c3 1 [ < ffffffc00008b600 > ] show_stack + 0x10 / 0x1c
 [ 18.592889 ] c3 1 [ < ffffffc001481194 > ] dump_stack + 0x74 / 0xc8
 [ 18.598792 ] c3 1 [ < ffffffc000202ee0 > ] kasan_report + 0x11c / 0x4d0
 [ 18.605038 ] c3 1 [ < ffffffc00020286c > ] __asan_load8 + 0x20 / 0x80
 [ 18.611115 ] c3 1 [ < ffffffc000bdefe8 > ] android_verity_ctr + 0x8cc / 0x1024
 [ 18.617976 ] c3 1 [ < ffffffc000bcaa2c > ] dm_table_add_target + 0x3dc / 0x50c
 [ 18.624832 ] c3 1 [ < ffffffc001bdbe60 > ] dm_run_setup + 0x50c / 0x678
 [ 18.631082 ] c3 1 [ < ffffffc001bda8c0 > ] prepare_namespace + 0x44 / 0x1ac
 [ 18.637676 ] c3 1 [ < ffffffc001bda170 > ] kernel_init_freeable + 0x328 / 0x364
 [ 18.644625 ] c3 1 [ < ffffffc001478e20 > ] kernel_init + 0x10 / 0xd8
 [ 18.650613 ] c3 1 ==================================================================== 

6. Top-byte lgnore

Starting in Android 11, for 64-bit processes, all heap allocations have an implementation-defined tag that is set in the top byte of the pointer on devices with kernel support for ARM Top-byte Ignore (TBI). When the tag is checked during collection, any app that modifies this tag is killed. This is required for future hardware that supports ARM Memory Tagging Extension (MTE). ARM's Top-byte Ignore feature is available for 64-bit code in all Armv8 AArch64 hardware. This feature means that the hardware ignores the top byte of a pointer when accessing memory. TBI requires a compliant kernel in order to correctly handle tagged pointers passed from userspace. The Android common kernel in 4.14 (Pixel 4) and higher has the required TBI patches. Devices that support TBI in the kernel are dynamically detected at process startup and an implementation-dependent tag is inserted in the top byte of the pointer for all heap allocations. Afterwards, the system runs a check to ensure that the tag has not been truncated when the memory is reclaimed. ARM's Memory Tagging Extension (MTE) can help address memory safety issues. MTE works by tagging address bits 56 to 59 of every memory allocation on the stack, heap, and global variables. The hardware and instruction set automatically check that the correct tag is used every time memory is accessed. Android apps that incorrectly store information in the top byte of a pointer will definitely break on devices with MTE enabled. With tagged pointers, incorrect uses of the top byte of a pointer can be more easily detected and rejected before MTE devices are available.

7. Control Flow Integrity (CFI)

As of 2016, approximately 86% of vulnerabilities on Android are related to memory safety. Most vulnerabilities are exploited by attackers who change the normal control flow of an app and gain all the permissions of the exploited app to perform arbitrary malicious activities. Control Flow Integrity (CFI) is a security mechanism that makes it difficult to perform such attacks by not allowing the original control flow graph of a compiled binary to be altered. LLVM's CFI implementation was enabled in the Android 8.1 media stack. CFI was enabled in more components and in the kernel in Android 9. System CFI is enabled by default, but kernel CFI needs to be enabled manually. LLVM's CFI requires compilation with Link-Time Optimization (LTO). LTO preserves the LLVM bitcode representation of object files until link time so that the compiler can better reason about which optimizations can be performed. Enabling LTO reduces the size of the final binary and improves performance, but increases compilation time. When tested on Android, using LTO and CFI together has minimal impact on code size and performance overhead; in a few cases, both are improved. If you want to turn on CFI in a module, you need to add the following lines of code to the makefile (such as /platform/frameworks/av/cmds/stagefright/Android.mk):

 #Specify CFI as a sanitizer during the build process
 LOCAL_SANITIZE : = cfi 

 #Turn on diagnostic mode for CFI. Diagnostic mode will output additional debugging information in logcat during crashes, which is useful when developing and testing builds
 LOCAL_SANITIZE_DIAG : = cfi 

 #Support components to selectively disable CFI instrumentation for individual functions or source code files
 LOCAL_SANITIZE_BLACKLIST : = cfi_blacklist .txt

The kCFI patch is included in all supported Android kernel versions, enabled by the CONFIG_CFI_CLANG option, and is set by default in the GKI. After enabling kCFI, fix any type mismatch errors that its drivers may have. Indirectly calling a function through an incompatible function pointer will cause a CFI failure. When a CFI failure is detected, the kernel outputs a warning that includes the called function and the stack trace that caused the failure. This can be fixed by ensuring that function pointers are always of the same type as the called function. To assist in debugging CFI failures, enable CONFIG_CFI_PERMISSIVE, which will output a warning (without causing the kernel to crash).

8. ShadowCallStack

ShadowCallStack(SCS) is an LLVM instrumentation mode that saves the return address of a function to a separately allocated ShadowCallStack in the function prolog of non-leaf functions, and loads the return address from the ShadowCallStack in the function epilog, thus preventing return address overwrites (such as stack buffer overflows). The return address is also stored on the regular stack for compatibility with unwinders, but is otherwise useless. This ensures that the attack behavior (modifying the return address on the regular stack) does not have any impact on the program control flow. On aarch64, this instrumentation mechanism uses the x18 register to reference the ShadowCallStack, which means that the reference to the ShadowCallStack does not have to be stored in memory. As a result, the implemented runtime avoids exposing the ShadowCallStack address to an attacker who can read arbitrary memory. To enable ShadowCallStack for the kernel, add the following line of code to the kernel configuration file:

 CONFIG_SHADOW_CALL_STACK = y

IX. Conclusion

In addition to the above kernel security features, Android provides some key security features, including:

• User-based permission model
• Process Isolation
• A scalable mechanism for implementing secure IPC

Remove unnecessary and potentially insecure parts of the kernel

We can also see that many of the above functions are implemented based on the LLVM compiler. In real work, LLVM is not only used as a compiler, but also very helpful for optimizing program performance and increasing security detection, including safe stack, CFI, LeakSanitizer, MemorySanitizer and other solutions. With the evolution of Android, the Android kernel has the advantage of integrating the mainline version of the Linux kernel, and then developing a kernel security solution suitable for its own ecosystem. Based on a large number of devices, this is both a challenge and an opportunity. We expect Android to give a perfect answer.

About the Author:

Xu Qingwei: Maintainer of the eBPF Technology Exploration SIG Group of the Longxi Community & Linux Kernel Security Researcher