This chapter is from the book
This chapter is from the book
This chapter is from the book
Booting Android from NOR Flash
QEMU doesn’t provide NOR flash emulation on the goldfish platform. To make things simple, we will use RAM to create a boot-up process that is similar to the boot process from NOR flash. This approach builds a binary image that includes U-Boot, the Linux kernel, and the RAMDISK image and passes this image to QEMU through the –kernel option.
Before we start, let’s look at how QEMU boots a Linux kernel. To boot up a Linux kernel, the bootloader prepares the following environment:
The processor is in SVC (Supervisor) mode and IRQ and FIQ are disabled.
MMU is disabled.
Register r0 is set to 0.
Register r1 contains the ARM Linux machine type.
Register r2 contains the address of the kernel parameter list.
After power-up, QEMU starts to run from address 0x00000000. Before it loads a kernel image, QEMU prepares the environment described previously; it then jumps to address 0x00010000. Figure 10.4 shows a memory dump before the point at which QEMU launches a kernel image. Notice the five lines of assembly code before control is transferred to the kernel image—these lines are hard-coded by QEMU when the system starts. The first line (0x00000000) sets register r0 to 0. The second line (0x00000004) and third line (0x00000008) set register r1 to 0x5a1, which is the machine type of the goldfish platform. The fourth line (0x0000000c) sets the value of register r2 to 0x100, which is the start address of the kernel parameter list. The fifth line (0x00000010) sets the register pc to 0x10000, so the execution jumps to address 0x10000. QEMU assumes the kernel image is loaded at address 0x10000.
)
Figure 10.4 Memory dump of mini-bootloader at reset
As outlined in Figure 10.5, we will create an image including U-Boot, the Linux kernel, and RAMDISK for testing. U-Boot is located at address 0x00010000, which is the address that QEMU will invoke. The Linux kernel is located at address 0x00210000, and the RAMDISK image is located at address 0x00410000. Both the kernel and RAMDISK images are placed at a distance of 2MB starting from address 0x00010000. After U-Boot is relocated, it will move itself to address 0x1ff59000 (this address may change for each build) and free about 2MB from the starting address 0x00010000. We can inform U-Boot about the kernel and RAMDISK image locations through the bootm command, given from the U-Boot command line. Alternatively, you can set the default bootm parameter in include/configs/goldfish.h. We can add the default bootm and kernel parameters in goldfish.h as follows:
#define CONFIG_BOOTARGS "qemu.gles=1 qemu=1 console=ttyS0 android.qemud=ttyS1 androidboot.console=ttyS2 android.checkjni=1 ndns=1" #define CONFIG_BOOTCOMMAND "bootm 0x210000 0x410000"
)
Figure 10.5 Memory relocation during boot-up
The U-Boot command bootm then copies the kernel image into 0x00010000 and the RAMDISK image into 0x00800000. At that point, U-Boot jumps to address 0x00010000 to start the Linux kernel.
Creating the RAMDISK Image
Besides U-Boot and the kernel image, we need a RAMDISK image to support the boot process. In Android, RAMDISK is used as the root file system. We can customize the boot process by changing the RAMDISK content. Let’s create a RAMDISK image so that we can build the flash image for testing. Given that we are using the Android emulator, we can take advantage of the RAMDISK image from the Android SDK as the base for our image. The RAMDISK image can be found in the system image folder in the Android SDK. For an example, the RAMDISK image for Android 4.0.3 (API 15) can be found at {Android SDK installation path}/system-images/android-15/armeabi-v7a/ramdisk.img.
If we want to modify this image, we can create a folder and extract the image to that folder using the following command:
$ mkdir initrd $ cd initrd $ gzip -dc < ../ramdisk.img | cpio --extract
Once we extract the RAMDISK image, we can see its content:
$ ls -F data/ dev/ init.goldfish.rc* proc/ sys/ ueventd.goldfish.rc default.prop init* init.rc* sbin/ system/ ueventd.rc
The RAMDISK includes the folders and startup scripts for the root file system. The actual system files are stored in system.img, and the user data files are stored in userdata.img. Both system.img and userdata.img are emulated as NAND flash. They are mounted as /system and /data folders, respectively, under the root file system.
We can inspect file systems after boot-up as follows:
shell@android:/ $ mount rootfs / rootfs ro 0 0 tmpfs /dev tmpfs rw,nosuid,mode=755 0 0 devpts /dev/pts devpts rw,mode=600 0 0 proc /proc proc rw 0 0 sysfs /sys sysfs rw 0 0 none /acct cgroup rw,cpuacct 0 0 tmpfs /mnt/secure tmpfs rw,mode=700 0 0 tmpfs /mnt/asec tmpfs rw,mode=755,gid=1000 0 0 tmpfs /mnt/obb tmpfs rw,mode=755,gid=1000 0 0 none /dev/cpuctl cgroup rw,cpu 0 0 /dev/block/mtdblock0 /system yaffs2 ro 0 0 /dev/block/mtdblock1 /data yaffs2 rw,nosuid,nodev 0 0 /dev/block/mtdblock2 /cache yaffs2 rw,nosuid,nodev 0 0 shell@android:/ $
Now we can change the files in this folder as desired. After we’ve made those changes, we can generate the new RAMDISK image using the following commands:
$ find . > ../initrd.list $ cpio -o -H newc -O ../ramdisk.img < ../initrd.list $ cd .. $ gzip ramdisk.img $ mv ramdisk.img.gz rootfs.img
Creating the Flash Image
Now that all of the image files (U-Boot, Linux kernel, and RAMDISK) are ready, we can start to create the flash image to boot the system.
U-Boot can boot a variety of file types (e.g., ELF, BIN), but these file types have to first be repackaged in the U-Boot image format (i.e., uImage). This format stores information about the operating system type, the load address, the entry point, basic integrity verification (via CRC), compression types, free description text, and so on.
To create a U-Boot image format, we need a utility called mkimage. If this tool is not installed in the host system, it can be installed in Ubuntu using the following command:
$ sudo apt-get install uboot-mkimage
With this utility, we can repackage the kernel image and RAMDISK image in the U-Boot format using the following commands:
$ mkimage -A arm -C none -O linux -T kernel -d zImage -a 0x00010000 -e 0x00010000 zImage.uimg $ gzip -c rootfs.img > rootfs.img.gz $ mkimage -A arm -C none -O linux -T ramdisk -d rootfs.img.gz -a 0x00800000 -e 0x00800000 rootfs.uimg
Once we have uImage files in hand, we can generate a flash image using the dd command as follows:
$ dd if=/dev/zero of= flash.bin bs=1 count=6M $ dd if=u-boot.bin of= flash.bin conv=notrunc bs=1 $ dd if= zImage.uimg of= flash.bin conv=notrunc bs=1 seek=2M $ dd if= rootfs.uimg of= flash.bin conv=notrunc bs=1 seek=4M
The file flash.bin includes all three images that we will use to boot up the system.
There are multiple steps to build the Linux kernel and generate all images. Please refer to Appendix A for the detailed procedures. All related Makefiles and scripts can be found in repository build in GitHub.
Booting Up the Flash Image
Finally, we are ready to boot the flash image that we built. Let’s run it in the Android emulator and stop in the U-Boot command-line interface first. In U-Boot, we set a 2-second delay before U-Boot starts autoboot. Before autoboot starts, any keystroke will take us to the U-Boot command prompt. We can use a U-Boot command to verify the kernel and RAMDISK image, thereby making sure they are correct:
$ emulator -verbose -show-kernel -netfast -avd hd2 -qemu -serial stdio -kernel flash.bin ... U-Boot 2013.01.-rc1-00003-g54217a1 (Feb 09 2014 - 23:28:59) U-Boot code: 00010000 -> 00029B0C BSS: -> 0002D36C IRQ Stack: 0badc0de FIQ Stack: 0badc0de monitor len: 0001D36C ramsize: 20000000 TLB table at: 1fff0000 Top of RAM usable for U-Boot at: 1fff0000 Reserving 116k for U-Boot at: 1ffd2000 Reserving 136k for malloc() at: 1ffb0000 Reserving 32 Bytes for Board Info at: 1ffaffe0 Reserving 120 Bytes for Global Data at: 1ffaff68 Reserving 8192 Bytes for IRQ stack at: 1ffadf68 New Stack Pointer is: 1ffadf58 RAM Configuration: Bank #0: 00000000 512 MiB relocation Offset is: 1ffc2000 goldfish_init(), gtty.base=ff012000 WARNING: Caches not enabled monitor flash len: 0001D0D4 Now running in RAM - U-Boot at: 1ffd2000 Using default environment Destroy Hash Table: 1ffeb724 table = 00000000 Create Hash Table: N=89 INSERT: table 1ffeb724, filled 1/89 rv 1ffb02a4 ==> name="bootargs" value="qemu. gles=1 qemu=1 console=ttyS0 android.qemud=ttyS1 androidboot.console=ttyS2 android.checkjni=1 ndns=1" INSERT: table 1ffeb724, filled 2/89 rv 1ffb0160 ==> name="bootcmd" value="bootm 0x210000 0x410000" INSERT: table 1ffeb724, filled 3/89 rv 1ffb02f8 ==> name="bootdelay" value="2" INSERT: table 1ffeb724, filled 4/89 rv 1ffb0178 ==> name="baudrate" value="38400" INSERT: table 1ffeb724, filled 5/89 rv 1ffb0154 ==> name="bootfile" value="/ tftpboot/uImage" INSERT: free(data = 1ffb0008) INSERT: done In: serial Out: serial Err: serial Net: SMC91111-0 Warning: SMC91111-0 using MAC address from net device ### main_loop entered: bootdelay=2 ### main_loop: bootcmd="bootm 0x210000 0x410000" Hit any key to stop autoboot: 0 Goldfish # iminfo 0x210000 ## Checking Image at 00210000 ... Legacy image found Image Name: Image Type: ARM Linux Kernel Image (uncompressed) Data Size: 1722596 Bytes = 1.6 MiB Load Address: 00010000 Entry Point: 00010000 Verifying Checksum ... OK Goldfish # iminfo 0x410000 ## Checking Image at 00410000 ... Legacy image found Image Name: Image Type: ARM Linux RAMDisk Image (uncompressed) Data Size: 187687 Bytes = 183.3 KiB Load Address: 00800000 Entry Point: 00800000 Verifying Checksum ... OK Goldfish #
In the preceding code, notice that we use the iminfo command to check the image at 0x00210000 and 0x00410000. U-Boot recognizes the data at these addresses as the Linux kernel image and Linux RAMDISK image, respectively. Also notice the load address: U-Boot loads the kernel image to address 0x00010000 and the RAMDISK image to address 0x00800000.
We can boot the system using the bootm command as follows:
Goldfish # bootm 0x210000 0x410000 ## Current stack ends at 0x1ffadb10 * kernel: cmdline image address = 0x00210000 ## Booting kernel from Legacy Image at 00210000 ... Image Name: Image Type: ARM Linux Kernel Image (uncompressed) Data Size: 1722596 Bytes = 1.6 MiB Load Address: 00010000 Entry Point: 00010000 kernel data at 0x00210040, len = 0x001a48e4 (1722596) * ramdisk: cmdline image address = 0x00410000 ## Loading init Ramdisk from Legacy Image at 00410000 ... Image Name: Image Type: ARM Linux RAMDisk Image (uncompressed) Data Size: 187687 Bytes = 183.3 KiB Load Address: 00800000 Entry Point: 00800000 ramdisk start = 0x00800000, ramdisk end = 0x0082dd27 Loading Kernel Image ... OK CACHE: Misaligned operation at range [00010000, 006a2390] OK kernel loaded at 0x00010000, end = 0x001b48e4 using: ATAGS ## Transferring control to Linux (at address 00010000)... Starting kernel ... Uncompressing Linux............................................................. ........................................... done, booting the kernel. goldfish_fb_get_pixel_format:167: display surface,pixel format: bits/pixel: 16 bytes/pixel: 2 depth: 16 red: bits=5 mask=0xf800 shift=11 max=0x1f green: bits=6 mask=0x7e0 shift=5 max=0x3f blue: bits=5 mask=0x1f shift=0 max=0x1f alpha: bits=0 mask=0x0 shift=0 max=0x0 Initializing cgroup subsys cpu Linux version 2.6.29-ge3d684d (sgye@sgye-Latitude-E6510) (gcc version 4.6.3 (Sourcery CodeBench Lite 2012.03-57) ) #1 Sun Feb 9 23:32:29 CST 2014 CPU: ARMv7 Processor [410fc080] revision 0 (ARMv7), cr=10c5387f CPU: VIPT nonaliasing data cache, VIPT nonaliasing instruction cache Machine: Goldfish Memory policy: ECC disabled, Data cache writeback Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048 Kernel command line: qemu.gles=1 qemu=1 console=ttyS0 android.qemud=ttyS1 androidboot.console=ttyS2 android.checkjni=1 ndns=1 Unknown boot option 'qemu.gles=1': ignoring Unknown boot option 'android.qemud=ttyS1': ignoring Unknown boot option 'androidboot.console=ttyS2': ignoring Unknown boot option 'android.checkjni=1': ignoring PID hash table entries: 2048 (order: 11, 8192 bytes) Console: colour dummy device 80x30 Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) Memory: 512MB = 512MB total Memory: 515456KB available (2944K code, 707K data, 124K init) Calibrating delay loop... 370.27 BogoMIPS (lpj=1851392) Mount-cache hash table entries: 512 Initializing cgroup subsys debug Initializing cgroup subsys cpuacct Initializing cgroup subsys freezer CPU: Testing write buffer coherency: ok net_namespace: 936 bytes NET: Registered protocol family 16 bio: create slab <bio-0> at 0 NET: Registered protocol family 2 IP route cache hash table entries: 16384 (order: 4, 65536 bytes) TCP established hash table entries: 65536 (order: 7, 524288 bytes) TCP bind hash table entries: 65536 (order: 6, 262144 bytes) TCP: Hash tables configured (established 65536 bind 65536) TCP reno registered NET: Registered protocol family 1 checking if image is initramfs... it is Freeing initrd memory: 180K goldfish_new_pdev goldfish_interrupt_controller at ff000000 irq -1 goldfish_new_pdev goldfish_device_bus at ff001000 irq 1 goldfish_new_pdev goldfish_timer at ff003000 irq 3 goldfish_new_pdev goldfish_rtc at ff010000 irq 10 goldfish_new_pdev goldfish_tty at ff002000 irq 4 goldfish_new_pdev goldfish_tty at ff011000 irq 11 goldfish_new_pdev goldfish_tty at ff012000 irq 12 goldfish_new_pdev smc91x at ff013000 irq 13 goldfish_new_pdev goldfish_fb at ff014000 irq 14 goldfish_new_pdev goldfish_audio at ff004000 irq 15 goldfish_new_pdev goldfish_mmc at ff005000 irq 16 goldfish_new_pdev goldfish_memlog at ff006000 irq -1 goldfish_new_pdev goldfish-battery at ff015000 irq 17 goldfish_new_pdev goldfish_events at ff016000 irq 18 goldfish_new_pdev goldfish_nand at ff017000 irq -1 goldfish_new_pdev qemu_pipe at ff018000 irq 19 goldfish_new_pdev goldfish-switch at ff01a000 irq 20 goldfish_new_pdev goldfish-switch at ff01b000 irq 21 goldfish_pdev_worker registered goldfish_interrupt_controller goldfish_pdev_worker registered goldfish_device_bus goldfish_pdev_worker registered goldfish_timer goldfish_pdev_worker registered goldfish_rtc goldfish_pdev_worker registered goldfish_tty goldfish_pdev_worker registered goldfish_tty goldfish_pdev_worker registered goldfish_tty goldfish_pdev_worker registered smc91x goldfish_pdev_worker registered goldfish_fb goldfish_pdev_worker registered goldfish_audio goldfish_pdev_worker registered goldfish_mmc goldfish_pdev_worker registered goldfish_memlog goldfish_pdev_worker registered goldfish-battery goldfish_pdev_worker registered goldfish_events goldfish_pdev_worker registered goldfish_nand goldfish_pdev_worker registered qemu_pipe goldfish_pdev_worker registered goldfish-switch goldfish_pdev_worker registered goldfish-switch ashmem: initialized Installing knfsd (copyright (C) 1996 okir@monad.swb.de). fuse init (API version 7.11) yaffs Feb 9 2014 23:30:30 Installing. msgmni has been set to 1007 alg: No test for stdrng (krng) io scheduler noop registered io scheduler anticipatory registered (default) io scheduler deadline registered io scheduler cfq registered allocating frame buffer 480 * 800, got ffa00000 console [ttyS0] enabled brd: module loaded loop: module loaded nbd: registered device at major 43 goldfish_audio_probe tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com> smc91x.c: v1.1, sep 22 2004 by Nicolas Pitre <nico@cam.org> eth0 (smc91x): not using net_device_ops yet eth0: SMC91C11xFD (rev 1) at e080c000 IRQ 13 [nowait] eth0: Ethernet addr: 52:54:00:12:34:56 goldfish nand dev0: size c5e0000, page 2048, extra 64, erase 131072 goldfish nand dev1: size c200000, page 2048, extra 64, erase 131072 goldfish nand dev2: size 4000000, page 2048, extra 64, erase 131072 mice: PS/2 mouse device common for all mice *** events probe *** events_probe() addr=0xe0814000 irq=18 events_probe() keymap=qwerty2 input: qwerty2 as /devices/virtual/input/input0 goldfish_rtc goldfish_rtc: rtc core: registered goldfish_rtc as rtc0 device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.14.0-ioctl (2008-04-23) initialised: dm-devel@redhat.com logger: created 64K log 'log_main' logger: created 256K log 'log_events' logger: created 64K log 'log_radio' Netfilter messages via NETLINK v0.30. nf_conntrack version 0.5.0 (8192 buckets, 32768 max) CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use nf_conntrack.acct=1 kernel parameter, acct=1 nf_conntrack module option or sysctl net.netfilter.nf_conntrack_acct=1 to enable it. ctnetlink v0.93: registering with nfnetlink. NF_TPROXY: Transparent proxy support initialized, version 4.1.0 NF_TPROXY: Copyright (c) 2006-2007 BalaBit IT Ltd. xt_time: kernel timezone is -0000 ip_tables: (C) 2000-2006 Netfilter Core Team arp_tables: (C) 2002 David S. Miller TCP cubic registered NET: Registered protocol family 10 ip6_tables: (C) 2000-2006 Netfilter Core Team IPv6 over IPv4 tunneling driver NET: Registered protocol family 17 NET: Registered protocol family 15 RPC: Registered udp transport module. RPC: Registered tcp transport module. 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com> All bugs added by David S. Miller <davem@redhat.com> VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 0 goldfish_rtc goldfish_rtc: setting system clock to 2014-02-20 08:54:53 UTC (1392886493) Freeing init memory: 124K mmc0: new SD card at address e118 mmcblk0: mmc0:e118 SU02G 100 MiB mmcblk0: init: cannot open '/initlogo.rle' yaffs: dev is 32505856 name is "mtdblock0" yaffs: passed flags "" yaffs: Attempting MTD mount on 31.0, "mtdblock0" yaffs_read_super: isCheckpointed 0 save exit: isCheckpointed 1 yaffs: dev is 32505857 name is "mtdblock1" yaffs: passed flags "" yaffs: Attempting MTD mount on 31.1, "mtdblock1" yaffs_read_super: isCheckpointed 0 yaffs: dev is 32505858 name is "mtdblock2" yaffs: passed flags "" yaffs: Attempting MTD mount on 31.2, "mtdblock2" yaffs_read_super: isCheckpointed 0 init: untracked pid 39 exited eth0: link up shell@android:/ $ warning: 'zygote' uses 32-bit capabilities (legacy support in use)
Source-Level Debugging of the Flash Image
At this point, we can use a flash image that includes both U-Boot and the goldfish kernel to boot up the system. But can we do source-level debugging as well? If we are working on a real hardware board with JTAG debugger, it is quite difficult to do source-level debugging for both U-Boot and the kernel. However, no such problem arises in a virtual environment. With this approach, we can closely observe the transition from bootloader to Linux kernel using source-level debugging. This is a convenient way to debug the U-Boot boot-up process. We can track the interaction between U-Boot and Linux kernel by tracing the execution of the source code.
Let’s start the Android emulator with gdb support:
$ emulator -verbose -show-kernel -netfast -avd hd2 -shell -qemu -s -S -kernel flash.bin
We connect to the Android emulator using gdb:
$ ddd --debugger arm-none-eabi-gdb u-boot/u-boot
As shown in Figure 10.6, we load U-Boot in gdb with source-level debugging information.
)
Figure 10.6 Loading U-Boot to gdb
Now we can perform source-level debugging for U-Boot. Since U-Boot will reload itself, we must use the same technique that we applied in Chapter 9 to continue the source-level debugging after memory relocation occurs.
Each time we start U-Boot in gdb, we have to go through a series of steps. It is much easier (and faster) to put these steps into a gdb script, as shown in Example 10.1. This script can be found in the folder bin of the repository build.
Example 10.1 GDB Startup Script for U-Boot (u-boot.gdb)
# Debug u-boot b board_init_f c b relocate_code c p/x ((gd_t *)$r1)->relocaddr d symbol-file ./u-boot/u-boot add-symbol-file ./u-boot/u-boot 0x1ff59000 b board_init_r
We can load this script in the gdb console using the following command:
(gdb) target remote localhost:1234 (gdb) source bin/u-boot.gdb
After running this script, we can see that U-Boot has stopped at board_init_f() and the U-Boot symbol has been reloaded to the memory address after its relocation, as shown in Figure 10.7.
)
Figure 10.7 Reload the U-Boot symbol after relocation
Let’s continue running U-Boot to a point after memory relocation. In the script u-boot.gdb, the breakpoint is set to board_init_r(). After U-Boot stops at this breakpoint, we can load the goldfish kernel symbol. The multiple steps to load the goldfish kernel can also be put into a gdb script, as shown in Example 10.2. This script can also be found in the folder bin of the repository build.
Example 10.2 GDB Script for Debugging Goldfish Kernel (goldfish.gdb)
# Debug goldfish kernel d symbol-file ./goldfish/vmlinux add-symbol-file ./goldfish/vmlinux 0x00010000 b start_kernel
We can load the script goldfish.gdb to the gdb console as follows:
(gdb) source bin/goldfish.gdb
add symbol table from file "/home/sgye/src/build/goldfish/vmlinux" at
.text_addr = 0x10000
Breakpoint 4 at 0xc00086b4: file /home/sgye/src/goldfish/init/main.c, line 535.
(2 locations)
...
warning: (Internal error: pc 0x10088 in read in psymtab, but not in symtab.)
(gdb) c
warning: (Internal error: pc 0x10088 in read in psymtab, but not in symtab.)
Breakpoint 4, start_kernel () at /home/sgye/src/goldfish/init/main.c:535
(gdb)
In the script goldfish.gdb, the kernel symbol is loaded from vmlinux at memory address 0x10000 and a breakpoint is set at start_kernel(). After loading the kernel symbol, we can continue running U-Boot. Now the system stops at the Linux kernel code, as shown in Figure 10.8.
)
Figure 10.8 The goldfish kernel at start_kernel()
As we can see in this session, we have much more control over the system in the virtual environment compared to what is possible in the real hardware. In turn, we can perform a deeper analysis of the code by tracing the execution path at the source level. We can work at the source level, starting from the first line of code and working all the way to the point at which the operating system fully boots up.