Home > Articles > Mobile Application Development & Programming

  • Print
  • + Share This
This chapter is from the book

Booting Android from NOR Flash

QEMU doesn’t provide NOR flash emulation on the goldfish platform. To make things simple, we will use RAM to create a boot-up process that is similar to the boot process from NOR flash. This approach builds a binary image that includes U-Boot, the Linux kernel, and the RAMDISK image and passes this image to QEMU through the –kernel option.

Before we start, let’s look at how QEMU boots a Linux kernel. To boot up a Linux kernel, the bootloader prepares the following environment:

  • The processor is in SVC (Supervisor) mode and IRQ and FIQ are disabled.

  • MMU is disabled.

  • Register r0 is set to 0.

  • Register r1 contains the ARM Linux machine type.

  • Register r2 contains the address of the kernel parameter list.

After power-up, QEMU starts to run from address 0x00000000. Before it loads a kernel image, QEMU prepares the environment described previously; it then jumps to address 0x00010000. Figure 10.4 shows a memory dump before the point at which QEMU launches a kernel image. Notice the five lines of assembly code before control is transferred to the kernel image—these lines are hard-coded by QEMU when the system starts. The first line (0x00000000) sets register r0 to 0. The second line (0x00000004) and third line (0x00000008) set register r1 to 0x5a1, which is the machine type of the goldfish platform. The fourth line (0x0000000c) sets the value of register r2 to 0x100, which is the start address of the kernel parameter list. The fifth line (0x00000010) sets the register pc to 0x10000, so the execution jumps to address 0x10000. QEMU assumes the kernel image is loaded at address 0x10000.

Figure 10.4

Figure 10.4 Memory dump of mini-bootloader at reset

As outlined in Figure 10.5, we will create an image including U-Boot, the Linux kernel, and RAMDISK for testing. U-Boot is located at address 0x00010000, which is the address that QEMU will invoke. The Linux kernel is located at address 0x00210000, and the RAMDISK image is located at address 0x00410000. Both the kernel and RAMDISK images are placed at a distance of 2MB starting from address 0x00010000. After U-Boot is relocated, it will move itself to address 0x1ff59000 (this address may change for each build) and free about 2MB from the starting address 0x00010000. We can inform U-Boot about the kernel and RAMDISK image locations through the bootm command, given from the U-Boot command line. Alternatively, you can set the default bootm parameter in include/configs/goldfish.h. We can add the default bootm and kernel parameters in goldfish.h as follows:

#define CONFIG_BOOTARGS "qemu.gles=1 qemu=1 console=ttyS0 android.qemud=ttyS1
androidboot.console=ttyS2 android.checkjni=1 ndns=1"

#define CONFIG_BOOTCOMMAND "bootm 0x210000 0x410000"
Figure 10.5

Figure 10.5 Memory relocation during boot-up

The U-Boot command bootm then copies the kernel image into 0x00010000 and the RAMDISK image into 0x00800000. At that point, U-Boot jumps to address 0x00010000 to start the Linux kernel.

Creating the RAMDISK Image

Besides U-Boot and the kernel image, we need a RAMDISK image to support the boot process. In Android, RAMDISK is used as the root file system. We can customize the boot process by changing the RAMDISK content. Let’s create a RAMDISK image so that we can build the flash image for testing. Given that we are using the Android emulator, we can take advantage of the RAMDISK image from the Android SDK as the base for our image. The RAMDISK image can be found in the system image folder in the Android SDK. For an example, the RAMDISK image for Android 4.0.3 (API 15) can be found at {Android SDK installation path}/system-images/android-15/armeabi-v7a/ramdisk.img.

If we want to modify this image, we can create a folder and extract the image to that folder using the following command:

$ mkdir initrd
$ cd initrd
$ gzip -dc < ../ramdisk.img | cpio --extract

Once we extract the RAMDISK image, we can see its content:

$ ls -F
data/         dev/   init.goldfish.rc*  proc/  sys/     ueventd.goldfish.rc
default.prop  init*  init.rc*           sbin/  system/  ueventd.rc

The RAMDISK includes the folders and startup scripts for the root file system. The actual system files are stored in system.img, and the user data files are stored in userdata.img. Both system.img and userdata.img are emulated as NAND flash. They are mounted as /system and /data folders, respectively, under the root file system.

We can inspect file systems after boot-up as follows:

shell@android:/ $ mount
rootfs / rootfs ro 0 0
tmpfs /dev tmpfs rw,nosuid,mode=755 0 0
devpts /dev/pts devpts rw,mode=600 0 0
proc /proc proc rw 0 0
sysfs /sys sysfs rw 0 0
none /acct cgroup rw,cpuacct 0 0
tmpfs /mnt/secure tmpfs rw,mode=700 0 0
tmpfs /mnt/asec tmpfs rw,mode=755,gid=1000 0 0
tmpfs /mnt/obb tmpfs rw,mode=755,gid=1000 0 0
none /dev/cpuctl cgroup rw,cpu 0 0
/dev/block/mtdblock0 /system yaffs2 ro 0 0
/dev/block/mtdblock1 /data yaffs2 rw,nosuid,nodev 0 0
/dev/block/mtdblock2 /cache yaffs2 rw,nosuid,nodev 0 0
shell@android:/ $

Now we can change the files in this folder as desired. After we’ve made those changes, we can generate the new RAMDISK image using the following commands:

$ find . > ../initrd.list
$ cpio -o -H newc -O ../ramdisk.img < ../initrd.list
$ cd ..
$ gzip ramdisk.img
$ mv ramdisk.img.gz rootfs.img

Creating the Flash Image

Now that all of the image files (U-Boot, Linux kernel, and RAMDISK) are ready, we can start to create the flash image to boot the system.

U-Boot can boot a variety of file types (e.g., ELF, BIN), but these file types have to first be repackaged in the U-Boot image format (i.e., uImage). This format stores information about the operating system type, the load address, the entry point, basic integrity verification (via CRC), compression types, free description text, and so on.

To create a U-Boot image format, we need a utility called mkimage. If this tool is not installed in the host system, it can be installed in Ubuntu using the following command:

$ sudo apt-get install uboot-mkimage

With this utility, we can repackage the kernel image and RAMDISK image in the U-Boot format using the following commands:

$ mkimage -A arm -C none -O linux -T kernel -d zImage -a 0x00010000 -e 0x00010000
$ gzip -c rootfs.img > rootfs.img.gz
$ mkimage -A arm -C none -O linux -T ramdisk -d rootfs.img.gz -a 0x00800000 -e
0x00800000 rootfs.uimg

Once we have uImage files in hand, we can generate a flash image using the dd command as follows:

$ dd if=/dev/zero of= flash.bin bs=1 count=6M
$ dd if=u-boot.bin of= flash.bin conv=notrunc bs=1
$ dd if= zImage.uimg of= flash.bin conv=notrunc bs=1 seek=2M
$ dd if= rootfs.uimg of= flash.bin conv=notrunc bs=1 seek=4M

The file flash.bin includes all three images that we will use to boot up the system.

There are multiple steps to build the Linux kernel and generate all images. Please refer to Appendix A for the detailed procedures. All related Makefiles and scripts can be found in repository build in GitHub.

Booting Up the Flash Image

Finally, we are ready to boot the flash image that we built. Let’s run it in the Android emulator and stop in the U-Boot command-line interface first. In U-Boot, we set a 2-second delay before U-Boot starts autoboot. Before autoboot starts, any keystroke will take us to the U-Boot command prompt. We can use a U-Boot command to verify the kernel and RAMDISK image, thereby making sure they are correct:

$ emulator -verbose -show-kernel -netfast -avd hd2 -qemu -serial stdio -kernel
U-Boot 2013.01.-rc1-00003-g54217a1 (Feb 09 2014 - 23:28:59)

U-Boot code: 00010000 -> 00029B0C  BSS: -> 0002D36C
IRQ Stack: 0badc0de
FIQ Stack: 0badc0de
monitor len: 0001D36C
ramsize: 20000000
TLB table at: 1fff0000
Top of RAM usable for U-Boot at: 1fff0000
Reserving 116k for U-Boot at: 1ffd2000
Reserving 136k for malloc() at: 1ffb0000
Reserving 32 Bytes for Board Info at: 1ffaffe0
Reserving 120 Bytes for Global Data at: 1ffaff68
Reserving 8192 Bytes for IRQ stack at: 1ffadf68
New Stack Pointer is: 1ffadf58
RAM Configuration:
Bank #0: 00000000 512 MiB
relocation Offset is: 1ffc2000
goldfish_init(), gtty.base=ff012000
WARNING: Caches not enabled
monitor flash len: 0001D0D4
Now running in RAM - U-Boot at: 1ffd2000
Using default environment

Destroy Hash Table: 1ffeb724 table = 00000000
Create Hash Table: N=89

INSERT: table 1ffeb724, filled 1/89 rv 1ffb02a4 ==> name="bootargs" value="qemu.
gles=1 qemu=1 console=ttyS0 android.qemud=ttyS1 androidboot.console=ttyS2
android.checkjni=1 ndns=1"
INSERT: table 1ffeb724, filled 2/89 rv 1ffb0160 ==> name="bootcmd" value="bootm
0x210000 0x410000"
INSERT: table 1ffeb724, filled 3/89 rv 1ffb02f8 ==> name="bootdelay" value="2"
INSERT: table 1ffeb724, filled 4/89 rv 1ffb0178 ==> name="baudrate" value="38400"
INSERT: table 1ffeb724, filled 5/89 rv 1ffb0154 ==> name="bootfile" value="/
INSERT: free(data = 1ffb0008)
INSERT: done
In:    serial
Out:   serial
Err:   serial
Net:   SMC91111-0
Warning: SMC91111-0 using MAC address from net device

### main_loop entered: bootdelay=2

### main_loop: bootcmd="bootm 0x210000 0x410000"
Hit any key to stop autoboot:  0
Goldfish # iminfo 0x210000

## Checking Image at 00210000 ...
   Legacy image found
   Image Name:
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1722596 Bytes = 1.6 MiB
   Load Address: 00010000
   Entry Point:  00010000
   Verifying Checksum ... OK
Goldfish # iminfo 0x410000

## Checking Image at 00410000 ...
   Legacy image found
   Image Name:
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    187687 Bytes = 183.3 KiB
   Load Address: 00800000
   Entry Point:  00800000
   Verifying Checksum ... OK
Goldfish #

In the preceding code, notice that we use the iminfo command to check the image at 0x00210000 and 0x00410000. U-Boot recognizes the data at these addresses as the Linux kernel image and Linux RAMDISK image, respectively. Also notice the load address: U-Boot loads the kernel image to address 0x00010000 and the RAMDISK image to address 0x00800000.

We can boot the system using the bootm command as follows:

Goldfish # bootm 0x210000 0x410000
## Current stack ends at 0x1ffadb10 *  kernel: cmdline image address = 0x00210000
## Booting kernel from Legacy Image at 00210000 ...
   Image Name:
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1722596 Bytes = 1.6 MiB
   Load Address: 00010000
   Entry Point:  00010000
   kernel data at 0x00210040, len = 0x001a48e4 (1722596)
*  ramdisk: cmdline image address = 0x00410000
## Loading init Ramdisk from Legacy Image at 00410000 ...
   Image Name:
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    187687 Bytes = 183.3 KiB
   Load Address: 00800000
   Entry Point:  00800000
   ramdisk start = 0x00800000, ramdisk end = 0x0082dd27
   Loading Kernel Image ... OK
CACHE: Misaligned operation at range [00010000, 006a2390]
   kernel loaded at 0x00010000, end = 0x001b48e4
using: ATAGS
## Transferring control to Linux (at address 00010000)...

Starting kernel ...

Uncompressing Linux.............................................................
........................................... done, booting the kernel.
goldfish_fb_get_pixel_format:167: display surface,pixel format:
  bits/pixel:  16
  bytes/pixel: 2
  depth:       16
  red:         bits=5 mask=0xf800 shift=11 max=0x1f
  green:       bits=6 mask=0x7e0 shift=5 max=0x3f
  blue:        bits=5 mask=0x1f shift=0 max=0x1f
  alpha:       bits=0 mask=0x0 shift=0 max=0x0
Initializing cgroup subsys cpu
Linux version 2.6.29-ge3d684d (sgye@sgye-Latitude-E6510) (gcc version 4.6.3
(Sourcery CodeBench Lite 2012.03-57) ) #1 Sun Feb 9 23:32:29 CST 2014
CPU: ARMv7 Processor [410fc080] revision 0 (ARMv7), cr=10c5387f
CPU: VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
Machine: Goldfish
Memory policy: ECC disabled, Data cache writeback
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048
Kernel command line: qemu.gles=1 qemu=1 console=ttyS0 android.qemud=ttyS1
androidboot.console=ttyS2 android.checkjni=1 ndns=1
Unknown boot option 'qemu.gles=1': ignoring
Unknown boot option 'android.qemud=ttyS1': ignoring
Unknown boot option 'androidboot.console=ttyS2': ignoring
Unknown boot option 'android.checkjni=1': ignoring
PID hash table entries: 2048 (order: 11, 8192 bytes)
Console: colour dummy device 80x30
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 512MB = 512MB total
Memory: 515456KB available (2944K code, 707K data, 124K init)
Calibrating delay loop... 370.27 BogoMIPS (lpj=1851392)
Mount-cache hash table entries: 512
Initializing cgroup subsys debug
Initializing cgroup subsys cpuacct
Initializing cgroup subsys freezer
CPU: Testing write buffer coherency: ok
net_namespace: 936 bytes
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
NET: Registered protocol family 2
IP route cache hash table entries: 16384 (order: 4, 65536 bytes)
TCP established hash table entries: 65536 (order: 7, 524288 bytes)
TCP bind hash table entries: 65536 (order: 6, 262144 bytes)
TCP: Hash tables configured (established 65536 bind 65536)
TCP reno registered
NET: Registered protocol family 1
checking if image is initramfs... it is
Freeing initrd memory: 180K
goldfish_new_pdev goldfish_interrupt_controller at ff000000 irq -1
goldfish_new_pdev goldfish_device_bus at ff001000 irq 1
goldfish_new_pdev goldfish_timer at ff003000 irq 3
goldfish_new_pdev goldfish_rtc at ff010000 irq 10
goldfish_new_pdev goldfish_tty at ff002000 irq 4
goldfish_new_pdev goldfish_tty at ff011000 irq 11
goldfish_new_pdev goldfish_tty at ff012000 irq 12
goldfish_new_pdev smc91x at ff013000 irq 13
goldfish_new_pdev goldfish_fb at ff014000 irq 14
goldfish_new_pdev goldfish_audio at ff004000 irq 15
goldfish_new_pdev goldfish_mmc at ff005000 irq 16
goldfish_new_pdev goldfish_memlog at ff006000 irq -1
goldfish_new_pdev goldfish-battery at ff015000 irq 17
goldfish_new_pdev goldfish_events at ff016000 irq 18
goldfish_new_pdev goldfish_nand at ff017000 irq -1
goldfish_new_pdev qemu_pipe at ff018000 irq 19
goldfish_new_pdev goldfish-switch at ff01a000 irq 20
goldfish_new_pdev goldfish-switch at ff01b000 irq 21
goldfish_pdev_worker registered goldfish_interrupt_controller
goldfish_pdev_worker registered goldfish_device_bus
goldfish_pdev_worker registered goldfish_timer
goldfish_pdev_worker registered goldfish_rtc
goldfish_pdev_worker registered goldfish_tty
goldfish_pdev_worker registered goldfish_tty
goldfish_pdev_worker registered goldfish_tty
goldfish_pdev_worker registered smc91x
goldfish_pdev_worker registered goldfish_fb
goldfish_pdev_worker registered goldfish_audio
goldfish_pdev_worker registered goldfish_mmc
goldfish_pdev_worker registered goldfish_memlog
goldfish_pdev_worker registered goldfish-battery
goldfish_pdev_worker registered goldfish_events
goldfish_pdev_worker registered goldfish_nand
goldfish_pdev_worker registered qemu_pipe
goldfish_pdev_worker registered goldfish-switch
goldfish_pdev_worker registered goldfish-switch
ashmem: initialized
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
fuse init (API version 7.11)
yaffs Feb  9 2014 23:30:30 Installing.
msgmni has been set to 1007
alg: No test for stdrng (krng)
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
allocating frame buffer 480 * 800, got ffa00000
console [ttyS0] enabled
brd: module loaded
loop: module loaded
nbd: registered device at major 43
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
smc91x.c: v1.1, sep 22 2004 by Nicolas Pitre <nico@cam.org>
eth0 (smc91x): not using net_device_ops yet
eth0: SMC91C11xFD (rev 1) at e080c000 IRQ 13 [nowait]
eth0: Ethernet addr: 52:54:00:12:34:56
goldfish nand dev0: size c5e0000, page 2048, extra 64, erase 131072
goldfish nand dev1: size c200000, page 2048, extra 64, erase 131072
goldfish nand dev2: size 4000000, page 2048, extra 64, erase 131072
mice: PS/2 mouse device common for all mice
*** events probe ***
events_probe() addr=0xe0814000 irq=18
events_probe() keymap=qwerty2
input: qwerty2 as /devices/virtual/input/input0
goldfish_rtc goldfish_rtc: rtc core: registered goldfish_rtc as rtc0
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.14.0-ioctl (2008-04-23) initialised: dm-devel@redhat.com
logger: created 64K log 'log_main'
logger: created 256K log 'log_events'
logger: created 64K log 'log_radio'
Netfilter messages via NETLINK v0.30.
nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
nf_conntrack.acct=1 kernel parameter, acct=1 nf_conntrack module option or
sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
ctnetlink v0.93: registering with nfnetlink.
NF_TPROXY: Transparent proxy support initialized, version 4.1.0
NF_TPROXY: Copyright (c) 2006-2007 BalaBit IT Ltd.
xt_time: kernel timezone is -0000
ip_tables: (C) 2000-2006 Netfilter Core Team
arp_tables: (C) 2002 David S. Miller
TCP cubic registered
NET: Registered protocol family 10
ip6_tables: (C) 2000-2006 Netfilter Core Team
IPv6 over IPv4 tunneling driver
NET: Registered protocol family 17
NET: Registered protocol family 15
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 0
goldfish_rtc goldfish_rtc: setting system clock to 2014-02-20 08:54:53 UTC
Freeing init memory: 124K
mmc0: new SD card at address e118
mmcblk0: mmc0:e118 SU02G 100 MiB
init: cannot open '/initlogo.rle'
yaffs: dev is 32505856 name is "mtdblock0"
yaffs: passed flags ""
yaffs: Attempting MTD mount on 31.0, "mtdblock0"
yaffs_read_super: isCheckpointed 0
save exit: isCheckpointed 1
yaffs: dev is 32505857 name is "mtdblock1"
yaffs: passed flags ""
yaffs: Attempting MTD mount on 31.1, "mtdblock1"
yaffs_read_super: isCheckpointed 0
yaffs: dev is 32505858 name is "mtdblock2"
yaffs: passed flags ""
yaffs: Attempting MTD mount on 31.2, "mtdblock2"
yaffs_read_super: isCheckpointed 0
init: untracked pid 39 exited
eth0: link up
shell@android:/ $ warning: 'zygote' uses 32-bit capabilities (legacy support in

Source-Level Debugging of the Flash Image

At this point, we can use a flash image that includes both U-Boot and the goldfish kernel to boot up the system. But can we do source-level debugging as well? If we are working on a real hardware board with JTAG debugger, it is quite difficult to do source-level debugging for both U-Boot and the kernel. However, no such problem arises in a virtual environment. With this approach, we can closely observe the transition from bootloader to Linux kernel using source-level debugging. This is a convenient way to debug the U-Boot boot-up process. We can track the interaction between U-Boot and Linux kernel by tracing the execution of the source code.

Let’s start the Android emulator with gdb support:

$ emulator -verbose -show-kernel -netfast -avd hd2 -shell -qemu -s -S -kernel

We connect to the Android emulator using gdb:

$ ddd --debugger arm-none-eabi-gdb u-boot/u-boot

As shown in Figure 10.6, we load U-Boot in gdb with source-level debugging information.

Figure 10.6

Figure 10.6 Loading U-Boot to gdb

Now we can perform source-level debugging for U-Boot. Since U-Boot will reload itself, we must use the same technique that we applied in Chapter 9 to continue the source-level debugging after memory relocation occurs.

Each time we start U-Boot in gdb, we have to go through a series of steps. It is much easier (and faster) to put these steps into a gdb script, as shown in Example 10.1. This script can be found in the folder bin of the repository build.

Example 10.1 GDB Startup Script for U-Boot (u-boot.gdb)

# Debug u-boot
b board_init_f
b relocate_code
p/x ((gd_t *)$r1)->relocaddr
symbol-file ./u-boot/u-boot
add-symbol-file ./u-boot/u-boot 0x1ff59000
b board_init_r

We can load this script in the gdb console using the following command:

(gdb) target remote localhost:1234
(gdb) source bin/u-boot.gdb

After running this script, we can see that U-Boot has stopped at board_init_f() and the U-Boot symbol has been reloaded to the memory address after its relocation, as shown in Figure 10.7.

Figure 10.7

Figure 10.7 Reload the U-Boot symbol after relocation

Let’s continue running U-Boot to a point after memory relocation. In the script u-boot.gdb, the breakpoint is set to board_init_r(). After U-Boot stops at this breakpoint, we can load the goldfish kernel symbol. The multiple steps to load the goldfish kernel can also be put into a gdb script, as shown in Example 10.2. This script can also be found in the folder bin of the repository build.

Example 10.2 GDB Script for Debugging Goldfish Kernel (goldfish.gdb)

# Debug goldfish kernel
symbol-file ./goldfish/vmlinux
add-symbol-file ./goldfish/vmlinux 0x00010000
b start_kernel

We can load the script goldfish.gdb to the gdb console as follows:

(gdb) source bin/goldfish.gdb
add symbol table from file "/home/sgye/src/build/goldfish/vmlinux" at
    .text_addr = 0x10000
Breakpoint 4 at 0xc00086b4: file /home/sgye/src/goldfish/init/main.c, line 535.
(2 locations)
warning: (Internal error: pc 0x10088 in read in psymtab, but not in symtab.)

(gdb) c
warning: (Internal error: pc 0x10088 in read in psymtab, but not in symtab.)

Breakpoint 4, start_kernel () at /home/sgye/src/goldfish/init/main.c:535

In the script goldfish.gdb, the kernel symbol is loaded from vmlinux at memory address 0x10000 and a breakpoint is set at start_kernel(). After loading the kernel symbol, we can continue running U-Boot. Now the system stops at the Linux kernel code, as shown in Figure 10.8.

Figure 10.8

Figure 10.8 The goldfish kernel at start_kernel()

As we can see in this session, we have much more control over the system in the virtual environment compared to what is possible in the real hardware. In turn, we can perform a deeper analysis of the code by tracing the execution path at the source level. We can work at the source level, starting from the first line of code and working all the way to the point at which the operating system fully boots up.

  • + Share This
  • 🔖 Save To Your Account