Getting started with memory layouts
So in previous blog post on solving CTF challenge created by me on real alike firmware, I did not touched subject of u-boot bootloader in depth and how such firmware was created in first place. Welcome back here again today you will see steps to create your own firmware for qemu and making it work with u-boot. Many of the stuff you might have encoutered during computer science class and you will be amazed how the concepts of fundamental system architecture and operating system works in action.
First thing you should know is how a typical linux os boots up. Today we will work with emulated ARM Cortex A9 board. Let’s have look at memory mapping for Cortex-A9 CPU.
| Address range | Size | Description |
|---|---|---|
| 0xE000_0000-0xFFFF_FFFF | 512MB | External AXI between daughterboards |
| 0xA000_0000-0xDFFF_FFFF | 1GB | Daughterboard, private |
| 0x8000_0000-0x9FFF_FFFF | 512MB | Local DDR2 |
| 0x8000_0000-0x81FF_FFFF | 64MB | Remappable memory location |
| 0x6000_0000-0x7FFF_FFFF | 512MB | Local DDR2 lower |
| 0x5C00-0000-0x5FFF_FFFF | 64MB | Reserved |
| 0x4000_0000-0x5BFF_FFFF | 448MB | Motherboard peripherals, typically, memory devices |
| 0x2000_0000-0x3FFF_FFFF | 512MB | Reserved |
| 0x1002_0000-0x1FFF_FFFF | ~256MB | Daughterboard, private |
| 0x1000_0000-0x1001_FFFF | 128KB | Motherboard peripherals, CS7 |
| 0x0000_0000-0x0FFF_FFFF | 64MB | Remappable memory section |
As you can see above RAM starts at 0x60000000 ends at 0x7FFFFFFF, Cortex-A9 supports upto 1GB RAM so basically for board will have two 512MB RAM banks all data above 512MB+ will be stored at 0x80000000-0x9FFFFFFF memory range. You can verify this in QEMU source code as well.
static void a9_daughterboard_init(const VexpressMachineState *vms,
ram_addr_t ram_size,
const char *cpu_type,
qemu_irq *pic)
{
MachineState *machine: MACHINE(vms);
MemoryRegion *sysmem: get_system_memory();
MemoryRegion *lowram: g_new(MemoryRegion, 1);
ram_addr_t low_ram_size;
if (ram_size > 0x40000000) {
/* 1GB is the maximum the address space permits */
error_report("vexpress-a9: cannot model more than 1GB RAM");
exit(1);
}
low_ram_size: ram_size;
if (low_ram_size > 0x4000000) {
low_ram_size: 0x4000000;
}
/* RAM is from 0x60000000 upwards. The bottom 64MB of the
* address space should in theory be remappable to various
* things including ROM or RAM; we always map the RAM there.
*/
memory_region_init_alias(lowram, NULL, "vexpress.lowmem", machine->ram,
0, low_ram_size);
memory_region_add_subregion(sysmem, 0x0, lowram);
memory_region_add_subregion(sysmem, 0x60000000, machine->ram); //RAM start address
/* 0x1e000000 A9MPCore (SCU) private memory region */
init_cpus(machine, cpu_type, TYPE_A9MPCORE_PRIV, 0x1e000000, pic,
vms->secure, vms->virt);
/* Daughterboard peripherals : 0x10020000 .. 0x20000000 */
/* 0x10020000 PL111 CLCD (daughterboard) */
sysbus_create_simple("pl111", 0x10020000, pic[44]);
/* 0x10060000 AXI RAM */
/* 0x100e0000 PL341 Dynamic Memory Controller */
/* 0x100e1000 PL354 Static Memory Controller */
/* 0x100e2000 System Configuration Controller */
sysbus_create_simple("sp804", 0x100e4000, pic[48]);
/* 0x100e5000 SP805 Watchdog module */
/* 0x100e6000 BP147 TrustZone Protection Controller */
/* 0x100e9000 PL301 'Fast' AXI matrix */
/* 0x100ea000 PL301 'Slow' AXI matrix */
/* 0x100ec000 TrustZone Address Space Controller */
/* 0x10200000 CoreSight debug APB */
/* 0x1e00a000 PL310 L2 Cache Controller */
sysbus_create_varargs("l2x0", 0x1e00a000, NULL);
}
https://github.com/qemu/qemu/blob/master/hw/arm/vexpress.c
One more thing to know is address of flash memory from which during boot process we will load things into RAM such as kernel, filesystem/ramfs and device tree blob ((dtb). In qemu Cortex-A9 board will have two Parallel NOR Flash of 64MB. So while making firmware maximum filesize we can use for firmware binary would be 128MB and pass to -pflash option when starting virtual machine. You can check out datasheets of these kind parallel NOR flash chips out in market https://www.micron.com/-/media/client/global/documents/products/data-sheet/nor-flash/parallel/m29ew/m29ew_32mb_128mb.pdf as real example. Enough jumping around lets again peek into Qemu source code.
static hwaddr motherboard_legacy_map[]: {
[VE_NORFLASHALIAS]: 0,
/* CS7: 0x10000000 .. 0x10020000 */
[VE_SYSREGS]: 0x10000000,
[VE_SP810]: 0x10001000,
[VE_SERIALPCI]: 0x10002000,
[VE_PL041]: 0x10004000,
[VE_MMCI]: 0x10005000,
[VE_KMI0]: 0x10006000,
[VE_KMI1]: 0x10007000,
[VE_UART0]: 0x10009000,
[VE_UART1]: 0x1000a000,
[VE_UART2]: 0x1000b000,
[VE_UART3]: 0x1000c000,
[VE_WDT]: 0x1000f000,
[VE_TIMER01]: 0x10011000,
[VE_TIMER23]: 0x10012000,
[VE_VIRTIO]: 0x10013000,
[VE_SERIALDVI]: 0x10016000,
[VE_RTC]: 0x10017000,
[VE_COMPACTFLASH]: 0x1001a000,
[VE_CLCD]: 0x1001f000,
/* CS0: 0x40000000 .. 0x44000000 */
[VE_NORFLASH0]: 0x40000000, //First 64MB parallel NOR flash
/* CS1: 0x44000000 .. 0x48000000 */
[VE_NORFLASH1]: 0x44000000, //Second 64MB parallel NOR flash
/* CS2: 0x48000000 .. 0x4a000000 */
[VE_SRAM]: 0x48000000,
/* CS3: 0x4c000000 .. 0x50000000 */
[VE_VIDEORAM]: 0x4c000000,
[VE_ETHERNET]: 0x4e000000,
[VE_USB]: 0x4f000000,
};
Putting these all information togather we get following ranges for RAM and flash memory
| Memory Address Range | Size | Description |
|---|---|---|
| 0x60000000-0x7FFFFFFF | 512MB | RAM <= 512MB |
| 0x80000000-0x9FFFFFFF | 512MB | RAM > 512MB |
| 0x40000000-0x44000000 | 64MB | NOR FLASH 1 |
| 0x48000000-0x4a000000 | 64MB | NOR FLASH 2 |
In this tutorial we will be only using 512MB RAM and NOR FLASH 1. Now lets see the build process
kernel, file system and u-boot compilation
Whenever I meet new people in infosec most of us are always seem to be affraid of compiling stuff from scratch but trust me with enough of practice you will develop intuition and thanks to age of automation it’s more fun to do now these days.
Kernel compilation steps
Download latest stable kernel from https://www.kernel.org/
wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.12.tar.xz
Now you need to install ARM cross compiler since target CPU we are dealing with is ARM
sudo apt update && sudo apt install gcc-arm-none-eabi -y
Generate intial configuration file
make ARCH=arm CROSS_COMPILE=arm-none-eabi- vexpress_defconfig
You can edit options with menuconfig, this step is optional like if you know what you are doing and which functionalities inside kernel you want to enable. Most of the time for research purposes people like to enable kernel debugging related ones. But for this post u can directly compile and boot into basic system without doing anything extra and skip command below.
make ARCH=arm CROSS_COMPILE=arm-none-eabi- menuconfig
menuconfig
After configuring desired options you can start compilation
make -j$(nproc) ARCH=arm CROSS_COMPILE=arm-none-eabi-
kernel compilation in progress
Grab cup of coffee because it’s gonna take some few 5-10min (depending on your system specs)
Once finished kernel zImage file should be at /arch/arm/boot in present working directory. Now copy compiled zImage and vexpress-v2p-ca9.dtb (Device Tree Blob) to your desired location.
cp ./arch/arm/boot/zImage ~/my_firmware && cp ./arch/arm/boot/dts/vexpress-v2p-ca9.dtb ~/my_firmware
Test them with qemu
qemu-system-arm -M vexpress-a9 -m 512M -kernel zImage -dtb vexpress-v2p-ca9.dtb -nographic
Output from successful boot
Booting Linux on physical CPU 0x0Linux version 5.6.12cjhackerz-arm-kernel (cjhackerz@pwn-machine) (gcc version 9.3.0 (Arch Repository)) #1 SMP Tue May 12 04:33:08 IST 2020
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: V2P-CA9
Memory policy: Data cache writeback
Reserved memory: created DMA memory pool at 0x4c000000, size 8 MiB
OF: reserved mem: initialized node vram@4c000000, compatible id shared-dma-pool
cma: Reserved 16 MiB at 0x7f000000
CPU: All CPU(s) started in SVC mode.
percpu: Embedded 19 pages/cpu s45708 r8192 d23924 u77824
Built 1 zonelists, mobility grouping on. Total pages: 130048
Kernel command line: console=ttyAMA0
printk: log_buf_len individual max cpu contribution: 4096 bytes
printk: log_buf_len total cpu_extra contributions: 12288 bytes
printk: log_buf_len min size: 16384 bytes
printk: log_buf_len: 32768 bytes
printk: early log buf free: 14864(90%)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 491996K/524288K available (7168K kernel code, 511K rwdata, 1732K rodata, 1024K init, 177K bss, 15908K reserved, 16384K cma-reserved)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
rcu: Hierarchical RCU implementation.
rcu: RCU event tracing is enabled.
rcu: RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
GIC CPU mask not found - kernel will fail to boot.
GIC CPU mask not found - kernel will fail to boot.
L2C: platform modifies aux control register: 0x02020000 -> 0x02420000
L2C: DT/platform modifies aux control register: 0x02020000 -> 0x02420000
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 dynamic clock gating disabled, standby mode disabled
L2C-310 cache controller enabled, 8 ways, 128 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x46420001
random: get_random_bytes called from start_kernel+0x2fc/0x498 with crng_init=0
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
clocksource: arm,sp804: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
Failed to initialize '/smb@4000000/motherboard/iofpga@7,00000000/timer@12000': -22
smp_twd: clock not found -2
Console: colour dummy device 80x30
Calibrating local timer... 92.29MHz.
Calibrating delay loop... 580.40 BogoMIPS (lpj=2902016)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
CPU: Testing write buffer coherency: ok
CPU0: Spectre v2: using BPIALL workaround
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x60100000 - 0x60100060
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated (580.40 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 0
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
cpuidle: using governor ladder
hw-breakpoint: debug architecture 0x4 unsupported.
Serial: AMBA PL011 UART driver
10009000.uart: ttyAMA0 at MMIO 0x10009000 (irq: 29, base_baud: 0) is a PL011 rev1
printk: console [ttyAMA0] enabled
1000a000.uart: ttyAMA1 at MMIO 0x1000a000 (irq: 30, base_baud: 0) is a PL011 rev1
1000b000.uart: ttyAMA2 at MMIO 0x1000b000 (irq: 31, base_baud: 0) is a PL011 rev1
1000c000.uart: ttyAMA3 at MMIO 0x1000c000 (irq: 32, base_baud: 0) is a PL011 rev1
OF: amba_device_add() failed (-19) for /smb@4000000/motherboard/iofpga@7,00000000/wdt@f000
OF: amba_device_add() failed (-19) for /memory-controller@100e0000
OF: amba_device_add() failed (-19) for /memory-controller@100e1000
OF: amba_device_add() failed (-19) for /watchdog@100e5000
irq: type mismatch, failed to map hwirq-75 for interrupt-controller@1e001000!
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource arm,sp804
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
TCP established hash table entries: 4096 (order: 2, 16384 bytes, linear)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes, linear)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 5 counters available
workingset: timestamp_bits=30 max_order=17 bucket_order=0
squashfs: version 4.0 (2009/01/31) Phillip Lougher
jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
9p: Installing v9fs 9p2000 file system support
jitterentropy: Initialization failed with host not compliant with requirements: 2
io scheduler mq-deadline registered
io scheduler kyber registered
i2c i2c-0: Added multiplexed i2c bus 2
drm-clcd-pl111 1001f000.clcd: assigned reserved memory node vram@4c000000
drm-clcd-pl111 1001f000.clcd: using device-specific reserved memory
drm-clcd-pl111 1001f000.clcd: initializing Versatile Express PL111
drm-clcd-pl111 1001f000.clcd: core tile graphics present
drm-clcd-pl111 1001f000.clcd: this device will be deactivated
Error: Driver 'vexpress-muxfpga' is already registered, aborting...
drm-clcd-pl111 10020000.clcd: initializing Versatile Express PL111
drm-clcd-pl111 10020000.clcd: DVI muxed to daughterboard 1 (core tile) CLCD
drm-clcd-pl111 10020000.clcd: found bridge on endpoint 0
drm-clcd-pl111 10020000.clcd: Using non-panel bridge
[drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[drm] No driver support for vblank timestamp query.
[drm] Initialized pl111 1.0.0 20170317 for 10020000.clcd on minor 0
Console: switching to colour frame buffer device 128x48
drm-clcd-pl111 10020000.clcd: fb0: pl111drmfb frame buffer device
physmap-flash 40000000.flash: physmap platform flash device: [mem 0x40000000-0x43ffffff]
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
physmap-flash 40000000.flash: physmap platform flash device: [mem 0x44000000-0x47ffffff]
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
Concatenating MTD devices:
(0): "40000000.flash"
(1): "40000000.flash"
into device "40000000.flash"
physmap-flash 48000000.psram: physmap platform flash device: [mem 0x48000000-0x49ffffff]
libphy: Fixed MDIO Bus: probed
libphy: smsc911x-mdio: probed
smsc911x 4e000000.ethernet eth0: MAC Address: 52:54:00:12:34:56
isp1760 4f000000.usb: bus width: 32, oc: digital
isp1760 4f000000.usb: NXP ISP1760 USB Host Controller
isp1760 4f000000.usb: new USB bus registered, assigned bus number 1
isp1760 4f000000.usb: Scratch test failed.
isp1760 4f000000.usb: can't setup: -19
isp1760 4f000000.usb: USB bus 1 deregistered
usbcore: registered new interface driver usb-storage
rtc-pl031 10017000.rtc: registered as rtc0
mmci-pl18x 10005000.mmci: Got CD GPIO
mmci-pl18x 10005000.mmci: Got WP GPIO
mmci-pl18x 10005000.mmci: mmc0: PL181 manf 41 rev0 at 0x10005000 irq 25,26 (pio)
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
aaci-pl041 10004000.aaci: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 24
aaci-pl041 10004000.aaci: FIFO 512 entries
oprofile: using arm/armv7-ca9
NET: Registered protocol family 17
9pnet: Installing 9P2000 support
Registering SWP/SWPB emulation handler
rtc-pl031 10017000.rtc: setting system clock to 2020-05-11T23:34:36 UTC (1589240076)
ALSA device list:
#0: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 24
input: AT Raw Set 2 keyboard as /devices/platform/smb@4000000/smb@4000000:motherboard/smb@4000000:motherboard:iofpga@7,00000000/10006000.kmi/serio0/input/input0
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/smb@4000000/smb@4000000:motherboard/smb@4000000:motherboard:iofpga@7,00000000/10007000.kmi/serio1/input/input2
VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
Please append a correct "root=" boot option; here are the available partitions:
1f00 131072 mtdblock0
(driver?)
1f01 32768 mtdblock1
(driver?)
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.12cjhackerz-arm-kernel #1
Hardware name: ARM-Versatile Express
[<801124f8>] (unwind_backtrace) from [<8010cc6c>] (show_stack+0x10/0x14)
[<8010cc6c>] (show_stack) from [<807a48b8>] (dump_stack+0x90/0xa4)
[<807a48b8>] (dump_stack) from [<8012b12c>] (panic+0x104/0x304)
[<8012b12c>] (panic) from [<80a01504>] (mount_block_root+0x1ec/0x280)
[<80a01504>] (mount_block_root) from [<80a016bc>] (mount_root+0x124/0x140)
[<80a016bc>] (mount_root) from [<80a0182c>] (prepare_namespace+0x154/0x190)
[<80a0182c>] (prepare_namespace) from [<807bc05c>] (kernel_init+0x8/0x110)
[<807bc05c>] (kernel_init) from [<801010e8>] (ret_from_fork+0x14/0x2c)
Exception stack(0x9e493fb0 to 0x9e493ff8)
3fa0: 00000000 00000000 00000000 00000000
3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---
If everything was done correctly till now your kernel should boot and stop at kernel panic. Don’t worry it’s okay because kernel was not able to find root file system. To exit out of qemu press Ctrl+a x (control a and then x).
Creating ramdisk for root filesystem in RAM with buildroot
Get cross compiler for our userspace programs
sudo apt install binutils-arm-linux-gnueabi gcc-arm-linux-gnueabi g++-arm-linux-gnueabi -y
Download latest stable buildroot release https://buildroot.org/download.html
wget https://buildroot.org/downloads/buildroot-2020.02.1.tar.gz && tar -xvf
Generate .config file
cd buildroot-2020.02.1 && make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- qemu_arm_vexpress_defconfig
Now we need to edit config and tell buildroot to generate gzip compressed ramdisk
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- xconfig
File system images > cpio the root file system
You can can also install some software such as Apache web server, tinyssh (SSH server) etc. from Target packages menu. Go to Kernel menu and uncheck it since we already build it seprately. When ready just do make. It will download stuff and start compilation.
make -j$(nproc) ARCH=arm
Time to get another cup of coffee this is gonna take more than hour :D
Successful compilation of filesystem
Your prepared filesystem files can be found under output/images
cp ./output/images/rootfs.cpio.gz ~/my_firmware
To test everything now you can directly boot into system from your files
qemu-system-arm -M vexpress-a9 -m 512M \
-kernel zImage -dtb vexpress-v2p-ca9.dtb \
-initrd rootfs.cpio.gz \
-append "root=/dev/ram console=ttyAMA0,38400n8" -nographic
Time to build u-boot and configure it
There are many ways to boot os from u-boot but today we will see single component uImage booting (http://www.denx.de/wiki/pub/U-Boot/Documentation/multi_image_booting_scenarios.pdf). uImage fileformat is wrapper for u-boot. It tells bootloader where to load given uImage file in memory, what kind of file it is (kernel, ramdisk, dtb etc) and is there any compression present or not.
Whenever linux system boots, bootloader puts kernel at start of RAM. After that intial filesystem and dtb is loaded into ram. Bootloader execute kernel by passing required kernel arguments. Think kernel as big binary program just like .ELF file. And like any other program it takes some arguments for example if you type ls -a it shows you all files and for ls -al is shows you all files with permission details. Format for kernel argument is var1=value1 var2=value2
For booting from u-boot we will pass following kernel arguments to kernel
root=/dev/ram console=ttyAMA0,38400n8
Above arguments tells kernel to seek root filesystem in ram and set console output to serial port on specified baudrate.
Let’s download and compile u-boot from source, pick any most recent version from https://ftp.denx.de/pub/u-boot/
wget https://ftp.denx.de/pub/u-boot/u-boot-2020.04.tar.bz2 && tar -xvf u-boot-2020.04.tar.bz2
cd u-boot-2020.04 && make ARCH=arm CROSS_COMPILE=arm-none-eabi- vexpress_ca9x4_defconfig && make -j$(nproc) ARCH=arm CROSS_COMPILE=arm-none-eabi-
Finally copy it to our project directory
cp ./u-boot ~/my_firmware
To test it we will pass file to -kernel option in qemu will put it at start of ram so it will directly boot into u-boot bootloader.
qemu-system-arm -M vexpress-a9 -m 512M -kernel u-boot
U-Boot 2020.04 (May 12 2020 - 08:55:56 +0530)
DRAM: 512 MiB
WARNING: Caches not enabled
Flash: 128 MiB
MMC: MMC: 0
*** Warning - bad CRC, using default environment
In: serial
Out: serial
Err: serial
Net: smc911x-0
Hit any key to stop autoboot: 0
MMC Device 1 not found
no mmc device at slot 1
Card did not respond to voltage select!
smc911x: MAC 52:54:00:12:34:56
smc911x: detected LAN9118 controller
smc911x: phy initialized
smc911x: MAC 52:54:00:12:34:56
BOOTP broadcast 1
DHCP client bound to address 10.0.2.15 (3 ms)
*** Warning: no boot file name; using '0A00020F.img'
Using smc911x-0 device
TFTP from server 10.0.2.2; our IP address is 10.0.2.15
Filename '0A00020F.img'.
smc911x: MAC 52:54:00:12:34:56
...
TFTP error: trying to overwrite reserved memory...
smc911x: MAC 52:54:00:12:34:56
Wrong Image Format for bootm command
ERROR: can't get kernel image!
If u get output as shown above it means now we can finally create our firmware image file
One last information you should be aware of is that after u-boot executes it will reallocate itself to 0x60800000 (+8MB from start of the RAM).
U-boot self reallocation
U-boot self reallocation hexdump and RAM comparison
This to make some space free for kernel and filesystem because after boot kernel will go at the start of the RAM. We need to know filesize of all files to avoid overwriting in memory by providing enough space between them. I decided to have approx 2MB distance between each file which gets loaded into RAM.
ls -al --block-size=K zImage rootfs.cpio.gz vexpress-v2p-ca9.dtb u-boot
-rw-r--r-- 1 cjhackerz cjhackerz 6294K May 12 07:59 rootfs.cpio.gz
-rwxr-xr-x 1 cjhackerz cjhackerz 4761K May 12 09:00 u-boot
-rw-r--r-- 1 cjhackerz cjhackerz 14K May 12 04:59 vexpress-v2p-ca9.dtb
-rwxr-xr-x 1 cjhackerz cjhackerz 4653K May 12 04:59 zImage
Our kernel(zImage) is almost 5MB, ramdisk(rootfs.cpio.gz) 6.5MB and device tree blob (vexpress-v2p-ca9.dtb) 14KB so basically we want to put our u-boot after 16MB (Address: 0x61000000) from start of the RAM. So our final memory mapping in RAM should be as table below:
| Address | Offset (from start of the RAM) | Filename | Size |
|---|---|---|---|
| 0x60010000 | 0x10000(64KB) | zImage | 4.5MB |
| 0x60600000 | 0x600000(6MB) | rootfs.cpio.gz | 6.1MB |
Now create uImage files for u-boot
mkimage -A arm -C none -O linux -T kernel -d zImage -a 0x60010000 -e 0x60010000 kernel.uImage
mkimage -A arm -C gzip -O linux -T ramdisk -d rootfs.cpio.gz -a 0x60600000 -e 0x60600000 rootfs.uImage
Now we need to output them to packed firmware file. To do that we will do is copy files from TFTP server to particular addresses empty NOR flash. And qemu will automatically store output of flash contents of our files once data is copied to it.
sudo apt install -y tftpd-hpa && sudo systemctl enable tftpd-hpa && sudo systemctl restart tftpd-hpa
Copy files to server root
sudo cp *.uImage zImage rootfs.cpio.gz /srv/tftp/
Start u-boot with -kernel option (let autoboot finish with error) and pass empty file to -pflash option
dd if=/dev/zero of=firmware.bin bs=1 count=64M
Note: Please follow my ARM lab setup guide to setup NAT networking in qemu.
sudo qemu-system-arm -M vexpress-a9 -m 512M -kernel u-boot -pflash firmware.bin -nic tap
Follow steps in video given below
So I am loading kernel.uImage at address 0x61000000, rootfs.uImage at 0x62000000 and vexpress-v2p-ca9.dtb at 0x63000000
cp [source address of RAM] [destination address in flash] [number of bytes in hex]
So we will be copying data to RAM from TFTP and from RAM to flash memory. In NOR Flash I leaving 1 sector empty so in flinfo command in u-boot I can visually see from padding that after empty sector new file beings. Before writing to flash you have to turn of write protection and format it with erase command.
=> flinfo
Bank # 1: CFI conformant flash (32 x 16) Size: 64 MB in 256 Sectors
Intel Extended command set, Manufacturer ID: 0x89, Device ID: 0x0018
Erase timeout: 16384 ms, write timeout: 3 ms
Buffer write timeout: 3 ms, buffer size: 2048 bytes
Sector Start Addresses:
40000000 E RO 40040000 RO 40080000 400C0000 40100000
40140000 40180000 401C0000 40200000 40240000
40280000 402C0000 40300000 40340000 40380000
403C0000 40400000 40440000 40480000 404C0000
40500000 40540000 40580000 405C0000 40600000
40640000 40680000 406C0000 40700000 40740000
40780000 407C0000 40800000 40840000 40880000
408C0000 40900000 40940000 40980000 409C0000
40A00000 40A40000 40A80000 40AC0000 40B00000
40B40000 40B80000 40BC0000 40C00000 40C40000
40C80000 40CC0000 40D00000 40D40000 40D80000
40DC0000 40E00000 40E40000 40E80000 40EC0000
40F00000 40F40000 40F80000 40FC0000 41000000
41040000 41080000 410C0000 41100000 41140000
41180000 411C0000 41200000 41240000 41280000 E
412C0000 41300000 41340000 41380000 413C0000
41400000 41440000 41480000 414C0000 41500000
41540000 41580000 415C0000 41600000 41640000
41680000 416C0000 41700000 41740000 41780000
417C0000 41800000 41840000 41880000 418C0000
41900000 41940000 41980000 419C0000 41A00000
41A40000 41A80000 41AC0000 41B00000 41B40000
41B80000 41BC0000 41C00000 41C40000 41C80000
41CC0000 41D00000 41D40000 41D80000 41DC0000
41E00000 41E40000 41E80000 41EC0000 41F00000
41F40000 41F80000 41FC0000 42000000 42040000
42080000 420C0000 42100000 42140000 42180000
421C0000 42200000 42240000 42280000 422C0000
42300000 42340000 42380000 423C0000 42400000
42440000 42480000 424C0000 42500000 42540000
42580000 425C0000 42600000 42640000 42680000
426C0000 42700000 42740000 42780000 427C0000
42800000 42840000 42880000 428C0000 42900000
42940000 42980000 429C0000 42A00000 42A40000
42A80000 42AC0000 42B00000 42B40000 42B80000 E
42BC0000 42C00000 E 42C40000 E 42C80000 E 42CC0000 E
42D00000 E 42D40000 E 42D80000 E 42DC0000 E 42E00000 E
42E40000 E 42E80000 E 42EC0000 E 42F00000 E 42F40000 E
42F80000 E 42FC0000 E 43000000 E 43040000 E 43080000 E
430C0000 E 43100000 E 43140000 E 43180000 E 431C0000 E
43200000 E 43240000 E 43280000 E 432C0000 E 43300000 E
43340000 E 43380000 E 433C0000 E 43400000 E 43440000 E
43480000 E 434C0000 E 43500000 E 43540000 E 43580000 E
435C0000 E 43600000 E 43640000 E 43680000 E 436C0000 E
43700000 E 43740000 E 43780000 E 437C0000 E 43800000 E
43840000 E 43880000 E 438C0000 E 43900000 E 43940000 E
43980000 E 439C0000 E 43A00000 E 43A40000 E 43A80000 E
43AC0000 E 43B00000 E 43B40000 E 43B80000 E 43BC0000 E
43C00000 E 43C40000 E 43C80000 E 43CC0000 E 43D00000 E
43D40000 E 43D80000 E 43DC0000 E 43E00000 E 43E40000 E
43E80000 E 43EC0000 E 43F00000 E 43F40000 E 43F80000 E
43FC0000 E
So I ended up with following mapping in NOR flash:
- 40040000 [kernel.uImage]
- 412C0000 [rootfs.uImage]
- 42BC0000 [vexpress-v2p-ca9.dtb]
After copying data to flash just directly close your qemu. Changes will now show up in firmware.bin file.
binwalk firmware.bin
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
262144 0x40000 uImage header, header size: 64 bytes, header CRC: 0xFCAABEFB, created: 2020-05-12 05:37:13, image size: 4763936 bytes, Data Address: 0x60010000, Entry Point: 0x60010000, data CRC: 0x92589585, OS: Linux, CPU: ARM, image type: OS Kernel Image, compression type: none, image name: ""
262208 0x40040 Linux kernel ARM boot executable zImage (little-endian)
280824 0x448F8 gzip compressed data, maximum compression, from Unix, last modified: 1970-01-01 00:00:00 (null date)
17039360 0x1040000 uImage header, header size: 64 bytes, header CRC: 0xB41113D6, created: 2020-05-12 05:37:40, image size: 6444900 bytes, Data Address: 0x60600000, Entry Point: 0x60600000, data CRC: 0x6711A199, OS: Linux, CPU: ARM, image type: RAMDisk Image, compression type: gzip, image name: ""
17039424 0x1040040 gzip compressed data, maximum compression, from Unix, last modified: 1970-01-01 00:00:00 (null date)
19660800 0x12C0000 uImage header, header size: 64 bytes, header CRC: 0xB41113D6, created: 2020-05-12 05:37:40, image size: 6444900 bytes, Data Address: 0x60600000, Entry Point: 0x60600000, data CRC: 0x6711A199, OS: Linux, CPU: ARM, image type: RAMDisk Image, compression type: gzip, image name: ""
19660864 0x12C0040 gzip compressed data, maximum compression, from Unix, last modified: 1970-01-01 00:00:00 (null date)
22576367 0x1587CEF MySQL MISAM index file Version 7
45875200 0x2BC0000 device tree image (dtb)
cd u-boot-2020.04 && make ARCH=arm CROSS_COMPILE=arm-none-eabi- menuconfig
Enable boot arguments (press space to select option) below that option press enter to input our boot argument
Entering value for bootargs
Enable a default value for bootcmd
Entering value for bootargs
Entering value for bootargs
Exit and Save changes finally compile it again
make -j$(nproc) ARCH=arm CROSS_COMPILE=arm-none-eabi-
cp u-boot ~/my_firmware/u-boot-flashboot
Testing for one last time
qemu-system-arm -M vexpress-a9 -m 512M -kernel u-boot-flashboot -pflash firmware.bin -nographic
Final outcome
TheEnd
And that’s it. Yes indeeed it is bit hard and time consuming process but one can learn a lot about u-boot bootloader functionalities. To sum up what we did was understood our memomry mappings, compiled our stuff and put everything inside a single file which will be used as emulated NOR Flash inside Qemu. And from flash memory bootloader will load stuff back into the RAM as we decided. If you liked this blog post please share it as much as you can so more people can learn this knowledge. Don’t forget to follow me on twitter as well for latest updates from me @cjhackerz