Alpine boot process on the Raspberry Pi
Today we will have a look at the Alpine Linux boot process on a Raspberry Pi in some detail.
The picture shows the contents of a “virgin” SD card with the Alpine image, which has not been booted yet.
This picture shows the contents of the boot folder.
Initial boot Stages on the Raspberry Pi
Refer to this link for an in-depth guide at different boot media:
The Raspberry Pi does not have a classical “BIOS” as in x86 based IBM PC compatible computers. It’s CPU should be thought of as an add-on to the VideoCore IV for boot purposes, rather than the other way around.
Thus, initially the VideoCore IV, the GPU, has control. It starts off from it’s own ROM (first stage), and checks the boot sources for a file called bootcode.bin. For our purposes we will assume that this file is loaded from the SD card.
bootcode.bin (51 KB) is loaded into the local 128K L2 cache of the GPU. It enables RAM and loads the third stage: start.elf (2759 KB)
start.elf will read config.txt, and set up the appropriate settings for the VideoCore IV. config.txt is a parameter file for the Raspberry Pi where you can set e.g. the video mode, overscan, audio options, MPEG codecs, overclocking settings, etc. – it is “like” the BIOS settings part of the traditional computer. start.elf itself is the firmware for the GPU, which is it’s own operating system called “VideoCore OS”.
NB: There are different versions of start.elf, including start_x.elf which includes camera drivers, and start_cd.elf – which is a cut down version of start.elf, to be used when the GPU memory is reduced to the maximum: gpu_mem=16. This latter version might be interesting for embedded systems. Refer to this site for more information: https://www.raspberrypi.org/documentation/configuration/boot_folder.md
config.txt is optional. If you use the same SD card (or image) for several Pi’s, you can set conditional filters to apply to certain scenarios; you can even apply a condition based on the serial of the Raspberry Pi, or on the state of a GPIO. There is also the possibility to include additional files into the config.txt. Taken together, you get a quite powerful system for the basic configuration of the Raspberry Pi.
Of course, there are also boot options in config.txt, which control how the further boot proceeds.
start.elf then loads the kernel (as specified by config.txt / defaults), and launches it with cmdline.txt as kernel command line .
config.txt boot options
have a look at:
to read more about the available boot options.
Alpine’s default config.txt
Let’s have a look at the config.txt which ships with Alpine’s 3.8.1 ARMHF release:
- this will disable the rainbow screen on start (purely cosmetic)
- do not wait in start.elf before loading the kernel. (a boot delay might be necessary for some SD cards to “get ready”)
- sets split of memory between GPU and CPU. The Pi’s memory is shared. Here we set 256 MB for the GPU, you might want to reduce this on embedded systems which do not drive displays.
- sets the split of memory for Pi’s with only 256 MB of total memory (only some old generation Pi 1B’s)
- conditional statement for Pi Zero; this covers Pi Zero and Pi Zero W
- all the statemens until the next conditional are going to be applied only if we run on a Pi Zero / Zero W
- the Pi Zero / Zero W still use an older ARMv6 ISA CPU (ARM11 @ 1 GHz)
- we will use a kernel compiled for ARMv6
- this specifies the kernel to load
- this must be an uncompressed kernel image file
- 32 bit kernels are loaded to address 0x8000 by default
- initramfs boot/initramfs-rpi
- “the initramfs command specifies both the ramfs filename and the memory address to which to load it”
- initramfs = initial RAM filesystem
- this file can be extracted with 7Zip, for instance (extract twice)
- NOTE: this configuration parameter is specified without the “=”
- this will apply the following configuration lines to all Raspberry Pi 1 – will not match on Pi 2, etc.
- initramfs boot/initramfs-rpi
- notice how we use a different kernel, since here we have a CPU which can run ARMv7 code
- initramfs boot/initramfs-rpi2
- and a different initramfs …
- initramfs boot/initramfs-rpi2
- initramfs boot/initramfs-rpi2
- the following line(s) will again apply to all Pi’s
- include usercfg.txt
- will include a file usercfg.txt into the configuration, which does not yet exist in the default Alpine image.
- here you could put additional custom configuration which you want, e.g. HDMI resolution, etc.
- alternatively you could just paste it below in config.txt
Note: the configuration used in Alpine does not specify a second parameter for the initramfs (address at which to mount the initramfs); thus it is not advisable to copy this 1:1. That being said, it seems to work, probably defaulting to a safe default!
Read more about config.txt:
The Kernel and the Initramfs
Depending on the config.txt settings, the correct kernel will be selected and loaded into RAM. Once it is loaded, the ARM processor core(s) (are/is) released from reset so (it/they) can boot the kernel.
It will be passed the contents of cmdline.txt as kernel command line. start.elf will also pass additional parameters of it’s own, for instance setting DMA channels, the MAC address of the SMSC LAN chip, etc.
Get the cmdline.txt:
dmesg | grep “Command line”
let’s have a look at Alpine’s default cmdline.txt:
modules=loop,squashfs,sd-mod,usb-storage quiet dwc_otg.lpm_enable=0 console=tty1
This loads the modules
- the loop device is used to mount a file as a file system
- compressed read-ony file system for Linux
- especially suitable for embedded systems
- this is used for modloop later on
- SCSI disk driver / block driver
- apparently copied from Alpine default configuration, e.g. here: https://wiki.alpinelinux.org/wiki/Create_a_Bootable_Compact_Flash
- Linux USB mass storage driver
NB: module names may contain both – and _, these symbols can apparently be interchanged.
- sets the default kernel log level to KERN_WARNING, which suppresses all but very serious log messages during boot (source: Raspberry Pi documentation)
- The USB Controller driver, disables USB link power management with this setting. (This is the recommended default)
- defines the serial console
” The initramfs command specifies both the ramfs filename and the memory address to which to load it. It performs the actions of both ramfsfile and ramfsaddr in one parameter. The address can also be followkernel (or 0) to place it in memory after the kernel image. Example values are: initramfs initramf.gz 0x00800000 or initramfs init.gz followkernel. NOTE: This option uses different syntax from all the other options, and you should not use a = character here.” – Raspberry Pi Documentation
There will be a kernel message like this “[ 0.143709] Trying to unpack rootfs image as initramfs…” in dmesg, showing that the image is being unpacked.
The initramfs allows the Linux kernel to add modules, and perform other tasks in preparing for the mounting of the actual file system. It thus makes for a very versatile boot system, without having to recompile the kernel / add overhead to the kernel. For instance for users with an encrypted filesystem, initramfs will ask for the passphrase before it can decrypt and mount the file systems.
Depending on the CPU in your Raspberry Pi, different initramfs files will be used, as set in config.txt:
- boot/initramfs-rpi for Pi 1 / 1B+ and Pi Zero / Zero W, and Compute Module 1
- boot/initramfs-rpi2 for all other Pi’s (2, 3B, 3B+, …)
The initramfs file can be extracted using, for instance, on Windows 7Zip. (You have to extract it twice: the first file contains a gzipped second file)
initramfs images can be in different formats, depending on which algorithms were compiled statically into the kernel (e.g. gzip / bzip2 / LZMA / XZ / LZO / LZ4).
This second file packages a folder and file structure, which you will recognize if you have used Linux before. It is in a special format, “cpio“, which is similar to tar.
if you are curious, you can read more about cpio here:
This is what the file contains:
in bin, it contains:
- compact executable containing several UNIX utilities in one application
- especially for embedded systems
in sbin, it contains
- Alpine package management tool
- this is a tool for profiling startup time
- is used to mount block devices and search them for available apkovl’s, if no apkovl= is passed in the kernel command line
the init file is launched by the Linux kernel as first process, and basically sets up Alpine.
Here is the tool used for generating the initramfs for Alpine:
this is a sequential excerpt from what the init script launched from the initramfs does:
- creates directories /usr/bin, /usr/sbin, /proc, /sys, /dev, /sysroot , /media/cdrom, /media/usb, /tmp, /run recursively
- some of these directories seem to be present already in the initramfs
- installs busybox (sets up symlinks to busybox’ embedded commands)
- sets up PATH to /usr/bin:/bin:/usr/sbin:/sbin
- creates /dev/null using mknod
- mounts proc to /proc
- proc is a process information pseudo-file system
- you can obtain information about the kernel by reading and configure certain things by writing to some files in this directory
- the numbered entries are directories containing information about the process with this particular process id
- … and sysfs to /sys
- this is information about hardware devices, device drivers and kernel subsystems
- parses the options passed via cmdline.txt, the following are recognized:
- alpine_dev autodetect autoraid chart cryptroot cryptdm cryptheader cryptoffset
- cryptdiscards cryptkey debug_init dma init_args keep_apk_new modules ovl_dev
- pkgs quiet root_size root usbdelay ip alpine_repo apkovl alpine_start splash
- blacklist overlaytmpfs rootfstype rootflags nbd resume s390x_net dasd ssh_key
Tip: try changing “quiet” in cmdline.txt to “noquiet” to get a more verbose boot, and see many messages from init
- mounts devtmpfs to /dev
- loads drivers to be able to mount modloop later on
- if the parameter nbd is present and configured (network block device), tries to bring up an ip address, and configure the network block device using nbd-client
- if the root parameter is set
- if [ -n “$KOPT_root” ]; then
- –n evaluates if the length of “$KOPT_root” is nonzero
- then nlplug-findfs is executed
- overlaytmpfs is optionally handled
- the current mount points are migrated to /sysroot
- switch_root is run and control is passed to /sbin/init
- (this is an entry point for installing Alpine Linux in a regular mode, not in read-only mode)
- nlplug-findfs is executed to find the boot media,
- crypto-options are possible and processed here
- found apkovls’ paths are added to /tmp/apkovl (using the -a switch, see https://pi3g.com/2019/01/09/nlplug-findfs-documentation/)
- apparently the file is only created if apkovl’s are found.
- apkovls are matched looking for “*.apkovl.tar.gz*”, that is why backup files which lbu commit creates are not found / applied automatically – they have a structure of hostname.timestamp.tar.gz, e.g. pidoctor.20190110180745.tar.gz
- if the parameter apkovl is set,
- and it is empty – and /tmp/apkovls exists (that is, files were found by nlplug-findfs) – the ovl is set from the first line of /tmp/apkovls
- and it starts with http:// / https:// / ftp:// – a network connection is tried to pull the apkovl
- otherwise the ovl is set to the string in the apkovl option
- if the apkovl exists,
- and is a .gz file, the apkovl is unpackaged tar -C “$dest” -zxvf “$ovl” > $ovlfiles and /tmp/ovlfiles is filled in with tar’s output
- there is code for a splash screen, fbsplash is one of the functions which is included in busybox – it displays fbsplash*.ppm files
- if [ -z “$ALPINE_REPO” ] -> means, if the string $ALPINE_REPO is empty
- under some circumstances default bootservices are added, for the runlevel sysinit, boot, shutdown and default (e.g. the service firstboot)
- apk repositories are added and expanded
- the package alpine-base depends on alpine-baselayout, alpine-conf and some other packages
- it also includes the file /etc/os-release -> which shows the name, version, and pretty name of Alpine Linu
- alpine-baselayout is the base directory layout
- alpine-conf includes, amongst others, the lbu script and setup scripts
- busybox includes the busybox binary
- busybox-initscripts includes some initscripts
- finally, the root directory is switched, and /sbin/init is started
- this will start OpenRC and execute the runlevels
- /sbin/init is a symlink to /bin/busybox
Please note: The apkovl is mounted before modloop and before installing the packages. This allows the apkovl to configure which packages are going to be installed, amongst other things.
sysinit -> boot -> default -> shutdown
modloop will be started after dev-mount, and before checkfs, fsck, hwdriver, modules, hwclock, dev, sysfs
hwdrivers will be started after modloop. hwdrivers will load the drivers using modprobe.
There is also the modloop file (modloop-rpi or modloop-rpi2).
how it is mounted
The right file for your system is determined in the script /etc/init.d/modloop in the function find_modloop()
The result of uname -r is checked against the directory modules/ – the appropriately named subdirectory needs to be in it.
For example: 4.14.69-0-rpi2
This way the right modules for your kernel are used (uname -r prints the kernel release).
modloop is mounted in the directory /.modloop
Symlinks are created from /lib/firmware to /lib/modules/firmware, and /lib/modules is symlinked to /.modloop/modules
Hint: if you want to patch the Raspberry Pi 3B+ WiFi firmware, you have to put it into modloop.
To explore modloop’s contents, you can use (tested on a mint system):
apt-get install squashfs-tools
It contains firmware and drivers.