Summary
This article is a tip that explains how it is possible to build a RIOT-OS
application with Podman and the official build container. And I would like to
take this opportunity to introduce you to Podman and RIOT-OS.
Podman
Some Linux distribution, like Fedora chosen to officially only support
Podman instead of Docker for some available reason:
- It is daemonless container engine.
- It is rootless.
- It follows Open Container Initiative (OCI) standards.
- It is safer than the Docker engine.
- It introduces the notion of Pods: a group of container(s) that share storage
or network resources.
Moreover, Podman is able to use the images built by the Docker engine and
has been stored in Docker registry.
However, most of the time the Podman commands are identical to that of
Docker, the a simple alias is enough to be misleading:
alias docker=podman.
But as Podman is rootless and safer than Docker, then sometimes it is
necessary to specify additional security parameters.
RIOT-OS
RIOT-OS is a memory-constrained RTOS, such as Contiki, that provides
real-time and multithreading abilities, and it runs on processors from 8bits to
32bits.
It was designed for IoT devices then to be low power consumption and it
provides three very complete network stacks including some protocols as:
The RIOT-OS project also provides some useful tools including a build
container (riotdocker).
And the build environment of RIOT-OS offers a Makefile to build an
application with this container simply by setting the variable BUILD_IN_DOCKER
to 1. Then the prebuilt image is downloaded and instantiated to execute the
make command.
By default, this feature is configured to be used with the Docker engine,
but it is possible to override some variables from the build environment
either to use a custom prebuilt image, either use another engine or to use
custom engine parameters.
Then here, we will use these environments variable to instantiate a container
with Podman (instead of Docker) and with the required parameters.
Tip of the day
In the following example, we build the Helloworld application for a STM32
Discovery board.
To do that we specify the engine by setting the variable DOCKER to the value
podman. The variable DOCKER_USER is set empty because in the variable
DOCKER_RUN_FLAGS the parameter --userns is set to keep-id to map the
uid:gid of the current rootless user (from host) with the values that will be
used into the container.
export BUILD_IN_DOCKER=1
export DOCKER="podman"
export DOCKER_USER=""
export DOCKER_RUN_FLAGS="--rm -i -t --security-opt seccomp=unconfined --security-opt label=disable --userns=keep-id"
export DOCKER_MAKE_ARGS="-j$(nproc)"
make BOARD=stm32l476g-disco
Launching build container using image "riot/riotbuild:latest".
podman run --rm -i -t --security-opt seccomp=unconfined --security-opt label=disable --userns=keep-id -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/tperrot/dev/tprrt/pwm-ramp-gen/RIOT:/data/riotbuild/riotbase:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -v '/home/tperrot/dev/tprrt/pwm-ramp-gen:/data/riotbuild/riotproject:delegated' -e 'RIOTPROJECT=/data/riotbuild/riotproject' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -v '/home/tperrot/dev/tprrt/pwm-ramp-gen/.git:/home/tperrot/dev/tprrt/pwm-ramp-gen/.git:delegated' -e 'BOARD=stm32l476g-disco' -w '/data/riotbuild/riotproject/' 'riot/riotbuild:latest' make 'BOARD=stm32l476g-disco' -j8
Building application "hello-world" for "stm32l476g-disco" with MCU "stm32".
[INFO] cloning stm32cmsis
fatal: not a git repository: /data/riotbuild/riotbase/../.git/modules/RIOT
Cloning into '/data/riotbuild/riotbase/cpu/stm32/include/vendor/cmsis/l4'...
remote: Enumerating objects: 364, done.
remote: Counting objects: 100% (364/364), done.
remote: Compressing objects: 100% (71/71), done.
remote: Total 364 (delta 309), reused 344 (delta 289), pack-reused 0
Receiving objects: 100% (364/364), 709.56 KiB | 561.00 KiB/s, done.
Resolving deltas: 100% (309/309), done.
HEAD is now at e442c72 Release v1.6.1
[INFO] updating stm32cmsis /data/riotbuild/riotbase/cpu/stm32/include/vendor/cmsis/l4/.pkg-state.git-downloaded
echo e442c72651e8d4757f6562acc14da949644944ce > /data/riotbuild/riotbase/cpu/stm32/include/vendor/cmsis/l4/.pkg-state.git-downloaded
[INFO] patch stm32cmsis
"make" -C /data/riotbuild/riotbase/boards/stm32l476g-disco
"make" -C /data/riotbuild/riotbase/core
"make" -C /data/riotbuild/riotbase/cpu/stm32
"make" -C /data/riotbuild/riotbase/drivers
"make" -C /data/riotbuild/riotbase/sys
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common
"make" -C /data/riotbuild/riotbase/cpu/stm32/periph
"make" -C /data/riotbuild/riotbase/drivers/periph_common
"make" -C /data/riotbuild/riotbase/cpu/stm32/stmclk
"make" -C /data/riotbuild/riotbase/sys/auto_init
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common/periph
"make" -C /data/riotbuild/riotbase/cpu/stm32/vectors
"make" -C /data/riotbuild/riotbase/sys/malloc_thread_safe
"make" -C /data/riotbuild/riotbase/sys/newlib_syscalls_default
"make" -C /data/riotbuild/riotbase/sys/pm_layered
"make" -C /data/riotbuild/riotbase/sys/stdio_uart
text data bss dec hex filename
8900 112 2300 11312 2c30 /data/riotbuild/riotproject/bin/stm32l476g-disco/hello-world.elf
Introduction
In this article, I will dissect how the chrt applet from the release 1.32.0 of Busybox works, what it does, etc.
This command is a Linux utils allowing to consult or to modify the scheduling attributes of a process.
chrt -m
SCHED_OTHER min/max priority : 0/0
SCHED_FIFO min/max priority : 1/99
SCHED_RR min/max priority : 1/99
SCHED_BATCH min/max priority : 0/0
SCHED_IDLE min/max priority : 0/0
SCHED_DEADLINE min/max priority : 0/0
pidof firefox
6987 6851 6825 6816 6800 6771 6767 6761 6720 6611
chrt -p 6987
pid 6987's current scheduling policy: SCHED_OTHER
pid 6987's current scheduling priority: 0
sudo chrt -f -p 1 6987
chrt -p 6987
pid 6987's current scheduling policy: SCHED_FIFO
pid 6987's current scheduling priority: 1
Busybox provides an applet which size, once compiled, and ten times smaller than that of the binary implementation
and with some limitations.
The dissection
The implementation of the chrt applet is in the file util-linux/chrt.c that containing several functions which are
called in the main function of this applet.
The main function of this applet is divised in three main parts:
- the first parses the command options
- the second prints the scheduler's information
- the last one, to apply scheduler changes in case of a set
At start of main, the character string containing the options are parsed to obtain a bitfield easier to use:
opt = getopt32(argv, "^"
"+" "mprfobi"
"\0"
/* only one policy accepted: */
"r--fobi:f--robi:o--rfbi:b--rfoi:i--rfob"
);
If the (-m) is set then the min and max valid priorities for each scheduling policies are shown and the command is existed:
if (opt & OPT_m) { /* print min/max and exit */
show_min_max(SCHED_OTHER);
show_min_max(SCHED_FIFO);
show_min_max(SCHED_RR);
show_min_max(SCHED_BATCH);
show_min_max(SCHED_IDLE);
fflush_stdout_and_exit(EXIT_SUCCESS);
}
The function show_min_max sends use the Posix functions sched_get_priority_max and sched_get_priority_min from the
standard C library to send a syscall to the kernel in order to obtain the min and max values accepted by each policy:
max = sched_get_priority_max(pol);
min = sched_get_priority_min(pol);
if ((max|min) < 0)
fmt = "SCHED_%s not supported\n";
Otherwise the required options and arguments to show or to apply real-time attributes of a process:
//if (opt & OPT_r)
// policy = SCHED_RR; - default, already set
if (opt & OPT_f)
policy = SCHED_FIFO;
if (opt & OPT_o)
policy = SCHED_OTHER;
if (opt & OPT_b)
policy = SCHED_BATCH;
if (opt & OPT_i)
policy = SCHED_IDLE;
argv += optind;
if (!argv[0])
bb_show_usage();
if (opt & OPT_p) {
pid_str = *argv++;
if (*argv) { /* "-p PRIO PID [...]" */
priority = pid_str;
pid_str = *argv;
}
/* else "-p PID", and *argv == NULL */
pid = xatoul_range(pid_str, 1, ((unsigned)(pid_t)ULONG_MAX) >> 1);
} else {
priority = *argv++;
if (!*argv)
bb_show_usage();
}
Then the applet uses the Posix function sched_getscheduler provides by the standard C library to obtain the scheduling attributes of the process specified by the pid.
print_rt_info:
pol = sched_getscheduler(pid);
if (pol < 0)
bb_perror_msg_and_die("can't %cet pid %u's policy", 'g', (int)pid);
Finally, when the chrt applet is used to modify scheduling attributes then the Posix function sched_getscheduler is used and the new scheduling attributes are showed:
if (sched_setscheduler(pid, policy, &sp) < 0)
bb_perror_msg_and_die("can't %cet pid %u's policy", 's', (int)pid);
if (!argv[0]) /* "-p PRIO PID [...]" */
goto print_rt_info;
The function sched_getscheduler and sched_getscheduler will send a syscall to the scheduler subsystem of the kernel Linux.
This subsystem also exposes this information from /proc:
cat /proc/6987/sched
WebExtensions (6987, #threads: 23)
-------------------------------------------------------------------
se.exec_start : 4421312.640001
se.vruntime : 344438.942254
se.sum_exec_runtime : 38238.466094
se.nr_migrations : 6811
nr_switches : 49452
nr_voluntary_switches : 21749
nr_involuntary_switches : 27703
se.load.weight : 1048576
se.runnable_weight : 1048576
se.avg.load_sum : 3415
se.avg.runnable_load_sum : 3415
se.avg.util_sum : 3497621
se.avg.load_avg : 74
se.avg.runnable_load_avg : 74
se.avg.util_avg : 74
se.avg.last_update_time : 4421312640000
se.avg.util_est.ewma : 75
se.avg.util_est.enqueued : 75
policy : 0
prio : 120
clock-delta : 89
mm->numa_scan_seq : 0
numa_pages_migrated : 0
numa_preferred_nid : -1
total_numa_faults : 0
current_node=0, numa_group_id=0
numa_faults node=0 task_private=0 task_shared=0 group_private=0 group_shared=0
Limitations
Below a short list of limitations that I observed during my analysis of this applet.
Resetting scheduling policy
The chrt applet doesn't offer an option (-R) to specify if the scheduling policy should be applied or reseted when a
process is fork to create children. This feature, introduced since Linux 2.6.32, can be only enabled or disabled at the
build of busybox and it is applied on all scheduling attributes modifications done with this applet.
Deadline support
The chrt applet doesn't provide the required scheduling options (-d, -T, -P and -D) to set the deadline scheduling attributes of a process.
Introduction
Since some years, I no longer built a embedded Linux without using a framework, like Open Embedded from the Yocto
project.
Then here, I wanted to make a guide to help you to build quickly, from "scratch" a very minimal embedded Linux to boot a
target.
The following examples have been writen to boot a virtual Qemu target but, they can be adapted to boot a real target.
Moreover, the build environment will be boot strapped with a prebuilt cross-toolchain, I have chosen to use one provided
by Bootlin and using glibc.
Setup the environment
First, It is required to install the packages that are needed to install and use the cross-toolchain but also to compile the host tools and to provide Qemu:
- The Ncurses libraries are only required to execute the command make menuconfig.
- The certificates and wget will be used to download the prebuilt toolchain.
- In the same way, git will be used to checkout the source of Busybox and Linux.
- The Qemu packages will be used to emulate systeme platform and to execute static binaries cross-compiled for aarch64 on the x86-64 host.
apt update
apt install -y --no-install-recommends \
bc \
build-essential \
ca-certificates \
cpio \
file \
flex \
git \
ipxe-qemu \
libncurses5-dev \
libncursesw5-dev \
libssl-dev \
qemu \
qemu-system-aarch64 \
qemu-user-static \
wget
Now, it is time to download and install the prebuild toolchain:
mkdir ~/src
cd ~/src
wget https://toolchains.bootlin.com/downloads/releases/toolchains/aarch64/tarballs/aarch64--glibc--stable-2020.08-1.tar.bz2
tar xvjf aarch64--glibc--stable-2020.08-1.tar.bz2
Once the toolchain has been extracted you have to set the required environment variables to cross-compile binaries:
- PATH: It shall be extended then the cross-tools from the cross-toolchain will be available from the environment
- CROSS_COMPILE: In order to clarify the prefix used by the cross-tools
- ARCH: The architecture of the target platform
ls ~/src/aarch64--glibc--stable-2020.08-1/bin/*gcc
~/src/aarch64--glibc--stable-2020.08-1/bin/aarch64-linux-gcc
export export PATH=~/src/aarch64--glibc--stable-2020.08-1/bin:$PATH
export CROSS_COMPILE=aarch64-linux-
Now, it is possible to call the cross-tools from the shell:
aarch64-linux-gcc -v
Using built-in specs.
COLLECT_GCC=~/src/aarch64--glibc--stable-2020.08-1/bin/aarch64-linux-gcc.br_real
COLLECT_LTO_WRAPPER=~/src/aarch64--glibc--stable-2020.08-1/bin/../libexec/gcc/aarch64-buildroot-linux-gnu/9.3.0/lto-wrapper
Target: aarch64-buildroot-linux-gnu
<...>
Thread model: posix
gcc version 9.3.0 (Buildroot 2020.08-14-ge5a2a90)
Concerning the variable PATH this one will be set afterwards because its value depends of the binary that will be built.
Build the Linux kernel
So, the environment is ready to pull the sources of the latest stable branch of the kernel Linux and to build them:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
cd linux
git checkout -b local/linux-5.4.y origin/linux-5.4.y
# git show HEAD
export ARCH=arm64
make defconfig
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#
# make menuconfig
make -j$(nproc)
<...>
AR drivers/net/ethernet/built-in.a
AR drivers/net/built-in.a
AR drivers/built-in.a
GEN .version
CHK include/generated/compile.h
LD vmlinux.o
MODPOST vmlinux.o
MODINFO modules.builtin.modinfo
LD .tmp_vmlinux.kallsyms1
KSYM .tmp_vmlinux.kallsyms1.o
LD .tmp_vmlinux.kallsyms2
KSYM .tmp_vmlinux.kallsyms2.o
LD vmlinux
SORTEX vmlinux
SYSMAP System.map
Building modules, stage 2.
MODPOST 531 modules
OBJCOPY arch/arm64/boot/Image
GZIP arch/arm64/boot/Image.gz
The command make defconfig will apply the default configuration for the target platform (cf. ARCH=arm64), and the
compilation will be performed by make -j$(nproc).
The commands git show HEAD and make defconfig are optional:
- the first is usefull to verify that the latest commit corresponding to the latest tag of the branch linux-5.4.y.
- the second can be used if you want to customize the kernel configuration.
NB. The kernel Linux but also Busybox and some projects use Kbuild to manage the build options
Populate the sysroot
The easy way to bootstrap a sysroot is to use Busybox that has been created to offer common UNIX tools into a single
executable and it has size-optimized. To create a sysroot, it is only required to add a few configuration files.
The steps to pull and build Busybox are similar to those of the kernel Linux.
git clone git://git.busybox.net/busybox
cd busybox
git checkout -b local/1_32_stable origin/1_32_stable
# git show HEAD
export ARCH=aarch64
export LDFLAGS="--static"
make defconfig
# make menuconfig
make -j$(nproc)
make install
Here, the LDFLAGS is set to force the static link of Busybox quickly, but it is also possible to use
make menuconfig to set CONFIG_STATIC=y. The advantage of the static executable is that it can be tested with Qemu:
qemu-aarch64-static busybox echo "Hello!"
Hello!
qemu-aarch64-static busybox date
Sat Jun 27 15:06:41 UTC 2020
The binary qemu-aarch64-static allows to execute a binary built for another architecture on the host computer, for
example here it allows to execute the Busybox binary compiled for an aarch64 target on a x86-64 host.
The last command make install created a tree into the _install directory that can be used to populate the sysroot:
ls -l _install
total 4
drwxr-xr-x. 1 tperrot tperrot 974 Nov 30 15:22 bin
lrwxrwxrwx. 1 tperrot tperrot 11 Nov 30 15:22 linuxrc -> bin/busybox
drwxr-xr-x. 1 tperrot tperrot 986 Nov 30 15:22 sbin
drwxr-xr-x. 1 tperrot tperrot 14 Nov 30 15:22 usr
ls -l _install/bin
<...>
lrwxrwxrwx. 1 tperrot tperrot 7 Nov 30 15:22 umount -> busybox
lrwxrwxrwx. 1 tperrot tperrot 7 Nov 30 15:22 uname -> busybox
lrwxrwxrwx. 1 tperrot tperrot 7 Nov 30 15:22 usleep -> busybox
lrwxrwxrwx. 1 tperrot tperrot 7 Nov 30 15:22 vi -> busybox
lrwxrwxrwx. 1 tperrot tperrot 7 Nov 30 15:22 watch -> busybox
lrwxrwxrwx. 1 tperrot tperrot 7 Nov 30 15:22 zcat -> busybox
In order, to finalize this minimal sysroot, it is required to create a rcS init script:
mkdir _install/proc _install/sys _install/dev _install/etc _install/etc/init.d
cat > _install/etc/init.d/rcS << EOF
#!/bin/sh
mount -t proc none /proc
mount -t sysfs none /sys
/sbin/mdev -s
[ ! -h /etc/mtab ] && ln -s /proc/mounts /etc/mtab
[ ! -f /etc/resolv.conf ] && cat /proc/net/pnp > /etc/resolv.conf
EOF
chmod +x _install/etc/init.d/rcS
Build the filesystem
The target of this step is to package the sysroot tree into a filesystem that can be mounted by the kernel.
There is two available possibilities, either build a ramfs or a rootfs.
Globally, the difference between both is that:
- the ramfs is a very simple filesystem that can be used by the kernel to create a block device into the RAM space from an archive.
- the rootfs is a filesystem mounted from a non volatile device by the kernel.
For more information about the difference between the ramfs and the rootfs, you can you refer to the kernel documentation.
Build a ramfs
To build the ramfs we will use cpio and gzip to construct the compressed archive after modifying the rights:
mkdir _rootfs
rsync -a _install/ _rootfs
chown -R root:root _rootfs
cd _rootfs
find . | cpio -o --format=newc > ../rootfs.cpio
cd ..
gzip -c rootfs.cpio > rootfs.cpio.gz
Build a rootfs
To build the rootfs, the first step is to create an empty binary blob that will be mounted into a loop device to be
formated to create a ext3 filesytem. Then the tree can be copied and the rights updated.
dd if=/dev/zero of=rootfs.img bs=1M count=10
mke2fs -j rootfs.img
mkdir _rootfs
mount -o loop rootfs.img _rootfs
rsync -a _install/ _rootfs
chown -R root:root _rootfs
sync
umount _rootfs
Boot the target
Following, the qemu commands to boot the minimal embedded Linux system that has been built.
# With the ramfs
qemu-system-aarch64 -nographic -no-reboot -machine virt -cpu cortex-a57 -smp 2 -m 256 \
-kernel ~/src/linux/arch/arm64/boot/Image \
-initrd ~/src/busybox/rootfs.cpio.gz \
-append "panic=5 ro ip=dhcp root=/dev/ram rdinit=/sbin/init"
# With the rootfs
qemu-system-aarch64 -nographic -no-reboot -machine virt -cpu cortex-a57 -smp 2 -m 256 \
-kernel ~/src/linux/arch/arm64/boot/Image \
-append "panic=5 ro ip=dhcp root=/dev/vda" \
-drive file=~/src/busybox/rootfs.img,format=raw,if=none,id=hd0 -device virtio-blk-device,drive=hd0
Then the target will be boot to shell, "It's alive!":
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd070]
[ 0.000000] Linux version 5.10.0-rc5 (tperrot@27ea4a863f61) (aarch64-linux-gcc.br_real (Buildroot 2020.08-14-ge5a2a90) 9.3.0, GNU ld (GNU Binutils) 2.33.1) #1 SMP PREEMPT Mon Nov 30 14:40:05 UTC 2020
[ 0.000000] Machine model: linux,dummy-virt
<...>
[ 0.858346] Sending DHCP requests ., OK
[ 0.870558] IP-Config: Got DHCP answer from 10.0.2.2, my address is 10.0.2.15
[ 0.870909] IP-Config: Complete:
[ 0.871199] device=eth0, hwaddr=52:54:00:12:34:56, ipaddr=10.0.2.15, mask=255.255.255.0, gw=10.0.2.2
[ 0.871566] host=10.0.2.15, domain=, nis-domain=(none)
[ 0.871825] bootserver=10.0.2.2, rootserver=10.0.2.2, rootpath=
[ 0.871866] nameserver0=10.0.2.3
[ 0.872389]
[ 0.875863] ALSA device list:
[ 0.876151] No soundcards found.
[ 0.879353] uart-pl011 9000000.pl011: no DMA platform data
[ 0.920237] Freeing unused kernel memory: 5952K
[ 0.921223] Run /sbin/init as init process
Please press Enter to activate this console.
Welcome,
After closing my last blog since seventeen years, in order to share my knowledges and my little experiments about
embedded open source. You would have understood it, this blog will mainly speak about embedded Linux operating system,
but also about open firmware and rtos, as well as related topics like virtualization, security, etc.
I hope you will like the articles of this blog, enjoy the reading.