Skip to content

Qt Embedded Systems – Part 1: Building a Linux Image with Yocto

In Part 1 of the series on Qt Embedded Systems, we build a custom Linux image with Yocto for the Raspberry Pi 3B. When we power on the embedded device, it starts an Internet radio application – called Cuteradio. This QML application is very simple: it can only play a single, hard-wired station. Despite its simplicity, the application requires the Qt modules Qml, Multimedia, Gui, Network and Core, GStreamer 1.x and the platform plugin eglfs, which is a simple, OpenGL accelerated windowing system.

Prerequisites

We need a powerful PC for building Linux images with Yocto. The PC runs a Ubuntu LTS version (16.04 or newer) natively and is powered by at least an Intel Core i7 with four cores and hyper-threading. It has a minimum of 16 GB RAM and 100 GB free disk space. The build of the example Linux image, for example, takes 53 GB disk space.

A very fast Internet connection is a big advantage. The first build downloads gigabytes of sources for hundreds of packages. With a bandwidth of 50 Mb per second, downloading may take longer than building the example image. Once we downloaded all sources, we can work without an Internet connection and share the sources between builds.

We install Docker as described in the post Using Docker Containers for Yocto Builds. Yocto versions are tied to certain versions of Linux distros. For example, Yocto 2.4 Rocko was released in October 2017. If we use Ubuntu 18.04 or newer for Rocko builds, the builds will fail with strange errors. The same build will work without any problems, if we use Ubuntu 17.04 or older. In the Docker container, we can run the Ubuntu version that best matches the Yocto version.

I bought a Raspberry Pi starter kit similar to this kit in 2017. It contained

  • a Raspberry Pi 3 Model B board,
  • a 7″ multi-touch display with 800×480 resolution,
  • an SD card preloaded with the latest Raspian Linux image,
  • a black casing and
  • a power supply.

The post How to Setup Your Raspberry Pi Touchscreen gives a step-by-step instruction how to connect the board of touch display with the board of the Raspberry Pi 3B. The photo shows the result.

The four coloured leads forward the power from the Raspberry Pi board to the touch display board. The ribbon cable on the right passes the display and touch data between the two boards. The bright red Ethernet cable connects the Raspberry Pi with the development PC over the office LAN. The left cable on the top connects 3.5 mm cinch connector with the portable loudspeaker, an Anker Soundcore Mini. The right cable on the top comes from the power supply.

The model 3B has been succeeded by the model 3B+ in 2018 (see here for the starter kit). The 3B+ is clocked 200 MHz faster than the 3B, has PoE (power over Ethernet), has faster Ethernet (1 Gb instead of 100 Mb) and faster Wifi (Wifi a/c instead of b/g/n). The 3B+ works with the same touch display as the 3B and fits in the same casing.

My understanding is that the Linux image for the 3B should work with little or no modifications on the 3B+. I haven’t tried yet for want of a 3B+. I would be interested in your experiences.

Setting Up a Docker Container

We create a Docker container, in which we can build the Linux images with Yocto. We clone the repository dr-yocto into a working directory (/public/Work for me) and switch to the tag ubuntu-18.04.

$ cd /public/Work
$ git clone https://github.com/bstubert/dr-yocto.git
$ cd dr-yocto
$ git checkout ubuntu-18.04

The repository contains two scripts and a Dockerfile:

  • build.sh builds the Docker image from the given Dockerfile.
  • run-shell.sh runs the Docker image and starts an interactive bash shell.
  • 18.04/Dockerfile specifies the build instructions for the Docker image. The directory name reflects the Ubuntu version on which the Docker image is based.

The calling syntax of the build script is

build.sh <tag> <dir-path>

The first argument is the <tag> for the Docker image. The name of the Docker image is dr-yocto:<tag>. The second argument <dir-path> is the path to the directory containing the Dockerfile. For example, we build the Yocto image dr-yocto:18.04 by calling

$ ./build.sh 18.04 ./18.04

in the directory /public/Work/dr-yocto.

I use the Ubuntu version for the tag and the directory path, because the same Ubuntu version works for several Yocto versions. It is important to use a Ubuntu version that was released before the Yocto version. If the Ubuntu version is newer than the Yocto version, we’ll see strange errors. Yocto 2.4 Rocko, for example, was released in October 2017. Rocko builds work with Ubuntu 17.04 or older. I tend to go back to the last LTS release of Ubuntu – Ubuntu 16.04 in this case – to be on the safe side.

As defined in the Dockerfile, the build process

  • uses Ubuntu 18.04 as the basic Linux distro,
  • installs all the packages needed for the Yocto build,
  • makes bash the default shell,
  • installs the repo tool for managing multiple git repositories,
  • sets the locale to en_US.UTF-8,
  • creates the user embeddeduse with the same user and group ID as the user calling the build script,
  • gives the user embeddeduse root privileges through sudo, and
  • sets default user to embeddeduse and the working directory to /public/Work.

Having root privileges in the container comes in handy, when we build a new Yocto version for the first time. There are always some packages missing from the official list of required packages. A quick sudo apt-get -y install <package> solves the problem. Once we know whether the package solved the build problem, we add it to the list of packages in the Dockerfile. This saves us from re-installing the packages for the container over and over again.

We run the Docker image by calling

run-shell.sh <tag>

The argument <tag> is the same tag as passed to build.sh. The command starts a bash shell in the container dr-yocto:<tag>. We test the Docker image dr-yocto:18.04 with the following command.

$ cd /public/Work
$ ./dr-yocto/run-shell.sh 18.04
embeddeduse@25a0938370ff:/public/Work$

Let me introduce some notation. Command lines starting with $ are executed on the host computer. Command lines starting with # are executed in the Docker container. Command lines starting with @ are executed on the target system (e.g, the Raspberry Pi).

Cloning the Layer Repositories

The thousands of metadata files (build recipes, configuration files, etc.) needed to build a Linux distro with Yocto are organised into layers. Our example system consists of five layers.

  • meta-cuteradio defines the Linux image for the Internet radio application.
  • meta-qt5 includes all the metadata files needed to build Qt.
  • meta-raspberrypi contains the BSP (Board Support Package) for all the SoCs (Systems on Chip) of the Raspberry Pi family.
  • poky defines a reference Linux distro as a blueprint for our own distros.
  • meta-openembedded provides the core build system for building embedded Linux systems.

Layers further up in the list (upper layers) depend on layers further down in the list (lower layers) – but never vice versa. The layering above is fairly typical for Qt embedded systems.

The top layer (meta-cuteradio above) defines the Linux image for a concrete product of a company. If the company builds several products on similar Linux images, it will extract the common parts of the product images into a company layer. The company layer is located between the top layer and the meta-qt5 layer.

The next version of our product or another product may use a different SoC. Then, we add the BSP layers for these SoCs after the meta-qt5 layer and before the poky layer.

Each layer resides in its own git repository. We install the repo tool on our development PCs to manage these repositories. The repo tool hides the complexity of git submodules. We install the tool on our development computer with the following commands in a directory included in $PATH (e.g., in $HOME/bin or /usr/local/bin)

$ cd ~/bin
$ wget https://storage.googleapis.com/git-repo-downloads/repo
$ chmod a+x ./repo

The repo tool needs to know from which URL to fetch the layer repositories and into which directory to put them. We define this information in a manifest file. We initialise the umbrella project in the directory /public/Work/cuteradio-thud with the commands

$ cd /public/Work
$ mkdir cuteradio-thud
$ cd cuteradio-thud
$ repo init -u https://github.com/bstubert/cuteradio.git -m thud.xml
...
repo has been initialized in /public/Work/cuteradio-thud

The repo init command clones the repository cuteradio.git into the working directory and selects the manifest thud.xml as the current manifest. The file .repo/manifest.xml reflects our choice of the current manifest.

$ cat .repo/manifest.xml
...
<manifest>
  <include name="thud.xml" />
</manifest>

We find the current manifest file and the other available manifest files in the directory .repo/manifests. The current manifest ./repo/manifests/thud.xml contains two entries for each repository layer.

<remote name="raspberrypi" fetch="https://git.yoctoproject.org/cgit/cgit.cgi"/>
<project name="meta-raspberrypi"
    remote="raspberrypi"
    revision="thud"
    path="sources/meta-raspberrypi"/>

The remote tag defines the name raspberrypi as an alias for the fetch URL https://git.yoctoproject.org/cgit/cgit.cgi. The project tag specifies that the revision thud of the remote raspberrypi shall be cloned into the directory path sources/meta-raspberrypi. The metadata files of the layer meta-raspberrypi will be found in the directory sources/meta-raspberrypi.

We clone the layer repositories into the directory ./sources with the command

$ cd /public/Work/cuteradio-thud
$ repo sync
...
repo sync has finished successfully.

This creates the following directory structure:

/public/Work/cuteradio-thud
    sources
        meta-cuteradio
        meta-openembedded
        meta-qt5
        meta-raspberrypi
        poky

The revision in the project tag can be a branch, tag or commit SHA. Branches and tags can and will change to stay up-to-date with bug and security fixes. Every time we run repo sync, we may get modified metadata files and hence images. This is OK during development. For releases, we need a specific revision that never changes. Hence, we specify the revision by a commit SHA.

When we want to apply an upstream or a local change to the current manifest file, we re-run repo sync. The command first updates the current manifest file with the upstream changes (if any) and then updates the layer repositories.

Configuring a Build Environment

A Yocto build environment is defined by two configuration files: the build environment layer configuration bblayers.conf and the build environment configuration local.conf. The bblayers.conf file contains a list of directory paths where the build process searches for metadata files. Hence, the build process reads bblayers.conf first.

The build process reads the local.conf file last, after parsing all the metadata files in the search paths given in bblayers.conf and after evaluating some more configuration files. The local.conf file is a last resort for defining the target Linux image. It is also a quick and dirty way to get a Linux image up and running on a target device. We’ll clean up the “dirt” in a later part of the series.

Build Environment Layer Configuration: bblayers.conf

The build environment layer configuration bblayers.conf for Cuteradio looks as follows.

POKY_BBLAYERS_CONF_VERSION = "4"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
  ${TOPDIR}/../sources/poky/meta \
  ${TOPDIR}/../sources/poky/meta-poky \
  ${TOPDIR}/../sources/meta-openembedded/meta-oe \
  ${TOPDIR}/../sources/meta-openembedded/meta-python \
  ${TOPDIR}/../sources/meta-raspberrypi \
  ${TOPDIR}/../sources/meta-qt5 \
  ${TOPDIR}/../sources/meta-cuteradio \

The build is only allowed to use metadata files from the directories listed in the bblayers.conf file. If a recipe in the meta-cuteradio directory depends, for example, on a recipe not contained in any other directory, the build will fail. Finding a working and minimal list of layer directories is an iterative process.

Some layers like poky or meta-openembedded are super layers that contain multiple layers. Listing just the directory of a super layer like

${TOPDIR}/../sources/poky

is not enough. We must list the directories of all the sub-layers we are interested in:

  ${TOPDIR}/../sources/poky/meta \
  ${TOPDIR}/../sources/poky/meta-poky \

Build Environment Configuration: local.conf

The build environment configuration local.conf for Cuteradio looks as follows.

MACHINE = "raspberrypi3"
DISTRO = "poky"
PACKAGE_CLASSES = "package_ipk"
SDKMACHINE = "x86_64"
USER_CLASSES = "image-mklibs image-prelink"
PATCHRESOLVE = "noop"

DL_DIR ?= "${TOPDIR}/../downloads"
SSTATE_DIR ?= "${TOPDIR}/../sstate-cache"

DISTRO_FEATURES_remove = " x11 wayland vulkan directfb"
DISTRO_FEATURES_append = " alsa opengl"

LICENSE_FLAGS_WHITELIST = "commercial"

CONF_VERSION = "1"

The build environment is specific to the SoC used. For the Cuteradio project, we set MACHINE to raspberrypi3. We find the available machine configuration files in the directory sources/meta-raspberrypi/conf/machine. Yocto Thud has machine configurations for the following Raspberry Pi SoCs.

raspberrypi0.conf  raspberrypi0-wifi.conf  raspberrypi2.conf    raspberrypi3-64.conf
raspberrypi3.conf  raspberrypi-cm3.conf    raspberrypi-cm.conf  raspberrypi.conf

The machine raspberrypi3 yields a 32-bit Linux image for the Raspberry Pi 3B. The BCM2837 in the Raspberry Pi 3B is a 64-bit processor. I haven’t tried to build for the machine raspberrypi3-64 yet, but it should work as well.

As the default distro is good enough to get started, we set DISTRO to poky. We exclude any X11, Wayland, Vulkan and Directfb features from the Cuteradio image with DISTRO_FEATURES_remove. The “window system” EglFs, which allows a single process to render to a single window, is sufficient for our first steps. As an Internet radio certainly needs sound and benefits from smooth animations, we include the ALSA and OpenGL features with DISTRO_FEATURES_append. It’s good practice to move distro-related configurations to a separate distribution configuration file.

Setting the directory DL_DIR, which holds the downloaded source archives, and the directory SSTATE_DIR, which caches intermediate build results, to a directory outside the build environment will save us a lot of time. ${TOPDIR} points to the build directory (the root directory of the build environment), which is typically a sibling of the sources directory. The directory cuteradio-thud will have the following structure.

/public/Work/cuteradio-thud
    build-thud-rpi3              # ${TOPDIR}
    downloads                    # ${DL_DIR}
    sources
    sstate-cache                 # ${SSTATE_DIR}

We use SysVinit as the system manager, because it is simpler and less resource-hungry than Systemd. SysVinit is the default system manager for Poky distros. I’ll explain the remaining variables later in the series. The curious can look up the remaining variables in the Yocto Project Reference Manual.

Building the Linux Image with Docker

Now, we have all the pieces together to spin up the Docker container, to intialise the build environment and to build the Linux image.

We start the Docker container in the working directory on the development computer and change to the directory cuteradio-thud in the Docker container.

// On development computer
$ cd /public/Work
$ ./dr-yocto/run-shell.sh 18.04

// In Docker container
# cd /public/Work/cuteradio-thud
# ls
sources

We initialise the build environment with the following two commands.

# export TEMPLATECONF=/public/Work/cuteradio-thud/sources/meta-cuteradio/custom
# source ./sources/poky/oe-init-build-env build-rpi3

You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you with some default values. To add additional metadata layers
into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/

# pwd
/public/Work/cuteradio-thud/build-rpi3

When we run the script oe-init-build-env for the first time for the build environment build-rpi3, it copies the sample configuration files bblayers.conf.sample and local.conf.sample from the directory $TEMPLATECONF to build-rpi3/conf/bblayers.conf and build-rpi3/conf/local.conf, respectively. The message tells us about this. The script changes the working directory to /public/Work/cuteradio-thud/build-rpi3 and sets some environment variables like BBPATH and BUILDDIR.

When we run the script oe-init-build-env the next times, we need not set TEMPLATECONF and we won’t see the messages about copying the configuration files. The script enters the already existing build environment silently.

It’s time for the big moment. We start the build of our custom Linux image with BitBake. BitBake is Yocto’s build engine. It takes the metadata files (recipes, configurations, etc.) as input. It figures out which tasks must be re-run for each recipe and schedules these tasks for highly parallel execution. The tasks for building a recipe including fetching the source code, patching the source code, configuring the build, running the build, installing the relevant files of a package (e.g., executables, libraries) and packaging the installed files in formats like ipk, deb and rpm.

# cd /public/Work/cuteradio-thud/build-rpi3
# bitbake cuteradio-image

WARNING: Layer cuteradio should set LAYERSERIES_COMPAT_cuteradio in its conf/layer.conf file to list the core layer names it is compatible with.
NOTE: Your conf/bblayers.conf has been automatically updated.
...
Parsing recipes: 100% |######| Time: 0:00:19
Parsing of 1996 .bb files complete (0 cached, 1996 parsed). 2936 targets, 310 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.40.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "ubuntu-18.04"
TARGET_SYS           = "arm-poky-linux-gnueabi"
MACHINE              = "raspberrypi3"
DISTRO               = "poky"
DISTRO_VERSION       = "2.6.4"
TUNE_FEATURES        = "arm armv7ve vfp thumb neon vfpv4 callconvention-hard cortexa7"
TARGET_FPU           = "hard"
meta                 
meta-poky            = "HEAD:958427e9d2ee7276887f2b02ba85cf0996dea553"
meta-oe              
meta-python          = "HEAD:446bd615fd7cb9bc7a159fe5c2019ed08d1a7a93"
meta-raspberrypi     = "HEAD:4e5be97d75668804694412f9b86e9291edb38b9d"
meta-qt5             = "HEAD:e6e464c9ed9266ce46452f953c1bdcb0e7b2d95f"
meta-cuteradio       = "HEAD:178e4af9e6bf6b45a69cfb6fe040ccd8c7b650d3"

Initialising tasks: 100% |######| Time: 0:00:03
Sstate summary: Wanted 1328 Found 0 Missed 0 Current 0 (0% match, 0% complete)
...
NOTE: Tasks Summary: Attempted 3773 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There were 2 WARNING messages shown.

The first BitBake run takes a couple of hours, as it must download the source code for hundreds of packages and execute nearly 4000 tasks. Bitbake shows progress information about the tasks it is working on. The next BitBake runs will be a lot faster, as BitBake performs incremental builds.

Running the Linux Image on a Raspberry Pi

Back on our development computer, we locate the Cuteradio image for installation on an SD card.

$ cd /public/Work/cuteradio-thud/build-rpi3
$ cd tmp/deploy/images/raspberrypi3
$ ls -1 *sdimg
cuteradio-image-raspberrypi3-20200524105934.rootfs.rpi-sdimg
cuteradio-image-raspberrypi3.rpi-sdimg

The first file is the Cuteradio Linux image, is time-stamped and has a size of 784 MB. The second file is a symbolic link pointing to the first file. The SD card, to which we write the image, should have a size of at least 1 GB.

We run

$ df
/dev/sda1       163633596  56431420   98866972  37% /
/dev/sda3       230548636     60624  218753800   1% /extra
/dev/sdb2      3356101776 675921260 2509677248  22% /public
/dev/sdb1      4368376688 433924352 3714275952  11% /private
...

to list the file systems available on our computer. We plug a USB adapter with the SD card into the computer and list the file systems again.

$ df
/dev/sda1       163633596  56431420   98866972  37% /
/dev/sda3       230548636     60624  218753800   1% /extra
/dev/sdb2      3356101776 675921260 2509677248  22% /public
/dev/sdb1      4368376688 433924352 3714275952  11% /private
...
/dev/sdd1           40314     17405      22910  44% /media/burkhard/raspberrypi
/dev/sdd2          693920    181504     474528  28% /media/burkhard/2b07815c...21f8

The new file systems – /dev/sdd1 and /dev/sdd2 on my computer – are the device files of the SD card. We would see two device files, if we wrote a Cuteradio image to the SD card before. A fresh SD card has only one device file. The names of the device files vary from computer to computer.

Caution: It is crucial to figure out the correct name of the SD card. Writing the Linux image to the device file will delete all the data on this device – past any recovery.

We unmount the device files of the SD card.

$ sudo umount /dev/sdd1
$ sudo umount /dev/sdd2

We write the Cuteradio image to the SD card.

$ sudo dd if=cuteradio-image-raspberrypi3.rpi-sdimg of=/dev/sdd bs=1M status=progress

We remove the SD card from the USB adapter and plug it into the SD-card slot of the Raspberry Pi. We power on the Raspberry Pi. After a couple of seconds, the Internet radio application starts automatically and plays the hard-wired radio station Antenne Bayern.

SSH Login to the Raspberry Pi

We can log in to the Raspberry Pi with SSH. Here are two ways to figure out the IP address of the Raspberry Pi.

First, we can log in to the router of the LAN that contains the Raspberry Pi and display a list of all devices in the LAN. The Raspberry Pi is the new device without a name. My device got the IP address 192.168.1.84.

Second, we can check the system messages when the Raspberry Pi powers up. We look for a message like

udhcp:  sending select for 192.168.1.84

The challenge is to spot the IP address before the Internet radio is displayed.

Once we know the IP address, we log in to the Raspberry Pi with this command:

$ ssh root@192.168.1.84
root@raspberrypi3:~# uname -a
Linux raspberrypi3 4.14.112 #1 SMP Sat May 16 16:52:12 UTC 2020 armv7l GNU/Linux
root@raspberrypi3:~#

About the Series

This post is part of a series on Qt Embedded Systems. I plan to write one post per month. The goal is to build a full-blown Internet radio running on a custom embedded Linux system powered by a Raspberry Pi. I’ll walk you through all the steps needed to build a product. Topics will include QML, Qt, C++, Wayland, Wifi, Bluetooth, Yocto, fast start-up, OTA updates, etc.

So far in the Series:

I have plenty of ideas for the next posts:

  • We learn how to write the recipes for the meta-cuteradio layer and clean up the existing recipes.
  • We extract reusable parts of meta-cuteradio into a separate layer. The new layer provides several base Linux images, on which products like Cuteradio can build.
  • We use the Linux package manager to update the packages on the target devices.
  • We upgrade the Linux system from Yocto Thud to Yocto Danfell.
  • The radio uses Wifi for the Internet connection and Bluetooth for the speaker.
  • The radio supports multiple radio stations from different categories.
  • We implement a Wayland-based window and application manager to accomodate multiple applications like alarm clock, image viewer and settings.
  • And more…

4 thoughts on “Qt Embedded Systems – Part 1: Building a Linux Image with Yocto”

  1. Thanks for nice post. For building yocto in docker I’m using kas tool which is wrapper around bitbake to simplify yocto setup and it provide rebuild docker images for building yocto.

      1. Yes. I really like it and yml file is easily readable + you can override also which is great. Thanks.

  2. Hi,
    great post. I am currently working on building an image for an Qt application myself.
    I had issues recently choosing the correct image to build. Yocto provides several out-of-the-box images (core-image-minimal, core-image-weston). However, core-image-minimal did not seem to work, since a lot of necessary features were missing. Currently, I am building an image with the core-image-weston image, hoping, it will be able to display the Qt widget app on my RasPi 4B and the screen (very similar to yours).
    However, you are using the command “bitbake cute-radio-image”, so you are creating a custom image, with no or little overhead software. How exactly did you specify, which windowing system is to be baked into the image, and how did you manage to let the radio app start right after booting?
    Thanks and sorry for the long question.
    Bjørn

Leave a Reply to Marek Cancel reply

Your email address will not be published. Required fields are marked *