After having built the reference Linux image from a SoM, SoC or terminal maker and having run it on the board, we must inevitably custom-tailor this image to our needs. We must create our own Yocto layer. We must remove all the unnecessary packages and make our core application start automatically on power-up. Here is a step-by-step guide how to turn the application layer for a Toradex Verdin iMX8M Plus board into our own custom layer. The guide should also work for other boards.
Motivation
The reference images from SoM and SoC manufacturers come with some undesirable features:
- They include many packages that we don’t need or they lack packages that we need.
- They auto-start demo web or Qt applications – known as “bloatware” on PCs. For example, some reference images start the Qt demo launcher with dozens of Qt demo applications. Others start a web browser running some web apps.
- They provide libraries under licenses with unwanted consequences. For example, Qt Virtual Keyboard and Qt Quick 3D must either be used under GPL or under a commercial license. These licenses force us either to publish the source code of our proprietary applications or to pay for Qt.
- The build of the reference image fails. The fix would require us to change an existing recipe.
Once we have successfully built the reference image and run it on the board, we will inevitably create a custom layer for our product with a custom image recipe and recipes for our proprietary applications. The custom image will contain only the necessary packages and will auto-start our core application.
My reference platform is the Toradex Verdin iMX8M Plus. The post Setting Up Yocto Projects with kas describes how to build the reference image and the post Installing Linux Images on Toradex Verdin Boards how to install the image on the board. The goal is to replace the layer meta-toradex-demos, which defines the reference image, with our own layer meta-embeddeduse.
How to Follow Along
We can follow along by cloning the repository embedded-linux-distros into a directory of our choice (for me: /public) and check out the commit 2a8a24ef.
$ cd /public
$ git clone git@github.com:bstubert/embedded-linux-distros.git
$ git checkout 2a8a24efWe build the reference image for the Toradex Verdin iMX8M Plus with the following commands – assuming that kas-container is in PATH (see Setting Up Yocto Projects with kas for more information how to work with kas).
$ cd /public/embedded-linux-distros/yocto-3.1
# Repeat checkout until all layers downloaded without errors
$ kas-container checkout configurations/toradex/verdin-imx8mp-5.7.1.yml
$ kas-container build configurations/toradex/verdin-imx8mp-5.7.1.ymlThe reference image for installation with TEZI can be found in the directory
build/tmp/deploy/images/verdin-imx8mp/
Verdin-iMX8MP_Reference-Multimedia-Image-Tezi_5.7.1-devel-20221102194433+build.0.tarThe full build of the Toradex reference image will take a couple of hours depending on the power of your build server. Once we have a full build and know that it’s running fine on the board, we run incremental builds of 1-5 minutes and try out the results on the board.
The fast incremental builds come in handy when we create our own custom layer meta-embeddeduse step by step. Of course, we want all our changes under version control so that we always have a fallback version when things go wrong. We store the custom layer in a repository with the same name and push our changes regularly to the repository.
For the remainder of the post, we can make our lives a little bit easier and check out a given commit from the repository. Let us get started with an almost empty layer directory containing only README.md.
$ cd yocto-3.1/layers
$ git clone git@github.com:bstubert/meta-embeddeduse.git
$ git checkout 5e40f90cCreating a Custom Layer
We will create a custom Yocto layer called meta-embeddeduse. We do this interactively with bitbake and some auxiliary tools in Yocto’s build environment. Hence, we enter the kas shell with
$ kas-container shell configurations/toradex/verdin-imx8mp-5.7.1.yml
builder@830bfe5ec31f:/build$We create the new layer meta-embeddeduse with the command
@ bitbake-layers create-layer --priority 25 /work/layers/meta-embeddeduse
Specified layer directory existsThe command fails, because the layer directory exists already. As there is no way to force the layer creation, we apply a workaround. We create the layer in /work and copy its contents to /work/layers/meta-embeddeduse.
@ bitbake-layers create-layer --priority 25 /work/meta-embeddeduse
@ cp -r /work/meta-embeddeduse/* /work/layers/meta-embeddeduse
@ rm -r /work/meta-embeddeduseThis time everything works fine.
Running bitbake-layers show-layers lists all layers with their paths and priorities and shows why we chose 25 as the priority. The meta-toradex-* layers distro, nxp, bsp-common and demos have priorities 21, 22, 23 and 24, respectively. The new layer should not have a priority higher than 24 (that is, a number lesser than or equal to 24), because it depends on the first three layers and replaces the fourth layer demos. Hence, priority 25 is a proper choice.
The above commands created the following files:
/work/layers/meta-embeddeduse/conf/layer.conf
/work/layers/meta-embeddeduse/README
/work/layers/meta-embeddeduse/recipes-example/example/example_0.1.bb
/work/layers/meta-embeddeduse/COPYING.MITBy default, layers are under the MIT license, which is OK in most cases. We will replace the recipe file example_0.1.bb by one for our core application. So, we can remove the directory meta-embeddeduse/recipes-example. We also emove README, as we already have README.md.
@ rm -r /work/layers/meta-embeddeduse/recipes-example/ /work/layers/meta-embeddeduse/READMEThe only important file is the layer configuration file conf/layer.conf. It defines where bitbake searches for classes (BBPATH), configuration files (BBPATH) and recipes (BBFILES), on which layers the current layer depends (LAYERDEPENDS), to which Yocto versions the current layer is compatible (LAYERSERIES_COMPAT) and some more.
The configuration files for the layers meta-toradex-demos and meta-embeddeduse differ clearly in the value of LAYERDEPENDS.
# meta-toradex-demos
LAYERDEPENDS_toradex-demos = " \
core yocto openembedded-layer gnome-layer multimedia-layer \
networking-layer freescale-layer freescale-distro \
"
# meta-embeddeduse
LAYERDEPENDS_meta-embeddeduse = "core"The layer names do not necessarily match the directory names of the layers. They are “set” by appending them to BBFILE_COLLECTIONS. We use the same dependencies for meta-embeddeduse as for meta-toradex-demos and add the missing layer qt5-layer.
When we run bitbake-layers show-layers, our new layer is not listed. This isn’t surprising, because we haven’t added meta-embeddeduse to bblayers.conf yet. Kas generates bblayers.conf from its configuration file verdin-imx8mp-5.7.1.yml, which doesn’t contain our new layer yet. We exit the kas shell.
@ exitWe add the layer meta-embeddeduse to the kas configuration file:
# configurations/toradex/verdin-imx8mp-5.7.1.yml (Commit: d6a29de1)
repos:
...
meta-embeddeduse:
path: "layers/meta-embeddeduse"The new entry tells kas to use the recipes and other files found in the directory layers/meta-embeddeduse. Kas picks up our local changes from the layer directory meta-embeddeduse instead of pulling down a given commit from the corresponding directory. This is very convenient while developing a layer. When we enter the kas shell again, kas will add the new layer to conf/bblayers.conf. The new layer will also show up in the list of layers.
$ kas-container shell configurations/toradex/verdin-imx8mp-5.7.1.yml
@ cat conf/bblayers.conf
# custom-bblayers-conf
...
BBLAYERS ?= " \
/work/layers/meta-embeddeduse \
/work/layers/meta-freescale \
...
@ bitbake-layers show-layers
layer path priority
==========================================================================
meta-embeddeduse /work/layers/meta-embeddeduse 25
...The Yocto build warns that the layer meta-embeddeduse doesn’t contain any recipes but runs successfully otherwise.
@ bitbake tdx-reference-multimedia-image
Loading cache: 100% |##############| Time: 0:00:01
Loaded 3905 entries from dependency cache.
WARNING: No bb files matched BBFILE_PATTERN_meta-embeddeduse '^/work/layers/meta-embeddeduse/'
...
Build Configuration:
...
meta-embeddeduse = "<unknown>:<unknown>"
meta-freescale = "HEAD:3cb29cff92568ea835ef070490f185349d712837"
...
NOTE: Tasks Summary: Attempted 7939 tasks of which 7930 didn't need to be rerun and all succeeded.
...
Summary: There was 1 WARNING message shown.Following along: We check out commit 49e18e18 from repository meta-embeddeduse and commit d6a29de1 from repository embedded-linux-distros.
Creating a Custom Image Recipe
We replace the layer meta-toradex-demos by our layer meta-embeddeduse. We start by removing the entry for meta-toradex-demos from the kas configuration file. As we removed the image recipe tdx-reference-multimedia-image.bb from the build, the kas build fails.
$ kas-container build configurations/toradex/verdin-imx8mp-5.7.1.yml
...
ERROR: Nothing PROVIDES 'tdx-reference-multimedia-image'. Close matches:
meta-multimedia-imageWe copy the image recipe and the other recipes in the same directory to our layer, while keeping the directory structure.
$ mkdir -p layers/meta-embeddeduse/recipes-images/images
$ cp -r layers/meta-toradex-demos/recipes-images/images/* layers/meta-embeddeduse/recipes-images/images
# Contents of directory layers/meta-embeddeduse/recipes-images/images:
packagegroup-tdx-cli.bb
packagegroup-tdx-graphical.bb
packagegroup-tdx-qt5.bb
tdx-reference-minimal-image.bb
tdx-reference-multimedia-image.bbWe rename the image recipe into emuse-qt-lgpl-image.bb.
$ cd layers/meta-embeddeduse/recipes-images/images
$ mv tdx-reference-multimedia-image.bb emuse-qt-lgpl-image.bb
$ cd -We change the SUMMARY, DESCRIPTION and IMAGE_BASENAME in the image recipe emuse-smart-hmi-image.bb. We set the variables APP_LAUNCH_WAYLAND, APP_LAUNCH_X11 and its machine-dependent overrides to the empty string. This saves us from building the Qt demos and ensures that no application is auto-started on power-up. So, we can see the difference between our image and the original Toradex image.
# meta-embeddeduse/recipes-images/images/emuse-smart-hmi-image.bb
require tdx-reference-minimal-image.bb
SUMMARY = "Embedded Linux Qt LGPL Image"
DESCRIPTION = "Embedded Linux image with Qt LGPLv3 for smart HMIs"
inherit populate_sdk_qt5
#Prefix to the resulting deployable tarball name
export IMAGE_BASENAME = "Smart-HMI-Image"
...
APP_LAUNCH_WAYLAND ?= ""
APP_LAUNCH_X11 ?= ""We must change the target in the kas configuration from tdx-reference-multimedia-image to emuse-smart-hmi-image. When we run the build, it fails with this error message:
ERROR: Nothing RPROVIDES 'timestamp-service' (but /work/layers/meta-embeddeduse/recipes-images/images/emuse-smart-hmi-image.bb RDEPENDS on or otherwise requires it)The build cannot find the recipe for timestamp-service, which we find in meta-toradex-demos/recipes-core/systemd. Looking around in the directory reveals that systemd_%.bbappend installs the RNDIS service. As the Toradex Easy Installer (TEZI) needs RNDIS, we include the service recipe in our image. We copy the complete directory into our layer.
$ mkdir -p layers/meta-embeddeduse/recipes-core/systemd
$ cp -r layers/meta-toradex-demos/recipes-core/systemd/* layers/meta-embeddeduse/recipes-core/systemdThe last error is gone and replaced by a new one. The build cannot find the recipe for gpicview, a “simple image viewer for X”. We remove gpicview from the image recipe, as we hardly need it in a product.
We see the same error for different packages a several more times. Each time, we must decide whether to copy the recipe file and its auxiliary files to the layer meta-embeddeduse or whether to remove the package from the image recipe or package-group recipes. We can always change our decisions later.
- We copy the recipes and their auxiliary files for the packages
mmc-utils-cosandtinycompress. - We remove the packages
media-files,packagegroup-dotnet-deps,cpuburn-a53,serial-test,hostapd-example,tdx-oak-sensors,bluez-alsaandclpeakfrom the image or package-group recipes.
Following along: We check out commit f34c4372 from repository meta-embeddeduse and commit ba0103a8 from repository embedded-linux-distros.
The build finishes without errors and produces our first custom image
build/tmp/deploy/images/verdin-imx8mp/Verdin-iMX8MP_Smart-HMI-Image-Tezi_5.7.1-devel-20221105154804+build.0.tarWhen we follow the installation procedure, TEZI shows the image first in the list identified by its SUMMARY: “Embedded Linux Qt LGPL Image”. After installing this image and rebooting the board, the image starts the Weston window manager. As expected, it doesn’t start the Qt cinematic demo application any more.
Right: Image starts Weston window manager but not Qt demo app.
Creating a Recipe for the Core Application
As an example core application, we use the Internet radio from my post Qt Embedded Systems – Part 1: Building a Linux Image with Yocto. The application is written in Qt 5, which matches the Qt 5.14 provided by our Yocto-3.1-based Linux image. We add the recipe recipes-apps/radio/cuteradio_1.0.bb to the layer meta-embeddeduse.
SUMMARY = "CuteRadio - Simple Internet radio for showing QML and Qt on embedded systems"
AUTHOR = "Burkhard Stubert (burkhard.stubert@embeddeduse.com)"
HOMEPAGE = "https://github.com/bstubert/cuteradio-apps"
SECTION = "app"
LICENSE = "MIT"
LIC_FILES_CHKSUM="file://LICENSE;md5=1ef68d7e526c164e11da8965fdbed52c"
inherit cmake_qt5
SRC_URI = "\
git://github.com/bstubert/cuteradio-apps.git;branch=master;protocol=https \
"
SRCREV = "${AUTOREV}"
PV = "1.0+git${SRCREV}"
S = "${WORKDIR}/git"
DEPENDS = "qtmultimedia"By inheriting the class cmake_qt5, we tell Yocto to use CMake to build the Qt5 Internet radio application cuteradio. The build donwloads the git repository from SRC_URI and checks out the latest version SRCREV = "${AUTOREV}". When we release the embedded system to customers, we’ll set SRCREV to a certain commit.
The CMakeLists.txt file of the cuteradio application must explicitly install the executable, libraries and other necessary files. For cuteradio, it only installs the executable to /usr/bin. The documentation for CMake install explains how to install all kinds of files in directories of your choice.
install(
TARGETS ${PROJECT_NAME}
RUNTIME
DESTINATION bin
COMPONENT runtime
)We add the package cuteradio to IMAGE_INSTALL in the image recipe emuse-smart-hmi-image.bb.
IMAGE_INSTALL += " \
...
v4l-utils \
\
cuteradio \
"Building the image and installing it on the board work fine. We can open a Wayland Terminal and start the Internet radio with
# /usr/bin/cuteradio &The application starts but doesn’t show the radio station “Antenne Bayern”. The fonts are missing, as we can see from the error messages in the terminal.
We choose the second suggestion of the message and switch to fontconfig. In the recipe extension recipes-qt/qt5/qtbase_git.bbappend, we enable fontconfig support in the Qt build.
# Enable fontconfig to get system freetype fonts
PACKAGECONFIG_FONTS += "fontconfig"When started in the new image, the Internet radio shows the radio station “Antenne Bayern” and plays it. The headset is connect to the green socket X20 in the top right corner of the board.
Following along: We check out commit 92b357eb from repository meta-embeddeduse and commit 995a0c94 from repository embedded-linux-distros.
Auto-Starting the Core Application
The Linux system for the Toradex Verdin boards uses systemd to start services and applications after power-up. After starting the weston service, systemd starts the wayland-app-launch service. The order is defined in the template service file wayland-app-launch.service.in:
[Unit]
Description=Start a wayland application
After=weston@root.service
Requires=weston@root.service
[Service]
Restart=on-failure
Type=forking
Environment=@@application-environment@@
ExecStart=/usr/bin/wayland-app-launch.sh
RestartSec=1
[Install]
WantedBy=multi-user.targetThe template script wayland-app-launch.sh.in for launching a Wayland application looks like this:
...
# wait for weston
while [ ! -e $XDG_RUNTIME_DIR/wayland-0 ] ; do sleep 0.1; done
sleep 1
cd @@initial-path@@
@@wayland-application@@ &The script checks every 0.1 seconds whether weston has already created the file $XDG_RUNTIME_DIR/wayland-0. Once this file exists, the script changes to the directory @@initial-path@@ and starts the application @@wayland-application@@ in the background.
The placeholders of the launch and service scripts are defined in the Wayland launch recipes like wayland-qtdemo-launch-cinematicexperience_1.0.bb for the Qt demo started by default. The included file wayland-app-launch.inc replaces the placeholders by the concrete values defined in Wayland launch recipe.
All four files just mentioned are located in the directory
meta-toradex-demos/recipes-graphics/wayland-app-launch/We copy the complete directory to the layer meta-embeddeduse, adapt files as necessary and remove files not needed.
$ mkdir -p layers/meta-embeddeduse/recipes-graphics/wayland-app-launch
$ cp -r layers/meta-toradex-demos/recipes-graphics/wayland-app-launch/* layers/meta-embeddeduse/recipes-graphics/wayland-app-launch/
$ cd layers/meta-embeddeduse/recipes-graphics/wayland-app-launch/
$ rm wayland-qtdemo-launch-analogclock_1.0.bb wayland-qtdemo-launch-qtsmarthome_1.0.bb
$ mv wayland-qtdemo-launch-cinematicexperience_1.0.bb wayland-cuteradio_1.0.bb
# Change wayland-cuteradio_1.0.bb (see below how)
$ cd -The finished recipe wayland-cuteradio_1.0.bb looks as follows:
# set the following variable to your one and only application which should
# be launched right after weston started
INITIAL_APP_PKGS ?= "cuteradio"
INITIAL_PATH ?= ""
APPLICATION_ENVIRONMENT ?= '\"QT_QPA_PLATFORM=wayland-egl\"'
WAYLAND_APPLICATION ?= "/usr/bin/cuteradio"
require wayland-app-launch.incIf we forget to set the variable QT_QPA_PLATFORM, the wayland-app-launch service will fail to start cuteradio. We don’t have to set this variable, when we start cuteradio from the Wayland Terminal. It seems that the execution environment for starting systemd services differs from starting applications from the Wayland Terminal.
In the image recipe emuse-smart-hmi-image.bb, we set the the variable APP_LAUNCH_WAYLAND to wayland-cuteradio and add debug-tweaks to IMAGE_FEATURES. This adds the package wayland-cuteradio to the image and enables us to log in to the board with ssh without a password. The ssh login makes it easier to find problems like the missing QT_QPA_PLATFORM setting.
IMAGE_FEATURES += " \
debug-tweaks \
...
"
APP_LAUNCH_WAYLAND ?= "wayland-cuteradio"On power-up, the Linux image automatically starts the cuteradio application and plays the radio station Antenne Bayern.
Following along: We check out commit a2a9481e from repository meta-embeddeduse and commit 995a0c94 from repository embedded-linux-distros.
Cleaning Up
The description of our image says “Embedded Linux image with Qt LGPLv3 for smart HMIs”. So, we should remove the Qt packages under commercial Qt licenses from packagegroup-tdx-qt5.bb (se Using Qt 5.15 and Qt 6 under LGPLv3 for more details about Qt licensing).
- We remove these packages under the Qt for Device Creation license:
qtnetworkauth,qtquick3d,qtvirtualkeyboard. - We remove these packages under the Qt Marketplace license:
qtcharts,qtcoap,qtdatavis3d,qtknx,qtmqtt,qtopcua. - We remove a couple of discontinued or unnecessary packages:
qt5ledscreen,qtgamepad,qtlottie,qtpurchasing,qtquickcontrols,qtscript,qtwebglplugin. - We remove all the demo and example packages:
cinematicexperience,qtsmarthome,qtbase-examples.
Our custom image builds and works fine on the board.
Following along: We check out commit a92965a8 from repository meta-embeddeduse and commit 995a0c94 from repository embedded-linux-distros.
We should probably clean up a little bit more.
- We should organise the recipes into package groups that may only used during development and package groups used in production.
- Our custom layer should work with the BSP from any terminal, SoM and SoC manufacturer not just with the BSP from Toradex.
- We should be able to generate the recipes of our custom layer with the correct links to our core application.
- We want to build our custom layer with Yocto 4.0 or 4.1 and with Qt 6.
These are good topics for future posts.