Mastering Embedded Linux Programming: Create fast and reliable embedded solutions with Linux 5.4 and the Yocto Project 3.1 (Dunfell)

Cover
Copyright
Contributors
Table of Contents
Preface
Section 1: Elements of Embedded Linux
Chapter 1: Starting Out
Choosing Linux
When not to choose Linux
Meeting the players
Moving through the project life cycle
The four elements of embedded Linux
Navigating open source
Licenses
Selecting hardware for embedded Linux
Obtaining the hardware for this book
The Raspberry Pi 4
The BeagleBone Black
QEMU
Provisioning your development environment
Summary
Chapter 2: Learning about Toolchains
Technical requirements
Introducing toolchains
Types of toolchains
CPU architectures
Choosing the C library
Finding a toolchain
Building a toolchain using crosstool-NG
Installing crosstool-NG
Building a toolchain for BeagleBone Black
Building a toolchain for QEMU
Anatomy of a toolchain
Finding out about your cross compiler
The sysroot, library, and header files
Other tools in the toolchain
Looking at the components of the C library
Linking with libraries – static and dynamic linking
Static libraries
Shared libraries
The art of cross-compiling
Simple makefiles
Autotools
Package configuration
Problems with cross-compiling
CMake
Summary
Further reading
Chapter 3: All about Bootloaders
Technical requirements
What does a bootloader do?
The boot sequence
Phase 1 – ROM code
Phase 2 – secondary program loader
Phase 3 – TPL
Moving from the bootloader to a kernel
Introducing device trees
Device tree basics
The reg property
Labels and interrupts
Device tree include files
Compiling a device tree
U-Boot
Building U-Boot
Installing U-Boot
Using U-Boot
Booting Linux
Porting U-Boot to a new board
Building and testing
Falcon mode
Summary
Chapter 4: Configuring and Building the Kernel
Technical requirements
What does the kernel do?
Choosing a kernel
Kernel development cycle
Stable and long-term support releases
Building the kernel
Getting the source
Understanding kernel configuration – Kconfig
Using LOCALVERSION to identify your kernel
When to use kernel modules
Compiling – Kbuild
Finding out which kernel target to build
Build artifacts
Compiling device trees
Compiling modules
Cleaning kernel sources
Building a 64-bit kernel for the Raspberry Pi 4
Building a kernel for the BeagleBone Black
Building a kernel for QEMU
Booting the kernel
Booting the Raspberry Pi 4
Booting the BeagleBone Black
Booting QEMU
Kernel panic
Early user space
Kernel messages
The kernel command line
Porting Linux to a new board
A new device tree
Setting the board's compatible property
Summary
Additional reading
Chapter 5: Building a Root Filesystem
Technical requirements
What should be in the root filesystem?
The directory layout
The staging directory
POSIX file access permissions
File ownership permissions in the staging directory
Programs for the root filesystem
Libraries for the root filesystem
Device nodes
The proc and sysfs filesystems
Kernel modules
Transferring the root filesystem to the target
Creating a boot initramfs
Standalone initramfs
Booting the initramfs
Booting with QEMU
Booting the BeagleBone Black
Building an initramfs into the kernel image
Building an initramfs using a device table
The old initrd format
The init program
Starting a daemon process
Configuring user accounts
Adding user accounts to the root filesystem
A better way of managing device nodes
An example using devtmpfs
An example using mdev
Are static device nodes so bad after all?
Configuring the network
Network components for glibc
Creating filesystem images with device tables
Booting the BeagleBone Black
Mounting the root filesystem using NFS
Testing with QEMU
Testing with the BeagleBone Black
Problems with file permissions
Using TFTP to load the kernel
Summary
Further reading
Chapter 6: Selecting a Build System
Technical requirements
Comparing build systems
Distributing binaries
Introducing Buildroot
Background
Stable releases and long-term support
Installing
Configuring
Running
Targeting real hardware
Creating a custom BSP
Adding your own code
License compliance
Introducing the Yocto Project
Background
Stable releases and supports
Installing the Yocto Project
Configuring
Building
Running the QEMU target
Layers
Customizing images via local.conf
Writing an image recipe
Creating an SDK
The license audit
Summary
Further reading
Chapter 7: Developing with Yocto
Technical requirements
Building on top of an existing BSP
Building an existing BSP
Controlling Wi-Fi
Controlling Bluetooth
Adding a custom layer
Capturing changes with devtool
Development workflows
Creating a new recipe
Modifying the source built by a recipe
Upgrading a recipe to a newer version
Building your own distro
When and when not to
Creating a new distro layer
Configuring your distro
Adding more recipes to your distro
Runtime package management
Provisioning a remote package server
Summary
Further reading
Chapter 8: Yocto Under the Hood
Technical requirements
Decomposing Yocto's architecture and workflow
Metadata
Build tasks
Image generation
Separating metadata into layers
Troubleshooting build failures
Isolating errors
Dumping the environment
Reading the task log
Adding more logging
Running commands from devshell
Graphing dependencies
Understanding BitBake syntax and semantics
Tasks
Dependencies
Variables
Functions
RDEPENDS revisited
Summary
Further reading
Section 2: System Architecture and Design Decisions
Chapter 9: Creating a Storage Strategy
Technical requirements
Storage options
NOR flash
NAND flash
Accessing flash memory from the bootloader
U-Boot and NOR flash
U-Boot and NAND flash
U-Boot and MMC, SD, and eMMC
Accessing flash memory from Linux
Memory technology devices
The MMC block driver
Filesystems for flash memory
Flash translation layers
Filesystems for NOR and NAND flash memory
JFFS2
YAFFS2
UBI and UBIFS
Filesystems for managed flash
Flashbench
Discard and TRIM
Ext4
F2FS
FAT16/32
Read-only compressed filesystems
SquashFS
Temporary filesystems
Making the root filesystem read-only
Filesystem choices
Summary
Further reading
Chapter 10: Updating Software in the Field
Technical requirements
From where do updates originate?
What to update
Bootloader
Kernel
Root filesystem
System applications
Device-specific data
Components that need to be updated
The basics of software updates
Making updates robust
Making updates fail-safe
Making updates secure
Types of update mechanism
Symmetric image update
Asymmetric image update
Atomic file updates
OTA updates
Using Mender for local updates
Building the Mender client
Installing an update
Using Mender for OTA updates
Using balena for local updates
Creating an account
Creating an application
Adding a device
Installing the CLI
Pushing a project
Summary
Chapter 11: Interfacing with Device Drivers
Technical requirements
The role of device drivers
Character devices
Block devices
Network devices
Finding out about drivers at runtime
Getting information from sysfs
Finding the right device driver
Device drivers in user space
GPIO
LEDs
I2C
SPI
Writing a kernel device driver
Designing a character driver interface
The anatomy of a device driver
Compiling kernel modules
Loading kernel modules
Discovering the hardware configuration
Device trees
The platform data
Linking hardware with device drivers
Summary
Further reading
Chapter 12: Prototyping with Breakout Boards
Technical requirements
Mapping schematics to the device tree's source
Reading schematics and data sheets
Installing Debian on the BeagleBone Black
Enabling spidev
Customizing the device tree
Prototyping with breakout boards
Closing the SPI jumper
Attaching the GNSS antenna
Attaching the SPI header
Connecting the SPI jumper wires
Probing SPI signals with a logic analyzer
Receiving NMEA messages over SPI
Summary
Further reading
Chapter 13: Starting Up – The init Program
Technical requirements
After the kernel has booted
Introducing the init programs
BusyBox init
Buildroot init scripts
System V init
inittab
The init.d scripts
Adding a new daemon
Starting and stopping services
systemd
Building systemd with the Yocto Project and Buildroot
Introducing targets, services, and units
How systemd boots the system
Adding your own service
Adding a watchdog
Implications for embedded Linux
Summary
Further reading
Chapter 14: Starting with BusyBox runit
Technical requirements
Getting BusyBox runit
Creating service directories and files
Service directory layout
Service configuration
Service supervision
Controlling services
Depending on other services
Start dependencies
Custom start dependencies
Putting it all together
Dedicated service logging
How does it work?
Adding dedicated logging to a service
Log rotation
Signaling a service
Summary
Further reading
Chapter 15: Managing Power
Technical requirements
Measuring power usage
Scaling the clock frequency
The CPUFreq driver
Using CPUFreq
Selecting the best idle state
The CPUIdle driver
Tickless operation
Powering down peripherals
Putting the system to sleep
Power states
Wakeup events
Timed wakeups from the real-time clock
Summary
Further reading
Section 3: Writing Embedded Applications
Chapter 16: Packaging Python
Technical requirements
Getting Docker
Retracing the origins of Python packaging
distutils
setuptools
setup.py
Installing Python packages with pip
requirements.txt
Managing Python virtual environments with venv
Installing precompiled binaries with conda
Environment management
Package management
Deploying Python applications with Docker
The anatomy of a Dockerfile
Building a Docker image
Running a Docker image
Fetching a Docker image
Publishing a Docker image
Cleaning up
Summary
Further reading
Chapter 17: Learning about Processes and Threads
Technical requirements
Process or thread?
Processes
Creating a new process
Terminating a process
Running a different program
Daemons
Inter-process communication
Threads
Creating a new thread
Terminating a thread
Compiling a program with threads
Inter-thread communication
Mutual exclusion
Changing conditions
Partitioning the problem
ZeroMQ
Getting pyzmq
Messaging between processes
Messaging within processes
Scheduling
Fairness versus determinism
Time-shared policies
Real-time policies
Choosing a policy
Choosing a real-time priority
Summary
Further reading
Chapter 18: Managing Memory
Technical requirements
Virtual memory basics
Kernel space memory layout
How much memory does the kernel use?
User space memory layout
The process memory map
Swapping
Swapping to compressed memory (zram)
Mapping memory with mmap
Using mmap to allocate private memory
Using mmap to share memory
Using mmap to access device memory
How much memory does my application use?
Per-process memory usage
Using top and ps
Using smem
Other tools to consider
Identifying memory leaks
mtrace
Valgrind
Running out of memory
Summary
Further reading
Section 4: Debugging and Optimizing Performance
Chapter 19: Debugging with GDB
Technical requirements
The GNU debugger
Preparing to debug
Debugging applications
Remote debugging using gdbserver
Setting up the Yocto Project for remote debugging
Setting up Buildroot for remote debugging
Starting to debug
Native debugging
Just-in-time debugging
Debugging forks and threads
Core files
Using GDB to look at core files
GDB user interfaces
Terminal User Interface
Data Display Debugger
Visual Studio Code
Debugging kernel code
Debugging kernel code with kgdb
A sample debug session
Debugging early code
Debugging modules
Debugging kernel code with kdb
Looking at an Oops
Preserving the Oops
Summary
Further reading
Chapter 20: Profiling and Tracing
Technical requirements
The observer effect
Symbol tables and compile flags
Beginning to profile
Profiling with top
The poor man's profiler
Introducing perf
Configuring the kernel for perf
Building perf with the Yocto Project
Building perf with Buildroot
Profiling with perf
Call graphs
perf annotate
Tracing events
Introducing Ftrace
Preparing to use Ftrace
Using Ftrace
Dynamic Ftrace and trace filters
Trace events
Using LTTng
LTTng and the Yocto Project
LTTng and Buildroot
Using LTTng for kernel tracing
Using BPF
Configuring the kernel for BPF
Building a BCC toolkit with Buildroot
Using BPF tracing tools
Using Valgrind
Callgrind
Helgrind
Using strace
Summary
Further reading
Chapter 21: Real-Time Programming
Technical requirements
What is real time?
Identifying sources of non-determinism
Understanding scheduling latency
Kernel preemption
The real-time Linux kernel (PREEMPT_RT)
Threaded interrupt handlers
Preemptible kernel locks
Getting the PREEMPT_RT patches
The Yocto Project and PREEMPT_RT
High-resolution timers
Avoiding page faults
Interrupt shielding
Measuring scheduling latencies
cyclictest
Using Ftrace
Combining cyclictest and Ftrace
Summary
Further reading
Why subscribe?
Other Books You May Enjoy
Index