Mastering Embedded Linux, Part 5: Platform Daemons

This is the fifth part of the Mastering Embedded Linux series, which is designed to help you become an expert at developing low-cost, customized embedded Linux systems. Previously, I provided an overview of the embedded Linux landscape, then dove straight into tutorials for compiling Raspberry Pi firmware using Buildroot.

In this article and the next, I’m discussing how to apply these new Buildroot skills to begin packaging custom software into the firmware image. To do this, this article will discuss my concept of a “platform daemon” and implement a simple example of one in Rust. Then in Part 6, I’ll walk through the steps of writing a package to port this software to Buildroot.

Platform daemons

What’s a platform daemon?

Up until now, most of this series has largely used “off the shelf” embedded software, including the Buildroot distribution itself and the individual software packages in it. Independent of this common library of software, every embedded system has its own “secret sauce”—usually software written specifically for that system.

One such common pattern is a piece of software that picks up where the kernel leaves off: even if the kernel provides drivers to interact with peripherals, it doesn’t help with getting data moved between peripherals. For example, if you have an analog/digital converter that needs to report its readings over USB, the kernel lets userland talk to both, but it’s up to userland to read from the ADC and write to USB. Such software is target-specific enough that you simply can’t provide a generic solution that’s shipped with Buildroot.

This scenario typically results in a piece of software I call the platform daemon.¹ This daemon is bespoke software written for a particular target or family of targets—a “platform,” if you will. Often, it even contains device driver code for custom peripherals, even though such code really ought to go in the kernel as a module. Overall, the platform daemon contains intimate knowledge of the embedded system, it manages the state of all peripherals as a holistic unified entity, and it knows how to route and translate data coming and going from the system.

In microcontrollers, this would just be “firmware”—a separate RTOS task that waits for events coming from device drivers. But in Linux, this can typically be implemented as low-level userland code. This is great because you can often write and debug the daemon on your computer and then port it to the hardware. (Sound familiar?) It also means that the system is better-behaved, because bugs in userland don’t crash the entire system.

A GPIO platform daemon

For this article, I’ve written a proof-of-concept platform daemon in Rust. This daemon provides direct HTTP access to GPIO pins. Usually, there are some more abstractions around this, like “system mode” or “peripheral enable,” but since this is a proof of concept, I’ll dispense with the extra work.

A quick aside: If you’re a systems programmer who hasn’t yet run across Rust, I encourage you to take a look at it. It is a systems language that eliminates entire classes of bugs at compile time. It’s mature enough to use in production, but the language is still innovating rapidly and cares very deeply about getting it right the first time. Learning Rust will make you a better C/C++ programmer.

There, I’m done evangelizing. The source code of the daemon is available on GitHub, or you can follow along below for the salient points.

GPIO access in Linux

The kernel provides userland programs with direct access to GPIOs via dedicated APIs. As with most kernel topics, you can read all about these interfaces in the excellent kernel documentation, which give a kernel-side view of how GPIOs are handled. This model extends pretty well to the userland API.

Here are the Cliff’s notes. Linux provides a GPIO character device in /dev for applications to read/write to control GPIO pins. There is one character device named after each GPIO controller peripheral present in the system; the exact names depend on the system’s device tree. The upshot is that a given physical pin on the SoC is known to userland as a [peripheral, pin number] pair.

Userland can use the device file to command the GPIO’s input/output state, read its high/low state, and if it is an “output,” command its high/low drive.² Under the hood, this is done by sending special ioctl commands to this file, but you don’t have to worry about the gory details. There are lots of command-line tools that help handle this, including the kernel’s own libgpiod. (And yes, this is packaged in Buildroot!)

There are also many different language bindings that make this API nice to use directly from custom programs, which is what I’ll be doing here.

Note

You may have also heard about the older /sys/class/gpio kernel ABI for manipulating GPIOs. This has been deprecated for many years now and will be removed after 2020. However, it is still present in many embedded systems with older kernels, and you’ll likely see it pop up in discussions from time to time.

In this system, I’ll be using the modern character device API.

Linux GPIO in Rust

There is a dedicated Rust wrapper for the Linux GPIO system called gpio-cdev. This library makes it pretty easy to toggle a GPIO pin given a GPIO character device name and a pin number within that device.

To use the library, I’ll write a quick wrapper that maps well to REST API calls. I’d like to allow the user to set the mode to “input” or “output.” If the user chooses “output,” they should be able to send the value. Here’s an enum type encapsulating this:

#[derive(Serialize,Deserialize,Debug)]
enum GpioCmd {
    In,
    Out {
        value: bool,
    },
}

An important detail of the chardev GPIO API is that the file descriptor must be kept open by the program using the GPIO. If it is closed, all the GPIO lines will be released. So, I’ll create a map of [chip, pin] to lines, then cache the GPIO line objects in there. This cache will last as long as the daemon runs.

type GpioPath = (String, u32);

async fn main() {
    let active_pins = BTreeMap::<GpioPath, Chip>::new();
    let shared_pins_state = Arc::new(RwLock::new(active_pins));

Now, the user specifies a pin using the previously-discussed [peripheral, pin number] pair, plus the state they’d like to command it to. This function needs a copy of the Arc shared pointer we created earlier:

type GpioModifyResult = Result<(), gpio_cdev::errors::Error>;

fn gpio_modify(chip: String, pin: u32,
               pins: Arc<RwLock<BTreeMap<GpioPath, Chip>>>,
               body: GpioCmd)
    -> GpioModifyResult
{
    // Lock the global map of pins so we can have exclusive access
    // to the mut methods on it and its Chips.
    let mut shared_pins = pins.write().unwrap();
    let mut our_pin_entry = shared_pins.entry((chip.clone(), pin));

    let chipdev = match our_pin_entry {
        Occupied(ref mut entry) => entry.get_mut(),
        Vacant(entry) => entry.insert(Chip::new(format!("/dev/{}", chip))?)
    };

    let line = chipdev.get_line(pin)?;

Now, handle the GpioCmd. This function returns “nothing” () on success, because this is a “set” operation.

    match body {
        GpioCmd::Out { value } => {
            line.request(LineRequestFlags::OUTPUT, 0, "http-gpio")?
                // set_value returns () on success, which is what we need
                .set_value(value as u8)
        }
        GpioCmd::In => {
            line.request(LineRequestFlags::INPUT, 0, "http-gpio")?;
            // Set-to-input successful; we can ignore the returned LineHandle
            Ok(())
        }
    }
}

Now I can move on to wrapping this in a web server.

Embedded web servers in Rust

There are several web server frameworks available for Rust, but the one I settled on is called Warp. There are several reasons I selected Warp:

• Runs on stable Rust: This rules out the otherwise very nice-looking Rocket (as of this writing in May 2020). Stable Rust is important to me because Rust provides extremely strong stability guarantees that make my life easier as a maintainer.
• Comparatively lightweight dependency list: Embedded systems don’t have the horsepower of a cloud server—in particular, they’re pretty light on CPU and memory. Selecting a small web framework helps ensure that compile times stay low and that the daemon itself remains lightweight at runtime. Because everything compiles down to native code, I don’t anticipate any speed problems.³
• Extremely straightforward setup: When you’re writing a platform daemon, typically you’re more concerned about talking to the hardware and managing the system than you are using the latest whiz-bang web framework.

Warp makes it trivial to have the webserver call your code. The only tricky bit is passing in the cache along with the request:

let with_pins_state = warp::any().map(move || shared_pins_state.clone());

// POST /gpio/chipname/pinnum -> String
let gpio_modify = warp::post()
    .and(warp::path!("gpio" / String / u32))
    .and(with_pins_state)
    .and(warp::body::json())
    .map(gpio_modify)
    .map(as_reply);

My as_reply helper transforms the Result into a reply for Warp:

fn as_reply(value: GpioModifyResult) -> Box<dyn warp::Reply> {
    // Return if success, or stringify the error if not
    match value {
        Ok(_) => Box::new("Success"),
        Err(err) => Box::new(
            warp::reply::with_status(err.to_string(),
                                     StatusCode::INTERNAL_SERVER_ERROR))
    }
}

Boom, done. A simple platform daemon in about 70 lines of Rust.

Exercising the code

Since I’m wearing my “developer hat,” I’ll try to test on my workstation, without cross-compiling just yet. Linux provides lots of “dummy” drivers to simulate actual hardware, and GPIO is no exception. The gpio-mockup kernel module provides the ABI but doesn’t actually use any hardware. Each pair of arguments in gpio_mockup_ranges describes how many pins should be attached to each dummy GPIO peripheral. The new peripherals appear in /dev/ as expected:

# modprobe gpio-mockup gpio_mockup_ranges=-1,32,-1,32
# ls -lh /dev/gpio*
crw------- 1 root root 254, 0 Apr  7 22:07 /dev/gpiochip0
crw------- 1 root root 254, 1 Apr  7 22:07 /dev/gpiochip1

Here’s how to call the daemon using httpie.⁴

$ http POST localhost:3030/gpio/gpiochip0/2 Out:='{"value": true}'
HTTP/1.1 200 OK
content-length: 7
content-type: text/plain; charset=utf-8
date: Wed, 08 Apr 2020 03:19:41 GMT

Success

This POSTs the following JSON to /gpio/gpiochip0/2, where Serde automatically translates it to the Rust type and Warp passes it to the callback:

{ "Out": { "value": true } }

Malformed requests are rejected automatically by the framework, and if anything goes wrong while setting the pin, as_reply sends back the error message. Overall, I am pleased with the proof of concept—it is concise, but thanks to the underlying libraries, it is largely correct and well-behaved.

Again, I encourage you to study the complete source code on GitHub and let me know if you have feedback – either via email, as a GitHub issue, or with the fancy new comments section I’ve added.

Ideas for improvement

Every platform daemon I’ve dealt with has ended up needing a small configuration file, either because I wanted to target multiple boards at once, or because I wanted to make my own life easier while I wrote the software. To facilitate this, a small YAML or INI-formatted configuration file in /etc could tell the platform daemon which GPIO character device it ought to use, or provide string aliases for some of the pins so clients could just POST /gpio/yellow-led.⁵

If I were to implement this, I’d make sure to provide a sample config file in the http-gpio README, then provide a suitable configuration file in my target’s overlay directory, just like the configuration files for other software in the last article.

Key takeaways

• If you’re doing anything clever with your embedded system, you’ll eventually wind up with a platform daemon of some kind.
• Wear multiple hats! Think about what would make the maintainer’s job easier while you’re writing custom software—chances are, you are the maintainer, and you’ll save yourself work down the road.

Recommended reading

• LWN is an extremely well-written news site that regularly delivers in-depth technical articles about the Linux kernel and various free software projects surrounding it. The articles are free to read after a period of time, and you can also buy a subscription to read them as soon as they come out. It’s well worth it—the articles often include code samples, historical notes, and a great comments section about whatever they’re writing about. For example, here’s a pair of LWN articles about Linux GPIO:
  • GPIO in the Kernel: an introduction
  • GPIO in the Kernel: future directions
• This excellent article gives a good command-line tutorial of character device GPIO control.
• Hackaday’s Linux-Fu series has in-depth discussion of various technical userland topics, not just GPIOs. Many of these will come in handy when you start to write your own daemon code.

Coming up

In the next article I’ll finish the ongoing Buildroot theme by packaging http-gpio for the Raspberry Pi I’ve been working with. This will include some “best practices” for your own packages. I’ll also have a deeper discussion of startup scripts and the init system, which I’ll use to start the daemon automatically.

Subscribe and comment

As always, you can subscribe to my blog for updates. Please also leave a comment below if you have a question or want to point something out that I’ve overlooked. I appreciate all the communication I get. Thanks for reading!