1. Start
  2. Press
  3. Press releases
  4. Heute: Pengutronix

Heute: Pengutronix

labgrid: Remote control & automation of embedded systems

Embedded Linux devices are at the heart of many products. These devices are usually specialised computers that are integrated into everyday objects to give them their functionality. From household appliances to medical products and industrial control systems, embedded Linux enables a wide range of applications.

Pengutronix helps its customers to develop and assemble customised operating systems for their embedded systems.

In order to be able to control these many devices automatically during development, labgrid [1] was developed - a library for remote control of embedded systems that can be distributed across different locations (offices, home offices, etc.).

The idea is to create an abstraction level of hardware control that allows automatic testing and software installation. But labgrid also helps with day-to-day development: simple hardware access or fast automation of recurring processes using an intuitive Python API.

Working remotely with labgrid

At Pengutronix, decentralised development teams need access to the few prototypes of the respective embedded hardware. In the simplest case, a serial console might suffice - but what if the hardware needs to be switched off and on again? New image on the SD card? Different boot medium via jumper? View the display in the home office?

Nothing could be easier: Simply create a "place" using labgrid, specify the hardware interfaces (switchable power supply, console, digital I/Os [2], SD card multiplexer [3], webcam and much more) and off you go.

This also helps against the flood of hardware on the desk: the devices can be set up in a cabinet or rack, for example, and their interfaces connected to a Linux computer. The "LXA Test Automation Controller" [4] offers an integrated solution for this with all kinds of interfaces.

Automation with labgrid

But labgrid can not only be used interactively. Recurring tasks can be easily automated.

For development purposes, the system is often booted from an SD card. Embedded systems receive their operating system as an image. This allows reproducible results whose defined interaction of a large number of software components can be easily tested with manageable effort.

Recurring work steps could therefore look something like this:

  • Switch off the power supply
  • flash SD card image
  • switch the power supply back on
  • check the boot loader to see if you have booted from the correct medium
  • boot into the Linux user space, log in with user/password
  • wait until a specific systemd service is started

With a small image, fast SD card and nimble fingers, these steps can be completed in a minute. During development, however, it can happen that these steps have to be repeated many times a day. And then there are the pull requests from the team, which also have to be built and their images tested. In short, it is worth automating these steps with labgrid.

Recognising regressions early

This automation can then also form the basis for automatic CI tests. Because if the boot loader and shell can already be reached automatically, the automatically built image can also be used and a few more integration tests can be carried out. This is conceivable, for example, for the aforementioned pull requests from the team. In this way, regressions are recognised before they are flagged.

Speaking of regressions: With labgrid it is also possible to find out which commit caused a regression. For example, the Linux kernel on an embedded device may no longer boot up to the init process after a kernel update. If it is not clear which change introduced the problem, the problematic commit can be identified with 'git bisect run <cmd>'. The specified command must be able to compile the kernel from git and then execute it automatically on the hardware with labgrid (if necessary via NFS or TFTP boot instead of SD card image). Based on the exit code, git bisect decides after a few runs which commit is responsible for the regression. identifiziert werden. Das angegebene Kommando muss in der Lage sein den Kernel aus dem git zu kompilieren und dieses dann mit labgrid automatisiert auf der Hardware auszuführen (ggf. per NFS- oder TFTP-Boot statt SD-Karten-Image). Anhand des Exit-Codes entscheidet git bisect nach einigen Durchläufen, welcher Commit für die Regression verantwortlich ist.

This brief insight shows how labgrid makes our everyday lives easier - whether interactive, semi-automated or fully automated.

You can find more material on this topic in our blog:

Pengutronix Logo