There are a lot of articles online about TDD, so it might seem that yet another post will not be too interesting for the software landscape. However I'd like to make a small contribution on this topic but emphasizing on embedded systems, which is the field I work on. My goal with these lines is to understand how can we improve the quality of our projects, so please, don't hesitate to contact me and discuss with me new ideas.

Can TDD be used on embedded software?

One of the identifying features of embedded systems software is the continuous access to external peripherals. Many of these devices spend most of the time reading an accelerometer through I2C, writing an external flash memory through SPI, toggling switches using GIPOs, receiving commands from a PC through UART... In computer software, some equivalent actions would be accessing to databases, files or sockets. What we are basically doing is accessing external resources that do not depend on us and that we can't know how are they going to respond.

Anyone with some experience on TDD will realize that this is what test doubles (mocks, fakes, stubs...) are meant for. I find that this tool really important in my workflow. In fact it's something I use everyday.

Let's see an example. Imagine that we want to connect and Arduino to an I2C accelerometer such as the HMC5983. These are the internal registers we can access to:

Usually, the first thing we do to test if a chip is working as expected is to read some set of registers which hold a constant and known value. In this case, registers 0x0A, 0x0B and 0x0C contain 'H', '4' and '3' respectively.

An unit test I'd create to validate what happens if the MCU receives the right identification bytes would look something like this (the testing framework is not important for the discussion, but if you wonder what I'm using in the following examples, it's Ceedling):

void test_when_ValidIDRead_should_DetectValidModule(void) {
    // Request register
    char req_expect[] = {0x0A};
    i2c_send_ExpectWithArrayAndReturn(HMC_ADDR, req_expect, sizeof(req_expect), 1, 1);

    // Get response
    i2c_receive_ExpectAndReturn(HMC_ADDR, NULL, 3, 3);
    i2c_receive_IgnoreArg_rx_data(); // Ignore rx_data (NULL) because we will fake it
    // Fake response (3 bytes)
    char expected_id[] = {'H', '4', '3'};
    i2c_receive_ReturnArrayThruPtr_rx_data(expected_id, sizeof(expected_id));

    // Returns true because the ID is correct
    TEST_ASSERT_TRUE(hmc5983_init());
}

What we are doing in the previous snippet is executing hmc5983_init() and making sure that it calls two functions in a given order:

1. i2c_send() to request the content of register 0x0A
2. i2c_receive() to read the response, which we'll fake with i2c_receive_ReturnArrayThruPtr_rx_data to be {'H', '4' y '3'}

I'd like to highlight that this test validates that the values returned by i2c_receive() are properly parsed, but this doesn't mean that if we call hmc5983_init() in our board, it is going to work as a charm. There could be an error on i2c_receive() or we could be using a wrong UART configuration. Any of those situations would break the communications between the MCU and the accelerometer.

Another test I could create would verify that an LED is turned on if the acceleration matches some value:

void test_when_MagicAccelValuesDetected_should_TurnLEDOn(void) {
    // Request register
    char req_expect[] = {0x03};
    i2c_send_ExpectWithArrayAndReturn(HMC_ADDR, req_expect, sizeof(req_expect), 1, 1);

    // Get response
    i2c_receive_ExpectAndReturn(HMC_ADDR, NULL, 2, 2);
    i2c_receive_IgnoreArg_rx_data(); // Ignore rx_data (NULL) because we will fake it
    // Fake response (2 bytes)
    char accelx_expect[] = {0x30, 0x21}; // Expected value
    i2c_receive_ReturnArrayThruPtr_rx_data(accelx_expect, sizeof(accelx_expect));

    led_toggle_Expect();

    hmc5983_xaxis_not_zero();
}

As you can see, with these tests we are validating -to some extent- the real hardware, but the best part is that we do not depend on it. Let me explain what I mean. In embedded systems we can run tests on three different environments:

• In the real hardware
• In a simulator
• In the development computer itself

All of them have pros and cons, but I personally find better not to depend on the hardware too much. I use this kind of tests when I'm not in the office and I need to keep working, or even when I'm waiting for a new hardware revision. Another advantage is that this option is the easiest one (but not the only one) if we want to run the tests in a CI server. We can say that this is the closest the embedded developers can get to digital nomads!

If we have a good test environment we can validate software without needing the hardware it's been designed for.

Sounds interesting, but what's the problem?

Probably no software dev will have found any original idea in what I've just explained, and I strongly believe that embedded developers should also be familiar with this content. I find having unit tests running periodically and automatically an essential activity, at least as a first line of defense of our firmware. This is specially true if the alternative is manual testing!

Nevertheless, despite of this being a widespread practice for our colleagues, this is not what I have found on embedded development teams. It is true that the context matters. There are some moments where making the effort of implementing TDD is not required. For example, I do not see it as necessary for small and simple pieces of code or proofs of concept, but that's basically it.

I don't know why this technique is not as common in embedded systems development, although I will dare to make some guesses:

• The main reason why I think TDD has not been successful on this sector is the complexity of the toolchains. Our environment should ideally allow us to run tests in our computer, in a simulator and in the final hardware as well. We would also like tests to automatically run after each commit, but we would need to link the VCS to our board somehow. It can be done but there are no plug-and-play solutions. The fact of depending on custom hardware and many times on custom tools for each chip manufacturer (simulators, compilers...) only makes things more difficult. This is also a complex task for software devs, in fact one of the job positions that has emerged with more impact in the past years, devops engineer, was born to deal with this!
• There are also historical reasons. As can be read in this other article, embedded developers traditionally lack of a software career. We usually start working on it because we need it, it's required for our hardware to work. That would explain why the evolution of the development processes seems slower compared with high-level software.
• The fact of C being the most common programming language doesn't help. Creating tools to assist us in the process of writing and executing this tests is more difficult to achieve with procedural programming. Nothing is impossible (Ceedling is actually a good example), but using object oriented languages would make things easier.

How would I approach a new project?

What do I propose to improve the quality of the code given the problems I've just mentioned? Which would be my first task when starting a new project?

The first thing I'd do would be a validation of the project's feasibility. I'd buy a development board and write some quick-and-dirty code. The shorter the duration of this stage the better. I'd forget about TDD at this point.

Once we know we can achieve our goals, I'd reserve one week to build a good toolchain, a development environment that helps me be more productive. It might seem a waste of time at this point, but it will pay off in the near future. For instance, imagine that we need 3 minutes and 3 different commands to compile, run the tests and program our hardware but with this week of work we can do it in 30 seconds with a single command. It doesn't feel like a bad investment, doesn't it?

My next task wouldn't be related to TDD but it's one of the most important steps for the product to succeed. The lean startup and MVPs terms have become a business buzzword recently, but they were born with the software world into mind. For those of us building physical products, following this techniques is more complicated, because hardware is not easily upgradeable. Because of this, the embedded systems engineers should do as much as we can to overcome this difficulty and create a system to update the software remotely as soon as possible. The sooner we have it, the better we can ship the product to our early adopters and improve it based on their feedback.

Finally, and this might be the trickiest and most important step, all the devs in the team must understand what are we trying to achieve with TDD and why it is so important. If only one of them doesn't use TDD (someone who keeps breaking tests or who writes code which is not covered by unit tests), the effort of the rest of the team will be useless.

This might not be the ideal workflow in all the scenarios, but it's the one my (reduced) experience tells me to choose in case I started working on a new product.

What do you think about this? Would you take the same steps in your projects or would you do something different? Have you also encountered that embedded engineers tend not give enough importance to unit testing?

Cover picture by Jonas Svidras.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License