Skip to content

A Mock QCanBusDevice for TDD

I am using TDD to implement a hardware-independent mock QCanBusDevice. We can use MockCanBusDevice in unit, integration and acceptance tests – without access to a hardware CAN bus. A terminal application can also use it in its own process to simulate the communication with a machine.

Motivation

The machine component in the hexagonal architecture of a harvester terminal comes with three adapers:

  • The Machine adapter communicates over 1-4 CAN busses with the other ECUs of the harvester.
  • The In-Process Simulator adapter runs on the development computer in the same process as the terminal application, simulates the ECUs and communicates over function calls with the terminal applicaton.
  • The Mock adapter is used by unit, integration and acceptance tests to simulate or mock the communication with the ECUs.
Hexagonal Architecture for harvester terminal with four ports and multiple adapters per port.
Hexagonal Architecture for harvester terminal

Unit tests must be able to force a CAN bus device into all kinds of exceptional scenarios. This is very difficult or nearly impossible for real CAN bus devices. Unit tests must also run very quickly (best in less than 3 seconds) to enable the many tiny steps of TDD. Hence, accessing sockets, files or databases is not an option. Therefore, unit tests do not use the hardware-dependent QCanBusDevice but a hardware-independent mock one

The MockCanBusDevice is both a mock and a fake (see p. 114 of James Grenning’s book Test-Driven Development for Embedded C). As a fake, it provides a partial and simplified implementation of a SocketCAN device. As a mock, it allows tests to control its behaviour. For example, tests can configure the mock so that reading or writing a frame fails or succeeds.

MockCanBusDevice is used by the in-process simulator and the mock adapter of the Machine component. It makes writing unit, integration and acceptance tests much easier and running these tests much faster. I am using TDD to implement a MockCanBusDevice that resembles the SocketCAN implementation of QCanBusDevice. You find the example code in the GitHub repository eu-terminal-apps.

Creating the Mock QCanBusDevice

We create a test skeleton with a single test testCreateCanBusDevice for creating a CAN bus device. The test always passes.

We instantiate the class MockCanBusDevice in the test function:

// File: test_can_bus_device.cpp
#include "mock_can_bus_device.h"

void TestCanBusDevice::testCreateCanBusDevice()
{
    MockCanBusDevice canBus;
}

We provide the minimal implementation of the class MockCanBusDevice that makes the test compile and hence pass. The class declaration is enough, as the compiler creates the default constructor (Commit: 5be529c).

We derive MockCanBusDevice from the abstract class QCanBusDevice and add to CMakeLists.txt the Qt5::SerialBus module, which contains all the CAN classes. 

// File: mock_can_bus_device.h
#include <QCanBusDevice>

class MockCanBusDevice : public QCanBusDevice
{
};

As expected, the compiler complains that the class MockCanBusDevice is abstract and that the pure virtual methods writeFrame, interpretErrorFrame, open and close have not been implemented. In QtCreator, we place the cursor in the word QCanBusDevice, hit Alt + Enter and select Insert Virtual Functions of Base Classes. As the dialog has all the pure virtual functions selected, we just click on OK and let QtCreator generate the declarations and definitions of the pure virtual functions. We add the missing return statements.

// File: mock_can_bus_device.cpp

bool MockCanBusDevice::writeFrame(const QCanBusFrame &frame)
{
    return false;
}

QString MockCanBusDevice::interpretErrorFrame(const QCanBusFrame &errorFrame)
{
    return {};
}

bool MockCanBusDevice::open()
{
    return false;
}

void MockCanBusDevice::close()
{
}

The test now compiles and passes.

Connecting to the Mock QCanBusDevice

After creating the MockCanBusDevice, we call QCanBusDevice::connectDevice to connect the device with the real CAN bus. True to its name, MockCanBusDevice will mock the real CAN bus.

The non-virtual function connectDevice calls the overridden function MockCanBusDevice::open in the device state ConnectingState. If open succeeds, it sets the state to ConnectedState and returns true. Otherwise, open returns false, connectDevice sets the state to UnconnectedState and returns immediately with false. We get this information from a look at the implementation of QCanBusDevice::connectDevice in qtserialbus/src/serialbus/qcanbusdevice.cpp and at the implementation of SocketCanBackend::open in qtserialbus/src/plugins/canbus/socketcan/socketcanbackend.cpp.

The test testConnectionSucceeded checks whether the device can connect to the (mock) CAN bus successfully when unconnected.

void TestCanBusDevice::testConnectionSucceeded()
{
    MockCanBusDevice device;
    QCOMPARE(device.state(), QCanBusDevice::UnconnectedState);
    QVERIFY(device.connectDevice());
}

The test fails, because open returns false. The simplest fix to make open return true.

bool MockCanBusDevice::open()
{
    return true;
}

If the connection succeeds, the device must be in the state QCanBusDevice::ConnectedState. 

void TestCanBusDevice::testConnectionSucceeded()
{
    MockCanBusDevice device;
    QVERIFY(device.connectDevice());
    QCOMPARE(device.state(), QCanBusDevice::ConnectedState);
}

The test fails, because the device is in QCanBusDevice::ConnectingState. The fix is a one-liner.

bool MockCanBusDevice::open()
{
    setState(QCanBusDevice::ConnectedState);
    return true;
}

We remove testCreateCanBusDevice, as all test functions will create a MockCanBusDevice.

Clients of MockCanBusDevice must handle the case, where the connection succeeds as well as where it fails. In the failure case, client tests must be able to control the return value of the function open. The test testConnectionFailed introduces this capability:

void TestCanBusDevice::testConnectionFailed()
{
    MockCanBusDevice device;
    device.setOpenSucceeded(false);
    QVERIFY(!device.connectDevice());
    QCOMPARE(device.state(), QCanBusDevice::UnconnectedState);
}

The function setOpenSucceeded stores its argument in the member variable m_openSucceeded (default: true). If open is expected to fail (m_openSucceeded == false), it immediately returns with false. If open is expected to succeed (m_openSucceeded == true), it sets the state to ConnectedState and returns true.

bool MockCanBusDevice::open()
{
    if (!m_openSucceeded)
    {
        return false;
    }
    setState(QCanBusDevice::ConnectedState);
    return true;
}

Both tests pass now.

A further look at the implementation reveals that QCanBusDevice::connectDevice returns with false without calling open, if the state is not UnconnectedState on entrance. Additionally, the function outputs a warning message and sets the error to ConnectionError. The resulting state is the same as on entrance. ConnectionError is also set, if the open call fails.

We must run testConnectionFailed on different combinations of open‘s return value and the state before calling connectDevice and check the result for the correct state and error. This is best done by data-driven tests. The data-driven version of testConnectionFailed looks as follows.

void TestCanBusDevice::testConnectionFailed_data()
{
    QTest::addColumn<QCanBusDevice::CanBusDeviceState>("stateBefore");
    QTest::addColumn<bool>("openSucceeded");
    QTest::addColumn<QCanBusDevice::CanBusDeviceState>("stateAfter");
    QTest::addColumn<QCanBusDevice::CanBusError>("canError");

    QTest::newRow("Unconnected + false -> Unconnected + NoError")
            << QCanBusDevice::UnconnectedState << false
            << QCanBusDevice::UnconnectedState << QCanBusDevice::ConnectionError;
}

void TestCanBusDevice::testConnectionFailed()
{
    QFETCH(QCanBusDevice::CanBusDeviceState, stateBefore);
    QFETCH(bool, openSucceeded);
    QFETCH(QCanBusDevice::CanBusDeviceState, stateAfter);
    QFETCH(QCanBusDevice::CanBusError, canError);

    MockCanBusDevice device;
    device.setState(stateBefore);
    device.setOpenSucceeded(openSucceeded);
    QVERIFY(!device.connectDevice());
    QCOMPARE(device.state(), stateAfter);
    QCOMPARE(device.error(), canError);
}

The code doesn’t compile, because QCanBusDevice::setState is protected. We reimplement it as a public function in MockCanBusDevice. The above test compiles but fails on the last QCOMPARE. We fix this by setting the error in the failure case of the open function.

bool MockCanBusDevice::open()
{
    if (!m_openSucceeded)
    {
        setError(u"Open failed"_qs, QCanBusDevice::ConnectionError);
        return false;
    }
    setState(QCanBusDevice::ConnectedState);
    return true;
}

We now put the data-driven test to good use and add more rows to testConnectionFailed_data.

void TestCanBusDevice::testConnectionFailed_data()
{
    ...
    QTest::newRow("Connected + true -> Connected + ConnectionError")
            << QCanBusDevice::ConnectedState << true
            << QCanBusDevice::ConnectedState << QCanBusDevice::ConnectionError;
    QTest::newRow("Connecting + false -> Connecting + ConnectionError")
            << QCanBusDevice::ConnectingState << false
            << QCanBusDevice::ConnectingState << QCanBusDevice::ConnectionError;
    QTest::newRow("Closing + true -> Closing + ConnectionError")
            << QCanBusDevice::ClosingState << true
            << QCanBusDevice::ClosingState << QCanBusDevice::ConnectionError;
}

We don’t add the row “Unconnected + true”, because this would be the same as testConnectionSucceeded. We could add the three rows “Connected + false”, “Connecting + true” and “Closing + false” to cover all possible combinations. I didn’t, because these three rows would add hardly any value. The rows would not test MockCanBusDevice but QCanBusDevice, which is not our objective.

Disconnecting from the Mock QCanBusDevice

The function QCanBusDevice::disconnect calls the pure virtual function QCanBusDevice::close, which is overridden by MockCanBusDevice::close. A look at the implementation of QCanBusDevice::disconnectDevice reveals the exact behaviour.

If the device is in ConnectedState, disconnectDevice sets the state to ClosingState and calls the virtual function close, which sets the state to UnconnectedState. We add this good case to the end of testConnectionSucceeded and rename the test function into testConnectAndDisconnectDevice. 

void TestCanBusDevice::testConnectAndDisconnectDevice()
{
    MockCanBusDevice device;
    QCOMPARE(device.state(), QCanBusDevice::UnconnectedState);

    QVERIFY(device.connectDevice());
    QCOMPARE(device.state(), QCanBusDevice::ConnectedState);

    device.disconnectDevice();
    QCOMPARE(device.state(), QCanBusDevice::UnconnectedState);
}

The test fails, because close does nothing so far. Let us change this.

void MockCanBusDevice::close()
{
    setState(QCanBusDevice::UnconnectedState);
}

We must check what happens when disconnectDevice is called in the other three states. If the device is Connecting, disconnectDevice behaves the same as for Connected. If the device is Unconnected or Closing, disconnectDevice outputs the warning “qt.canbus: Can not disconnect an unconnected device” and returns without calling close. The state on exit is the same as on entrance. No error is reported. This demands another data-driven test.

void TestCanBusDevice::testDisconnectDevice_data()
{
    QTest::addColumn<QCanBusDevice::CanBusDeviceState>("stateBefore");
    QTest::addColumn<QCanBusDevice::CanBusDeviceState>("stateAfter");

    QTest::newRow("Connecting -> Unconnected")
            << QCanBusDevice::ConnectingState << QCanBusDevice::UnconnectedState;
    QTest::newRow("Closing -> Closing")
            << QCanBusDevice::ClosingState << QCanBusDevice::ClosingState;
    QTest::newRow("Unconnected -> Unconnected")
            << QCanBusDevice::UnconnectedState << QCanBusDevice::UnconnectedState;
}

void TestCanBusDevice::testDisconnectDevice()
{
    QFETCH(QCanBusDevice::CanBusDeviceState, stateBefore);
    QFETCH(QCanBusDevice::CanBusDeviceState, stateAfter);

    MockCanBusDevice device;
    device.setState(stateBefore);
    device.disconnectDevice();
    QCOMPARE(device.state(), stateAfter);
    QCOMPARE(device.error(), QCanBusDevice::NoError);
}

Writing To the Mock QCanBusDevice

If everything is alright, the function SocketCanBackend::writeFrame writes the given frame to the CAN bus without any queuing, emits the signal framesWritten(1) and returns true. Let us get the implementation of MockCanBusDevice::writeFrame rolling with this test.

void TestCanBusDevice::testWriteFrame()
{
    MockCanBusDevice device;
    QSignalSpy spy{&device, &QCanBusDevice::framesWritten};
    QVERIFY(device.writeFrame(QCanBusFrame{}));
    QCOMPARE(spy.count(), 1);
}

The test fails, because writeFrame is hard-wired to return false and because it doesn’t emit the signal framesWritten. The fix is easy.

bool MockCanBusDevice::writeFrame(const QCanBusFrame &frame)
{
    emit framesWritten(1);
    return true;
}

We know that a frame was written but not which frame. We make MockCanBusDevice record all the frames written and later also the frames read. Tests of MockCanBusDevice‘s clients can check whether a request (a written frame) sees the right response (a read frame). We extend testWriteFrame a little bit.

void TestCanBusDevice::testWriteFrame()
{
    MockCanBusDevice device;
    QSignalSpy spy{&device, &QCanBusDevice::framesWritten};
    QVERIFY(device.writeFrame(QCanBusFrame{}));
    auto frames = device.recordedFrames();
    QCOMPARE(frames.count(), 1);
    QCOMPARE(spy.count(), frames.count());
    QCOMPARE(frames[0].toString(), QCanBusFrame{}.toString());
}

As the test writes one frame, the device must record one frame as well (1st QCOMPARE). Every time writeFrame is called, device emits the signal framesWritten(1) (2nd QCOMPARE). The recorded frame must be the same as the written frame (3rd QCOMPARE). As QCanBusFrame lacks an equality operator, we compare the string representations of the frames.

The function MockCanBusDevice::writeFrame appends the written frame to the vector m_recordedFrames of QCanBusFrames. Clients can access the recorded frames through the function recordedFrames.

bool MockCanBusDevice::writeFrame(const QCanBusFrame &frame)
{
    m_recordedFrames.append(frame);
    emit framesWritten(1);
    return true;
}

The SocketCAN implementation of QCanBusDevice handles a couple of error conditions.

  • When the device is not connected, the function writeFrame returns with false immediately without setting any error.
  • When writeFrame fails to write the frame to the socket representing the real CAN bus, it flags a WriteError and returns false.
  • When writeFrame tries to write an invalid frame, it flags a WriteError and returns false.
  • When writeFrame tries to write a frame in flexible data-rate (FD) format but FD is not enabled, it flags a WriteError and returns false.

I’ll only look at the first two error conditions in this post. You’ll find the third one in the example code. I won’t implement the fourth one, as none of the CAN devices, with which I have been working so far, has used FD. Let us add the test for writing a CAN frame when the device is not connected.

void TestCanBusDevice::testWriteNoFrameWhenNotConnected()
{
    MockCanBusDevice device;
    QVERIFY(!device.writeFrame(QCanBusFrame{}));
}

This test fails, because writeFrame always returns true. We make this test pass by adding a bouncing condition.

bool MockCanBusDevice::writeFrame(const QCanBusFrame &frame)
{
    if (state() != ConnectedState)
        return false;
    m_recordedFrames.append(frame);
    emit framesWritten(1);
    return true;
}

testWriteNoFrameWhenNotConnected passes but testWriteFrame fails. We fix this failure by connecting the device before calling writeFrame.

void TestCanBusDevice::testWriteFrame()
{
    MockCanBusDevice device;
    QSignalSpy spy{&device, &QCanBusDevice::framesWritten};
    device.connectDevice();
    QVERIFY(device.writeFrame(QCanBusFrame{}));
    ...

We complete testWriteNoFrameWhenNotConnected by checking that the signal framesWritten is never emitted and that no frames were recorded.

The next test handles the error scenario, where writing to the socket representing the real CAN bus fails. Reasons could be that the CAN bus is overloaded or that the CAN connector has a defect. In the failure case, writeFrame flags a WriteError, returns false, does not record the frame and does not emit the framesWritten signal.

We must introduce a flag m_writeSucceeded similar to m_openSucceeded. The flag allows test functions to control whether the write fails or succeeds.

void TestCanBusDevice::testWriteFrameFails()
{
    MockCanBusDevice device;
    device.connectDevice();
    device.setWriteSucceeded(false);
    QVERIFY(!device.writeFrame(QCanBusFrame{}));
    QCOMPARE(device.error(), QCanBusDevice::WriteError);
}

We implement setWriteSuceeded similar to setOpenSucceeded and extend writeFrame by a new if-clause.

bool MockCanBusDevice::writeFrame(const QCanBusFrame &frame)
{
    if (state() != ConnectedState)
        return false;
    if (!m_writeSucceeded)
    {
        setError(u"Writing frame failed"_qs, QCanBusDevice::WriteError);
        return false;
    }
    m_recordedFrames.append(frame);
    emit framesWritten(1);
    return true;
}

Again, we complete the test by checking that the signal framesWritten is never emitted and that no frames were recorded.

Reading From the Mock QCanBusDevice

QCanBusDevices notify their clients like the test functions through the signal framesReceived that new CAN frames have arrived. The signal framesReceived is emitted by the protected function QCanBusDevice::enqueueReceivedFrames. When clients receive the signal framesReceived, they consume the frames with readFrame, framesAvailable and readAllFrames.

A function receiveFrames, which simulates or mocks the reception of some CAN frames, adds the received frames to the incoming queue and enqueueReceivedFrames does the rest as with frames received from the real CAN bus. Let us start with the following test.

void TestCanBusDevice::testReadFramesOneByOne()
{
    MockCanBusDevice device;
    QSignalSpy spy{&device, &QCanBusDevice::framesReceived};
    device.connectDevice();
    device.receiveFrames({frame1, frame2});
    QCOMPARE(spy.count(), 1);
}

frame1 and frame2 are valid frames defined as constant member variables of TestCanBusDevice. The test passes once we provide the following implementation for receiveFrames.

void MockCanBusDevice::receiveFrames(const QVector<QCanBusFrame> &frames)
{
    enqueueReceivedFrames(frames);
}

We strengthen the test by checking the number of available frames before and after reading a single frame with readFrame.

void TestCanBusDevice::testReadFramesOneByOne()
    ...
    QCOMPARE(device.framesAvailable(), 2);
    auto frame = device.readFrame();
    QCOMPARE(frame.toString(), frame1.toString());

    QCOMPARE(device.framesAvailable(), 1);
    frame = device.readFrame();
    QCOMPARE(frame.toString(), frame2.toString());

    QCOMPARE(device.framesAvailable(), 0);

We test readAllFrames in pretty much the same way.

void TestCanBusDevice::testReadAllFrames()
{
    MockCanBusDevice device;
    QSignalSpy spy{&device, &QCanBusDevice::framesReceived};
    device.connectDevice();
    device.receiveFrames({frame1, frame2});
    QCOMPARE(spy.count(), 1);

    QCOMPARE(device.framesAvailable(), 2);
    auto frames = device.readAllFrames();
    QCOMPARE(frames[0].toString(), frame1.toString());
    QCOMPARE(frames[1].toString(), frame2.toString());
    QCOMPARE(device.framesAvailable(), 0);
}

In contrast to the pure virtual function QCanBusDevice::writeFrame, QCanBusDevice::readFrame and QCanBusDevice::readAllFrames are non-virtual functions and already perform much of the error handling. They bail out immediately with an OperationError if the device is not in ConnectedState. readFrame doesn’t flag any error but returns an invalid frame, if it is called on an empty queue of incoming frames. We don’t have to test these error situations, because the tests of QCanBusDevice do it already.

This leaves us with one error situation: reading from the socket representing the real CAN bus fails. We introduce a flag m_readSucceeded similar to m_writeSucceeded and control the flag through the function setReadSucceeded. The following test gets the implementation rolling.

void TestCanBusDevice::testReceivingFramesFails()
{
    MockCanBusDevice device;
    device.connectDevice();
    device.setReadSucceeded(false);
    device.receiveFrames({frame1, frame2});
    QCOMPARE(device.error(), QCanBusDevice::ReadError);
    QCOMPARE(device.framesAvailable(), 0);
}

We extend receiveFrames by the error case.

void MockCanBusDevice::receiveFrames(const QVector<QCanBusFrame> &frames)
{
    if (!m_readSucceeded)
    {
        setError(u"Reading frames failed"_qs, QCanBusDevice::ReadError);
        return;
    }
    enqueueReceivedFrames(frames);
}

The equivalent of receiveFrames in the SocketCAN implementation (SocketCanBackend::readSocket) works the same in principle. It flags a ReadError and returns without enqueueing the received frames, if it encounters a problem reading from the CAN socket.

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *