The ports-and-adapters architecture should be the standard architecture for HMI applications. Its parts are loosely coupled, cohesive, easy to test and easy to extend. We can apply the reverse Conway manoeuvre to create self-dependent teams with minimal dependencies on other teams. I’ll motivate the ports-and-adapters or hexagonal architecture with USB ports and adapters and look at the architecture pattern from the production, testing and team perspective. I’ll apply the architecture to the HMI terminal of a harvester.
Introduction
In his original work, Alistair Cockburn explains the ports-and-adapters or hexagonal architecture in the context of enterprise software. More recent descriptions by Juan Manuel Garrido de Paz and by Valentina Cupać stay in the same context. Enterprise software is not at all my focus area and probably not yours either. A seasoned embedded developer told me: “I have known hexagonal architecture for years, but I never thought of applying it to embedded systems.”
It’s a real pity that the ports-and-adapters architecture is only known by a few people in the embedded field and applied by even fewer. I think that this architecture should be the de-facto standard for HMI applications of machines, vehicles and devices. Most of the time, it will be a good choice leading to long-lived, easy-to-test and easy-to-change software.
A presumptive architecture is […] dominant in a particular domain. Rather than justifying their choice to use it, developers in that domain may have to justify a choice that differs from the presumptive architecture.
George Fairbanks, Just Enough Software Architecture: A Risk-Driven Approach, p. 23
The ports-and-adapters architecture is such a presumptive architecture for HMI applications of agricultural, construction and industrial machines, medical devices, infotainment systems of cars, TVs, STBs, home appliances and other embedded devices. It significantly increases our chances to base our software on a good, right and successful architecture.
Good, right and successful architectures:
Dana Bredemeyer and Ruth Malan, Software Architecture Workshop (training), 2021
Good – technically sound
Right – meeting stakeholder needs
Successful – delivering value
In other words, we significantly reduce the risk to end up with a big ball of mud, agonisingly slow feature development, a flood of bugs and costly rewrites of large parts of software.
Application Context
I’ll use the main HMI Terminal of a harvester as my running example (see the success stories of the Krone forage harvesters and the ROPA sugar-beet harvesters, and my posts about agriculture for some background). I’ll derive some architecturally significant requirements (ASRs) from the system diagram above. If the ports-and-adapters architecture makes it easy to satisfy these ASRs, we have chosen a good and technically sound architecture.
ASR-1 – Changing the Cutting Length
The Driver changes the cutting length through the HMI Terminal in steps of 1 mm. The HMI Terminal sends the changed cutting length to the Machine.
ASR-2 – Displaying the Engine Speed
Every 10 ms, the Machine sends the engine speed as a J1939 message over the CAN bus to the HMI Terminal, which displays it to the Driver.
ASR-3 – Use MQTT for ECU communication
Currently, the ECUs communicate with J1939 messages over CAN bus. In the future, the ECUs shall communicate with MQTT messages over Ethernet.
ASR-4 – Recording the Harvested Area
Every second, the Machine sends the harvested area as a J1939 message to the HMI Terminal, which forwards it as an MQTT message over a 4G/5G connection to Accounting. Accounting stores the harvested area for each job (harvesting a field owned by a farmer).
ASR-5 – MQTT Broker in IoT Cloud
The HMI Terminal provides an MQTT client and Accounting an MQTT broker. The broker can be hosted on any IoT cloud like AWS IoT and Azure IoT.
ASR-6 – Showing the Video from the Rear-View Camera
The rear-view Camera streams its video over Ethernet to the HMI Terminal. The HMI Terminal displays the video so that the Driver can see what’s going behind the harvester.
I can think of many more ASRs like OTA updates, remote support, full control of the UI by touch, keys or voice, power management, readability in daylight and vibration resistance of the board (see Architecture of Qt Embedded Systems: Single vs. Multiple GUI Applications and Architecture of Qt Embedded Systems: Operating Conditions for more ASRs). For vetting the ports-and-adapters architecture, the ASRs above are enough.
USB Ports and Adapters
We can understand the idea of the ports-and-adapters architecture by looking at USB ports and adapters. While embedded devices still sport many different ports like CAN, RS232, LAN and HDMI, more and more computers come with only one type of interface: USB ports. We must then buy adapters to connect the USB port of the computer with the CAN, RS232 or LAN port of the embedded device. USB adapters bring WLAN or Bluetooth Low Energy to computers that don’t have them built in. They also enable laptops to be connected to monitors with HDMI or DP cables.
USB standardises the physical interfaces (hardware) and protocols (software) for the communication between two devices. The adapter converts the mechanical, electronic and protocol characteristics of, say, a CAN connection into those of a USB connection. It adapts CAN to USB. The computer doesn’t know whether it transfers data via CAN, RS232 or LAN. The USB port does what good interfaces do: It hides the concrete technology and implementation of the adapter.
Thanks to the standardised USB port, the computer and adapter manufacturers can build and test their products independently. Different manufacturers can build the same type of adapter (e.g., USB-to-CAN adapters) and compete against each other. One manufacturer can build different types of adapters (e.g., USB-to-CAN , USB-to-RS232 or USB-to-LAN adapters). As long as the manufacturers diligently implement the USB specification, their adapters will work with all computers providing USB ports following the same USB specification.
The Ports-and-Adapters Pattern
The Production Perspective
When designed as a modular monolith, the HMI Terminal Application comprises the Application Core (blue hexagon), the ports UI, Machine and Accounting (red rectangles) and the adapters GUI, Voice UI, J1939 Machine and Local DB (orange rectangles). The blue line depicts the application boundary.
As the Application Core codifies the business rules, it is also called the business logic or business domain. The Application Core (hexagon) is the inside and the adapters are the outside of the application. The ports are the interfaces between the inside and the outside. An adapter converts data received from another system into an object defined in the port so that the Application Core can apply some business rules – and vice versa.
Let us look at ASR-1 when the Driver changes the cutting length from 3 mm to 5 mm.
- The Driver taps on a plus button twice to increment the cutting length from 3 mm to 5 mm.
- The GUI adapter converts the display value of the button into the cutting length and sends the cutting length to the UI port.
- The Application Core sends the cutting length to the Machine port whenever it learns about a change from the UI port.
- The Machine port forwards the cutting length to the J1939 Machine adapter, which encodes it in a J1939 message and sends it to the Machine via CAN bus.
For ASR-2 – displaying the engine speed – the Machine becomes the source and the Driver the target. The above steps are executed in reverse order and each step is reversed (e.g., encoding becomes decoding).
The most important responsibility of a port is to hide the technologies used in the adapters from the Application Core. Thanks to the UI port, the Application Core has no idea whether we implement the UI adapter with Qt, Slint, Flutter or any other UI framework. The UI port enables us to replace one UI framework by another one without the rest of the application noticing. The Application Core also has no idea whether the Driver interacted with the application by touch or voice to set the cutting length to 5 mm. The GUI and the Voice UI adapters work side by side in the application. They convert touch gestures and voice commands into actions provided by the UI port.
The strict information hiding of the ports almost inevitably leads to an important characteristic of the application: The Application Core and the ports are I/O-free. The adapters are solely responsible for I/O.
ASR-7 – Making Application Core and Ports I/O-Free
The Application Core and the ports are I/O-free. The adapters are solely responsible for I/O.
The accounting adapter, the Local DB, stores the harvested area (ASR-4) in a database, possibly in an SQL database. Neither the Accounting port, the Application Core nor any other adapter use SQL code. The only part that depends on an SQL library like SQLite or Qt SQL is the Local DB. Any other parts must not depend an SQL library. So, executing SQL queries and processing their results in GUI code is strictly forbidden.
The J1939 Machine adapter takes care of the J1939 communication with the ECUs (electronic control units) on the Machine. If we decide in the future to use MQTT over an Ethernet connection (ASR-3) instead of J1939 over a CAN connection, we develop an MQTT Machine adapter that conforms to the Machine port. We can then replace the J1939 Machine adapter by the MQTT Machine adapter on the HMI Terminal. As the rest of the HMI Terminal Application isn’t aware of this replacement, it works as before. Similarly, we can add ECUs, which communicate via CANOpen instead of J1939, to the Machine by adding a CANOpen Machine adapter and by extending the Machine port accordingly.
If the Application Core depended on any adapter, it would also depend on the specific technology or implementation of the adapter. This would make it a lot harder (read: more expensive) to replace or add an adapter. Hence, the Application Core never depends on an adapter. On the contrary, all adapters depend on the ports and, therefore, on the Application Core. All the orange and brown arrows go from the outside (the adapters) via the ports to the inside (the Application Core). The outside depends on the inside but never the other way round.
ASR-8 – Outside Depends on Inside
All adapters depend on their ports and hence on the Application Core. The Application Core never depends on an adapter.
We satisfy ASR-8 most easily by applying the Dependency Inversion Principle (DIP).
There are many ways to express the dependency inversion principle:
- Abstractions should not depend on details
- Code should depend on things that are at the same or higher level of abstraction
- High level policy should not depend on low level details
- Capture low-level dependencies in domain-relevant abstractions
The common thread throughout all of these is about the view from one part of your system to another; strive to have dependencies move towards higher-level (closer to your domain) abstractions.
Brett L. Schuchert, DIP in the Wild (emphasis mine)
If the Application Core called the function setCuttingLength(5) on the object HeaderEcu and the function wrote the J1939 frame with the cutting length of 5 mm to the CAN bus (ASR-1), our design would violate the DIP in two ways.
- As the J1939 Machine adapter containing
HeaderEcudepended on J1939 and CAN objects, the Application Core would depend on these low-level details as well. - The Application Core would assume the existence of an ECU for the header. However, this ECU may be integrated into a single central ECU together with many other ECUs. These technical details are irrelevant for harvesting maize, the business domain. The Driver need not know these details and neither does the Application Core.
As usual for a layered architecture, the dependency goes from a higher layer, the Application Core, to a lower layer, the J1939 Machine. The DIP tells us to invert this dependency. The Application Core should depend on an abstraction, the Machine port, and not on the concrete implementation, J1939 Machine. The ports belong to the Application Core. More general: Interfaces belong to the clients using them. As the ports and core change together, they are deployed together.
The ports are shaped by how drivers get their work done best. Their work is cutting maize such that the cows produce the most milk and the work costs are minimal. Depending on its ripeness, drivers must cut maize shorter or longer. If drivers go too fast, they may choke the engine. If they go too slow, they may not use the power of the engine efficiently. Drivers must keep the engine speed in the optimal range and adapt the cutting length according to the ripeness.
Based on our knowledge of the business domain, we could add an class CuttingMaize to the Machine port. This class notifies the Application Core about changes of the engine speed and allows the core to change the cutting length. The class J1939CuttingMaize, provided by the J1939 Machine adapter, implements the CuttingMaize interface. It receives the engine speed from the engine ECU and sends the cutting length to the header ECU.
J1939CuttingMaize knows which ECUs are connected to which CAN bus and which protocol the ECUs understand (J1939). In contrast, CuttingMaize has no clue about these things. However, it knows what the Driver needs. It knows the business domain. So, CuttingMaize is on a higher abstraction level than HeaderEcu.
The Testing Perspective
The ports-and-adapters architecture comes with built-in testing of the Application Core. We replace the production adapters (orange) by test adapters (brown). We use the Acceptance Tests as the UI adapter, the Mock Machine as the Machine adapter and the Mock DB as the Accounting adapter. The Application Core and the test adapters are compiled into one or more Test Executables.
We have learned above that the driver’s tasks define the responsibilities of the Machine and Accounting ports. The Driver starts an interaction with the application like changing the cutting length. In response, the Machine adapter changes the distance between the knives according to the cutting length. The port starting an interaction is called a driving port and the port responding a driven port. Driving ports like UI are shown on the left side of the hexagon and driven ports like Machine and Accounting on the right side.
For me, the distinction between driving and driven ports is still a bit hazy. It gets clearer when we try to test the Application Core through the ports. We write Acceptance Tests as test adapters for the driving ports (the input side). We write mocks for the driven ports (the output side).
For example, an acceptance test changes the cutting length to 5 mm (ASR-1). The Mock Machine records the received cutting length. The test verifies that the mock recorded 5 mm as the new cutting length. In case of an error, the mock responds with an error message received by the UI port. The test checks that the cutting length didn’t change and that the error message arrived.
We could argue that the Machine port is driving as well. In ASR-2, the Machine adapter starts the interaction by sending the engine speed. The acceptance test sends the engine speed to the Machine port, which forwards it through the Application Core to the UI port. A UI mock records the received engine speed. The test checks with the mock that the sent engine speed is the received engine speed.
The symmetry of the ports-and-adapters architecture would allow us to reverse the roles of the UI and Machine port. The consequence would be that the acceptance tests of the CuttingMaize functionality would be in two places: a sign of low cohesion. As the Driver decides how to respond to the engine speed, I would put the tests into the UI and not the Machine test adapter. Even if the CuttingMaize module were a fully automated system, the Driver would have the final say and could overrule the system. So, the UI port is driving and the Machine port is driven.
In the section The Production Perspective, we elaborated that the Application Core and the ports should be I/O-free. The test adapters should be I/O-free, too. This makes testing so much easier. We don’t have to chase intricate bugs because one test left a configuration file in the wrong state for another test or writing to the CAN bus failed for some strange reason. Our tests also run a lot faster if they don’t have to wait for responses from an SQL database or for the CAN bus timing out after 5 seconds in case of an error. The correct functioning of I/O is verified by running tests of a port-adapter pair on the HMI Terminal.
ASR-9 – Making Application Core Easily Testable
The Application Core and the ports shall be easily testable. This goal is easier to reach if the test adapters are I/O-free like the Application Core and the ports.
Let us look a bit closer at the test adapters. The UI test adapter, Acceptance Tests, is free of GUI code. If we start replicating code from the GUI adapter in the test cases, we found code from the business logic that leaked into the GUI. We shouldn’t ignore this feedback but act on it. We should move this code from the GUI into the Application Core and increase the abstraction level of the UI port.
Mock DB, the Accounting test adapter, could implement SQL tables as maps from a key to a data structure representing a record. Test cases can fill the tables with data as needed. Mock DB translates the functions from the Accounting port into queries of the mock database and returns the results through the port. Mock DB does not contain any SQL queries or any other I/O (ASR-4).
Mock Machine, the Machine test adapter, follows the example of Mock DB. It doesn’t know anything of J1939, CANOpen, CAN or MQTT. It could store parameter values like the cutting length or engine speed in a hash map. The implementation of the mocks should be as simple as possible.
The Team Perspective
[…] organizations which design systems […] are constrained to produce designs which are copies of the communication structures of these organizations.
Melvin E. Conway, How Do Committees Invent
Conway’s Law says that the software structure mirrors the communication or organisation structure. That’s why most software ends up as a big ball of mud. The way to avoid such a mess is to apply the reverse Conway manoeuvre. We first choose the architecture and then design the organisation structure according to the architecture. We are on the right track, as we have already chosen the loosely coupled and cohesive ports-and-adapters architecture. How could we divide the organisation into loosely coupled and cohesive teams?
The natural split is to have a team each for the Application Core, the Machine adapter, the Accounting adapter and the UI adapter. The Machine team develops both the Machine adapter for the HMI Terminal and the firmware for the ECUs. This team depends only on the Application Core team. Both teams collaborate on defining the Machine port.
In the field, the Machine team is often split into two teams. The first team develops the Machine adapter on the HMI Terminal, the second team develops the firmware for the other ECUs. The two teams communicate with J1939 messages over the CAN bus with each other. The first team depends on both the second team and the Application Core team. This at least doubles the collaboration efforts for the first team.
From my own bad experience, I know that the first team must find and work around the many non-standard “features” and bugs from the second team, even if this costs the machine manufacturers dearly. So, the best solution is to make the firmware team deal with both sides of the communication and hide their mess behind the Machine port. The Application Core team or a separate QA team could run acceptance tests against the Machine adapter. Having a single Machine team prevents the all too common finger pointing: “It’s your fault!” – “No, it’s your fault!”
For similar reasons, the Accounting team should be responsible for the end-to-end solution with the local and remote database and the synchronisation of both. All too often, this team is split into a client and a server team, where the server work is often outsourced. Communication problems between the teams become evident by bugs in the client-server communication. Buying an end-to-end solution such as AWS IoT, Azure IoT or Memfault IoT avoids these problems (ASR-5). The Application Core team would adapt the Machine port to the client of the chosen solution. This saves us an extra Accounting team.
The two UI adapters, GUI and Voice UI, might be developed by two different teams. They have little overlap and their implementation requires different skills. The two feature teams must only cooperate on the definition and refinement of the UI port.
Let us assume that our management decides that we should use MQTT over Ethernet instead of J1939 over CAN for the communication between the ECUs (ASR-3). After all, the HMI Terminal and the Accounting cloud use MQTT already. Some developers of the Machine team will work on the MQTT Machine, while the rest maintains the J1939 Machine. As the two adapters are independent of each other except for the Machine port, the developers of the MQTT Machine avoid the Sisyphean task of forever merging the changes from the J1939 Machine into their code. This increases the chances of finishing the implementation of the new adapter significantly.
Once the new adapter is ready, the team flicks the switch and deploys the MQTT Machine instead of the J1939 Machine adapter. The team could also deploy both adapters and let the HMI Terminal Application decide dynamically which adapter to use.
Most project teams for developing the ECUs (including the HMI Terminal) of an agricultural or construction machine are too small to split them up into four sub-teams. We can “save” one team by combining the Application Core team and the teams responsible for the driving adapters (just the UI team in our example) into a core team. This combination makes sense, as the driving adapters have a strong influence on the shape of the driven ports.
We can bring the number of teams down by one by buying an end-to-end IoT solution and by letting the core team integrate it as the Accounting adapter. This leaves us with the core and the Machine team. If these two teams have a total of 7-8 developers or less, it might make sense to start with a single team. The team members internalise the architecture and the development process. When the team grows, say, over 10 developers and is split up in sub-teams, the original team members pass on their knowledge to the new members.
My Talk at QtGreece 2023
At QtGreece 2023, I gave a talk based on this post. I explain in more detail how the ports-and-adapters architecture helps with I/O-free tests for the Core (slide 14) and I/O-based tests for the system (slide 15). I illustrate the team perspective with three new diagrams (slides 17 and 18). Here are the slides from my talk.
Great article! — Decoupling is very important in application architectures. Excellent ideas for accomplishing this.