Skip to content

QML Engine Deletes C++ Objects Still In Use – Revisited with Address Sanitizers

Two years ago, I spent three days to figure out why the driver terminal of a sugar beet harvester crashed (see my original post). The crash happened after going through the same six-step interaction at least four times. The reason was that the C++ code accessed an object that the QML engine had already deleted. For the last two years, I have heard a lot of good things about address sanitizers. When I threw address sanitizers at the old problem, they identified the problem right away. Recently, address sanitizers helped me to locate and fix some strange crashes on a legacy application. Sanitizers will be part of my debugging toolbox from now on.

Reproducing the Crash

The example application, which you can download from here, shows the same crash as the original driver terminal.

Press Open and Close a couple of times to reproduce the crash

You can reproduce the problem by repeating the following steps a couple of times.

  • Press the Open button. The application shows one of three customer names picked randomly.
  • Press the Close button. The application shows a blank screen.

On MacOS, the example application usually crashes by repeating the above steps 3-5 times. This matches the behaviour of the real-life application (a driver terminal for sugar beet harvesters) both on an embedded and a desktop Linux system. On a Linux system, the example application only crashes when I close it.

Understanding the Code

After startup, the application shows the yellowish screen on the left side. When users press the Open button, the onClicked handler of the Button in main.qml dynamically creates a CustomerInfo item and initialises the property customer of this item with the value returned by the function g_customerMgr.randomCustomer(). When users press the Close button in the screen on the right, the onClicked handler destroys the CustomerInfo item again. The QML code of the handler looks as follows.

onClicked: { 
    if (!view.isCustomerInfoShown) {
        loader.setSource("CustomerInfo.qml", 
            {"customer": g_customerMgr.randomCustomer() }) 
    } 
    else { 
        loader.setSource("", {}) 
    } 
    view.isCustomerInfoShown = !view.isCustomerInfoShown 
}

CustomerInfo.qml accesses the first and last name of the customer through the properties customer.firstName and customer.lastName and shows them on the screen.

Rectangle {
    property Customer customer 
    Text { 
        text: customer.firstName + " " + customer.lastName 
    }
}

Customer is a simple C++ class derived from QObject and is registered with the QML engine in main().

    qmlRegisterType("Customer.Models", 1, 0, "Customer");

The setSource() call in the onClicked handler

    loader.setSource("CustomerInfo.qml", 
        {"customer": g_customerMgr.randomCustomer() })

assigns the return value of the Q_INVOKABLE function randomCustomer() to the property customer of the CustomerInfo item. g_customerMgr is a C++ object of type CustomerManager, which is added to the root context of the QML engine in main() and is globally available in QML.

    engine.rootContext()->setContextProperty("g_customerMgr", 
        &customerMgr);

The function randomCustomer() selects one of three customers randomly and returns the selected customer.

Q_INVOKABLE Customer *randomCustomer() const 
{
    return m_customers[qrand() % 3];
}

Address Sanitizers

Finding the problem in the real application with 25K lines of code took me three days. And, I found only one problem. The application may contain some more memory problems. I tried Valgrind’s memory checker at the time. It produced thousands of false negatives and made the application run so excruciatingly slow that I gave up quickly.

Over the last two years, I heard and read many good things about address sanitizers. They sounded a lot more effective than the memory checkers of yore. When I had to find some strange crashes in a legacy application recently, I took address sanitizers for a spin – successfully. The simple application above is the ideal candidate to explain how to use address sanitizers.

GCC supports address sanitizers starting from version 4.8. I used version 7.4. You must add the option -fsanitize=address to the compiler and linker options. You should add the option-O1 or higher to make your application run reasonably fast. The application will run roughly 2x slower with the instrumentation for the sanitizer. You should also add the option -fno-omit-frame-pointer and perform a debug build (option -g) to get readable stack traces. A CMake file looks like this.

set(asan_options -fsanitize=address -O1 -fno-omit-frame-pointer)

target_compile_options(${PROJECT_NAME} PUBLIC ${asan_options})

target_link_libraries(${PROJECT_NAME}
    Qt5::Gui Qt5::Qml Qt5::Quick ${asan_options}
)

Rebuild the project in debug mode, run the application and press the Open and Close button a couple of times. If you close the application, the address sanitizer will print the following messages. I removed the long file paths for readability.

==483==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030000d7b60 at pc 0x000000405b24 bp 0x7ffebaa6fcd0 sp 0x7ffebaa6fcc0
READ of size 8 at 0x6030000d7b60 thread T0
    #0 0x405b23 in void qDeleteAll<Customer* const*>(Customer* const*, Customer* const*) qalgorithms.h:320
    #1 0x405c41 in void qDeleteAll<QVector<Customer*> >(QVector<Customer*> const&) qalgorithms.h:328
    #2 0x405c41 in CustomerManager::~CustomerManager() CustomerManager.h:25
    #3 0x402f87 in main main.cpp:16
    #4 0x7fcb9c2a982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #5 0x4029b8 in _start (OwnershipTransferFromCppToQml+0x4029b8)

0x6030000d7b60 is located 0 bytes inside of 32-byte region [0x6030000d7b60,0x6030000d7b80)
freed by thread T0 here:
    #0 0x7fcb9e3ca310 in operator delete(void*) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe1310)
    #1 0x40930e in Customer::~Customer() Customer.h:34
    #2 0x7fcb9d4c35da in QV4::MemoryManager::sweep(bool, void (*)(char const*)) (libQt5Qml.so.5+0xc65da)
    #3 0x8565ad6dffffffff  (<unknown module>)

previously allocated by thread T0 here:
    #0 0x7fcb9e3c9498 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe0498)
    #1 0x404ef6 in CustomerManager::CustomerManager(QObject*) (OwnershipTransferFromCppToQml+0x404ef6)
    #2 0x402be0 in main main.cpp:16
    #3 0x7fcb9c2a982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

The first line above says that the application ran into a heap-use-after-free error when accessing the memory address 0x6030000d7b60. The application accessed this memory address after the object at this address had been deleted.

The first stack trace shows the function calls leading to the offending memory access. When you close the application, the destructor of CustomManager deletes all Customer objects. At least one of the Customer objects was destroyed before and is now destroyed a second time.

The second stack trace explains where the guilty Customer object was destroyed the first time. The QV4::MemoryManager from the shared library libQt5Qml.so.5 destroyed the Customer object. QV4 is a hint to QML’s JavaScript engine version 4, which is part of the QML engine. So, some memory manager inside the QML engine deletes the Customer object.

The third stack trace shows the place where the guilty Customer object was created. The CustomerManager constructor creates all the Customer objects.

In summary, both the QML engine and the C++ code destroy the same object. Where does the QML engine take over ownership of the Customer objects? Where does the C++ world pass Customer objects to the QML world?

The only place in the example application is the second argument of this setSource call.

    loader.setSource("CustomerInfo.qml", 
        {"customer": g_customerMgr.randomCustomer() })

The documentation of Qt 5.12 or newer for Q_INVOKABLE guides you to the reason of the crash (emphasis mine).

If an invokable member function returns a pointer to a QObject or a subclass of QObject and it is invoked from QML, special ownership rules apply. See Data Type Conversion Between QML and C++ for more information.

When you follow the link in the second sentence, you’ll find these ownership rules (emphasis mine).

When data is transferred from C++ to QML, the ownership of the data always remains with C++. The exception to this rule is when a QObject is returned from an explicit C++ method call: in this case, the QML engine assumes ownership of the object, unless the ownership of the object has explicitly been set to remain with C++ by invoking QQmlEngine::setObjectOwnership() with QQmlEngine::CppOwnership specified.

Additionally, the QML engine respects the normal QObject parent ownership semantics of Qt C++ objects, and will never delete a QObject instance which has a parent.

Of course, the function randomCustomer() is the exception of the rule, as a look into the CustomerManager constructor reveals.

CustomerManager(QObject *parent = nullptr) 
    : QObject(parent) 
{ 
    m_customers.append(new Customer("Joe", "Smith"));
    m_customers.append(new Customer("Jack", "Miller")); 
    m_customers.append(new Customer("Harold", "Beck")); 
}

If you used the static function QQmlEngine::setObjectOwnership() to enforce C++ ownership for each Customer object, you would make CustomerManager depend on QmlEngine. CustomerManager is a non-visual model class for some visual QML components and should not depend on QmlEngine, which displays QML components.

This leaves you with the second option to keep ownership in the C++ world. The newly created Customer objects are parented to the CustomerManager. Then, the CustomerManager destructor must not destroy the Customer objects any more. The modified constructor and destructor look as follows.

CustomerManager(QObject *parent = nullptr) 
    : QObject(parent) 
{ 
    m_customers.append(new Customer("Joe", "Smith", this));
    m_customers.append(new Customer("Jack", "Miller", this)); 
    m_customers.append(new Customer("Harold", "Beck", this)); 
}

~CustomerManager() {}

An alternative solution is to change the Q_INVOKABLE function into a Qt property. The documentation of enum QQmlEngine::ObjectOwnership gives the crucial hint (emphasis mine).

Objects not-created by QML have CppOwnership by default. The exception to this are objects returned from C++ method calls; their ownership will be set to JavaScriptOwnership. This applies only to explicit invocations of Q_INVOKABLE methods or slots, but not to property getter invocations.

You don’t have to parent the Customer objects to the CustomerManager, if you use a Q_PROPERTY instead of the Q_INVOKABLE.

// CustomerManager.h
private:
    Q_PROPERTY(QObject *randomCustomer READ randomCustomer CONSTANT)

public:
    Customer *randomCustomer() const
    {
        return m_customers[qrand() % 3];
    }

The QML code changes to

   loader.setSource("CustomerInfo.qml", 
        {"customer": g_customerMgr.randomCustomer })

Note that the customer is not accessed by a function call randomCustomer() any more but by a property access randomCustomer (no parentheses at the end).

Building a Sanitized Qt

Jumping from QV4::MemoryManager to the memory manager of the QML engine probably looked a little bit like magic. If you build Qt with address sanitization turned on, the second stack trace will be much more meaningful.

0x603000072eb0 is located 0 bytes inside of 32-byte region [0x603000072eb0,0x603000072ed0)
freed by thread T0 here:
    #0 0x7f79f1abf310 in operator delete(void*) (libasan.so.4+0xe1310)
    #1 0x404a0f in Customer::~Customer() OwnershipTransferFromCppToQml/Customer.h:34
    #2 0x7f79ef4b2e77 in QV4::QObjectWrapper::destroyObject(bool) qml/jsruntime/qv4qobjectwrapper.cpp:1178
    #3 0x7f79eef1b0ba in QV4::MemoryManager::sweep(bool, void (*)(char const*)) qml/memory/qv4mm.cpp:931
    #4 0x7f79eef1c2b4 in QV4::MemoryManager::~MemoryManager() qml/memory/qv4mm.cpp:1183
    #5 0x7f79ef1910d2 in QV4::ExecutionEngine::~ExecutionEngine() qml/jsruntime/qv4engine.cpp:725
    #6 0x7f79ef0fe97b in QJSEngine::~QJSEngine() qml/jsapi/qjsengine.cpp:375
    #7 0x7f79ef923719 in QQmlEngine::~QQmlEngine() qqmlengine.cpp:1032
    #8 0x7f79efb28473 in QQmlApplicationEngine::~QQmlApplicationEngine() qqmlapplicationengine.cpp:244
    #9 0x402f4c in main ../OwnershipTransferFromCppToQml/main.cpp:18
    #10 0x7f79ed24982f in __libc_start_main (libc.so.6+0x2082f)

The stack trace explicitly mentions the QML engine and a JavaScript V4 runtime.

You add the option -sanitize to the Qt configure command. For example, the following configure command works fine for building a sanitized Qt on a Linux X11 system:

configure -sanitize address \
    -opensource -confirm-license -debug \
    -prefix /usr/local/Qt-5.14.0-Asan -xcb \
    -no-gstreamer -nomake examples -nomake tests -v \
    -skip qtwebengine -skip qtwebsockets -skip qtwebview

If you include the web modules, the build will fail. The error message suggests to add the option -feature-webengine-sanitizer to the configure command. The build of the web modules will probably fail again saying that something is wrong with the Ninja generators. That was the point when I gave up.

Integration into CMake Projects with Multiple Targets

The above CMakeLists.txt file is OK for a project with a single target: the executable of the application demonstrating the crash. Adding ${asan_options} to the target_compile_options and target_link_libraries commands of 20, 50 and more CMakeLists.txt files is cumbersome.

Asan configuration for address sanitizers in QtCreator

In QtCreator, we clone the Debug configuration and call it Asan. We add -fsanitize=address -O1 -fno-omit-frame-pointer to the variables CMAKE_CXX_FLAGS_INIT and CMAKE_EXE_LINKER_FLAGS_INIT. When CMake generates the the build system (e.g., make or ninja files), it combines the values of CMAKE_CXX_FLAGS_INIT and with those of the environment variable CXXFLAGS and adds them to each C++ compiler call. Similarly, it combines the values of CMAKE_EXE_LINKER_FLAGS_INIT with those of the environment variable LDFLAGS and adds them to each linker call.

Pressing the button Re-configure with Initial Parameters calls CMake to generate the build system. On the command line, we would run the following command.

$ cmake -S <source-dir> -B <build-dir> -DCMAKE_BUILD_TYPE:STRING=Debug \
      '-DCMAKE_EXE_LINKER_FLAGS_INIT:STRING=-fsanitize=address -O1 -fno-omit-frame-pointer' \
      '-DCMAKE_CXX_FLAGS_INIT:STRING=-fsanitize=address -O1 -fno-omit-frame-pointer' \
      <other options>

Then, we can build and run our tests and applications with the address sanitizers enabled.

If we generate the build systems for multiple configurations like Asan, Debug and Release with a single CMake call, we should use the configuration specific variables CMAKE_CXX_FLAGS_DEBUG_INIT and CMAKE_EXE_LINKER_FLAGS_DEBUG_INIT instead. We should also put the CMAKE_<LANG>_FLAGS_<CONFIG>_INIT variables in a CMake toolchain file.

Conclusion

My advice from the original post was to find all code locations, where ownership of C++ objects is passed to the QML world, in a code review. Besides being tedious and error-prone, this approach will only find a single class of problems.

Address sanitizers give us an automated way to find the ownership transfers from the C++ to the QML world and to find uses of dangling pointers in general. They find other problems like stack or heap buffer overflows, memory leaks and static initialisation order bugs (see this link). We get all these benefits by compiling and linking our executable with the option -fsanitize=address. Address sanitizers can help us to reduce our debugging times significantly.

Address sanitizers are most effective, if we have a test suite with high coverage for your application. We run the test suite with address sanitization turned on. We best do this in an automated way with our CI/CD pipeline.

Also Interesting

If you find address sanitizers (ASan) useful, you may want to have a look at memory sanitizers (MSan), undefined-behaviour sanitizers (UBSan) and thread sanitizers (TSan). MSan detects whether memory is read before its written. UBSan finds integer and float overflows and underflows, division by 0 and 0.0, null pointer accesses, out-of-bounds array accesses and a lot more. TSan detects data races between multiple threads.

You find my original post QML Engine Deletes C++ Objects Still In Use here. I recycled the sections Reproducing the Crash and Understanding the Code with slight modifications above to make the new post self-contained.

Tags:

1 thought on “QML Engine Deletes C++ Objects Still In Use – Revisited with Address Sanitizers”

  1. Siddharth M
    Reply

    Hey Burkhard,

    This was a really nice (nicely crafted) and very informative post.
    I sincerely thank you for sharing this knowledge with us! 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *