
GIMP: Improved Graphical User Interface testing
The GIMP image editing software has introduced a new graphical user interface testing platform to detect bugs early in development.
GIMP has a lot of intertwined menus, dialogs and tools. When developers change a part of the interface, another component may stop working, sometimes with problems going unnoticed until users report them. To address this, a new GUI testing platform has been implemented that automatically tests the interface with each code change.
Testing limitations
GIMP previously had unit tests that checked small parts of code, such as how the program reads XCF files. However, these tests didn't open real windows, so they couldn't verify if dialogs opened correctly or if menu items performed their functions. Legacy GUI tests that directly interacted with core code often broke without detecting bugs and couldn't catch certain types of issues.
UI tests check the program as a whole, in the same way as you use it.
Test functionality
Each test starts a real GIMP instance, performs a small task, and then checks the result. Currently, there are 13 tests that verify various functions including program startup, dialog behavior, file operations, tool usage, color management, layer manipulation, undo/redo functionality, zoom controls, and preference saving.
Implementation details
The tests don't require a physical screen, instead using a hidden desktop environment that runs GIMP and is removed afterward. This allows tests to run on CI servers without display capabilities. Rather than analyzing screen pixels, the tests use accessibility information that GIMP provides to tools like screen readers, which also improves software accessibility.
The tests use dogtail, a Python tool for UI testing. To enable GIMP to run on servers without screens, a new feature was contributed to dogtail itself, benefiting other projects as well. Currently, tests only work on Linux systems (both X11 and Wayland) as dogtail doesn't support Windows or macOS.
Future development
While 13 tests provide a good foundation, GIMP has many more dialogs and tools that could be tested. Developers can contribute by writing tests for features they use most frequently. A guide on writing new tests and troubleshooting failures is available on developer.gimp.org. This testing infrastructure aims to help find bugs earlier and maintain GIMP's reliability for all users.