My team sometimes discovers bugs in our build system where modifying one particular file doesn't trigger a rebuild of some dependent file.
While our continuous integration system (Jenkins) catches problems when building the whole project, we don't currently have any way to detect problems with partial rebuilds — other than developers getting temporarily confused when they make a change, rebuild, and don't see their change take effect.
As we're in the process of moving from qmake to CMake, now seems like a good time to add some tests of partial rebuilds.
Have you written similar tests? Did you use a shell script (touch, rebuild, grep), or are more advanced tools available?
The solution we arrived at was to write a https://behat.org/ backend to perform automated tests of our incremental builds. The Behat backend copies the project's source tree to a temporary folder, then runs a scripted test suite on it, which looks like this:
Feature: Incremental builds Check that modifying certain source files causes certain build steps to be performed (or not performed).
Scenario: Editing a core type header Given a completed build When I edit type/integer.h And I build
Then it should update lib/MyFramework.framework/Headers/integer.h And it should update lib/MyFramework.framework/MyFramework And it should update bin/MyApp.app/Contents/Frameworks/MyFramework.framework/Headers/integer.h And it should update bin/MyApp.app/Contents/Frameworks/MyFramework.framework/MyFramework And it should update the compiler's cache
Feature: Compiler developer mode Check that compiler developer mode properly reduces build dependencies, to reduce edit-compile-run cycle time when working on the compiler.
Scenario: Enabling compiler developer mode Given a completed build When I run CMake with arguments "-DCOMPILER_DEVELOPER=ON" And I build Then it shouldn't do anything
Scenario: Editing a compiler source file when in compiler developer mode Given a completed build When I edit compiler/compiler.cc And I build
Then it should update lib/libMyCompiler.a And it should update bin/my-compiler
But it shouldn't update the compiler's cache And it shouldn't update bin/something-compiled-using-my-compiler