Unit Test Autodiscovery with CMake and Google Test

Since last year at the Product Strategy group at Canonical, we have shifted our focus to quality, testing and best practices development.

One commitment was to use a unified framework for Unit Testing and Continuous Integration so that we could move to Test Driven Development. For that, in the Compiz and Unity projects we have chosen to use CMake/CTest and Google Test.

Google Test comes with some code to integrate with CTest, such that you can run make test and the buildsystem will run each test in each test case independently, and give you per-test resolution rather than per-executable resolution. It looks something like this:

gtest_discover_tests (BINARY SRC COMMENT)

This allows for it to scan for tests in the source file SRC to run against the binary BINARY with a comment COMMENT at build system generation time.

This is great, and we’ve been using it to generate our “make test” command for quite some times. However, because it does the scanning at build system generation time and not at build time, the level of introspection provided is not complete and it has a few drawbacks.

  1. The code assumes that each test SRC has at least one TEST defined in it, otherwise it fails with a non obvious error (probably more or an implementation bug)
  2. The code assumes that there is no space between TEST and the opening parens, any spaces that lie there will be assumed not be tests and ignored
  3. No support for value parameterized or typed tests, which are really useful if you want to run the same test multiple times with lots of different value sets, or test an implementation conforms with the contract an interface is supposed to have (as they use templates)

As such, in about half a day of hacking, I wrote a quick program used to introspect the test binaries at runtime, so that these cases will be supported properly. As such, CTest will now show all the tests, and not just the ones supported by the source scanner.

You can find the implementation here.

 

4 thoughts on “Unit Test Autodiscovery with CMake and Google Test

    1. Oh interesting – looks like I did not specify a licence for it in the header file -.-. Massive fail on my part.

      That’s a hard question for me to answer, because the copyright in the file would be assigned to Canonical. I think it would be safest to assume that the file is GPLv3 licensed.

      I’m assuming that your project is closed source and that you don’t distribute the tests along with the rest of the project. Personally I think there’s an interesting question as to whether or not distributing the binary itself would be considered a “distribution” of a derivative work of that file, since that file has no part to play in the creation of the binary.

      However, if you take a conservative view and say that it does affect the binary, then I was considering re-writing that code under a more liberal licence and making it available on GitHub recently. I haven’t seen it in about two and a half years now, so it’d be a complete clean-room re-implementation.

      I can keep you posted.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s