Stability Adventures Part 1 – Adding unit tests to compiz

As part of the Quality Assurance commitment I am making for compiz, one of the biggest parts of this is unit and regression testing. Previously, compiz has had no such infrastructure for doing so and this has allowed for subtle regressions to be introduced into earlier revisions and then not become noticable until much later ones, making the regression very difficult to track down. One of the more annoying ones is one I have just finished debugging at 3:45AM, bug 804683, where due to the way that clock_gettime () works, it can return time values which are less than what it previously would have returned a few nanoseconds ago. Luckily, I have a fix for this. However, it was difficult to find and a lot of time was wasted debugging one of the problems that came out of it (wallpaper plugin auto-cycling not being called accurately).

As such, I believe it is time to start adding unit tests to compiz. There are a lot of fragile parts in the window manager and a change to any part of it can cause a number of different interdependent units to fail in very subtle ways. Without knowing what exactly is failing, it can be very difficult to determine the source of the problem after a change.

The biggest challege so far to adding unit testing to compiz is the sheer size of the CompScreen and CompWindow classes. The temptation has been to store all plain-old-data which was either going to be global or per-window inside of these classes. As such, a number of different processes within the classes are now interdependent upon each other and cannot be split out into separate unit tests without instantiating a CompWindow or CompScreen and this bringing in dependencies on X11 (defeating the purpose). For example, CompScreen handles options, file-descriptor polling, timeouts, plugin private storage, X events, image reading / writing, window properties read/write, grabs, output devices (monitors), multiple desktops, stacking, session events, window match synchronization, etc etc. The real failure here is demonstrated by the fact that it has nine friend classes because so much private data is needed by other classes.

At current, all the above listed items simply cant be unit tested because they are dependent upon CompScreen to function. As such, I will be splitting them out into separate objects using an anonymous namepsace with a global instance which CompScreen then sets as the “default” one, or a unit test can set as the “default” one. that way, we can test each module individually and also test interactions between modules without running compiz.

Part II will be about the X11-Application-From-Hell as a means of testing a window manager at runtime.


5 thoughts on “Stability Adventures Part 1 – Adding unit tests to compiz

  1. this sounds fantastic 🙂 i for one, would make use of this to do bug reporting, and i would always be testing against git master. i think proper compiz-specific debugging/testing tools are long over due, good on you!

    do still have plans of making the gnome3 script you had planned on making (in your june article), or (i would assume) this is better, anyway..?

    …as for fixes affecting other plugins, there is a lot of breakage in master even since you fixed the bad merge. – windows misaligned using “grid”, dialog pop ups crashing compiz, or drawing incorrectly, compiz crashes more frequently, sometimes even by just closing a window (animation effects?)etc.

    i’ve ended up switching back to june4th build again. (was the last stable build for me).

    when you have the unit testing framework done, you should post a blog on how to use it – so people like myself can help debug 🙂

  2. I’d be curious to see a backtrace from the crash on close, I can’t say I’ve ever seen that before.

    The dialog thing is a more complicated issue.

    1. i’ve recompiled the latest-git. and am still getting the messed up dialogue boxes (intermittently). i’m going to do some testing later, and GDB to see if i can see anything. but i haven’t been able to reproduce the ‘crashing on close’ problem. which is great and – i say this because i had switched around/disabled certain plugins/animations and you have also added one more commit i think.

      ..but atleast it’s not doing it anymore, i just wish i had been smart enough to do a backtrace then, rather than the approach i took. lesson learned. 😉

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s