Server Grabs and getting stuck in _XReply

Here’s a quick reference to anyone in the future who ever gets stuck debugging some external library being stuck in _XReply

#0 _XReply

#1 ???

#2 some_library_do_thing

# 3 my_app_do_thing

Check if your app is using server grabs (XGrabServer, XUngrabServer).

Some third party libraries open up new X Server connections in the same process as yours. They shouldn’t do this, but they do anyways (graphics drivers are notorious culprits for this).

If you are holding a server grab and then call into a third party library which makes a synchronous call on its own X connection you will get a deadlock. This is because of how server grabs work – they prevent all connections from reading or writing any data until your connection has released the server grab.

When using server grabs (please don’t unless you really need atomic semantics around a series of X calls), always try to limit them to the closest possible subset of protocol requests you actually need atomic semantics around. Don’t just wrap some entire function in a server grab. Especially not if you plan to be calling callbacks which you don’t control or calling into libraries that are subject to change.

Also remember that XUngrabServer is not a synchronous operation – you will need to call XSync after using it in order to completely release the grab for other connections.

Coffee Party

I was glad to see Mark take leadership today and post an apology for the “tea party” comment. Thought leaders make mistakes and comments they later regret, as have I and plenty of others. While it is always important to hold leaders to account, it is also important to recognise when they have taken steps to address criticism directed towards them. I don’t have too much more to add, but I wholeheartedly agree with the final thoughts on that post, which I’ll leave here.

For the record, technical critique of open source software is part of what makes open source software so good. It is welcome and appreciated very much at Canonical; getting reviews and feedback and suggestions for improvement from smart people who care is part of why we enjoy writing open source software. There isn’t anything in what I said to suggest that I don’t welcome such technical feedback, but some assumed I was rejecting all feedback including technical commentary. […]

I was talking about criticism of software which does not centre on the software itself, but rather on some combination of the motivations of the people who wrote it, or the particular free  software license under which it is published, or the policies of the company, or the nationality of the company behind it. Unless critique is focused on improving the software in question it is pretty much a waste of the time of the people who are trying to improve the software in question. That waste of time is what I had in mind with the comment; nevertheless, it was a thoughtless use of an irrelevant label.

[emphasis mine]

I’ll add a quick thought in here: Much of the hysteria and partisan criticism which is levelled at Ubuntu these days is objectively false and we as a community have a duty to examine the state of public debate to ensure that we are not distracted from what we each individually want to achieve.  This does not excuse the fact that there will always be difficulty in creating a product identity from a culture that is free-form in nature, but we should always recognise that individual actors in the ecosystem have the right to use their resources as they see fit to contribute in a way that may only benefit themselves, so long as those contributions do not pose a real practical harm to other actors within the community.

On another note, I make no apology to the real “Tea Party”.

Of Tea Parties and Armchairs

I was a little taken aback today by Mark’s “tea party” comment on the Trusty Tahr release announcement today. To try and put it in as much context as possible:

Mir is really important work. When lots of competitors attack a project on purely political grounds, you have to wonder what THEIR agenda is. At least we know now who belongs to the Open Source Tea Party ;)  And to put all the hue and cry into context: Mir is relevant for approximately 1% of all developers, just those who think about shell development. Every app developer will consume Mir through their toolkit. By contrast, those same outraged individuals have NIH’d just about every important piece of the stack they can get their hands on… most notably SystemD, which is hugely invasive and hardly justified. What closely to see how competitors to Canonical torture the English language in their efforts to justify how those toolkits should support Windows but not Mir. But we’ll get it done, and it will be amazing.

I can tell you what the agenda of the Mir team is: speed, quality, reliability, efficiency. That’s it. From what I’ve seen on the smartphone, Mir is going to be a huge leap forward for gaming performance, battery life and next-generation display capabilities. So thank you for the many contributions we had to Mir, and to everyone who is testing it in more challenging environments than the smartphone. I’m enjoying it on my laptop and loving the gaming benchmarks for native Mir. So to that team, and the broader community who are helping test and refine Mir, thank you.

I was a little disappointed that this comment came out, since I really thought that the inevitable controversy that comes with any major direction-setting decision had come and gone. It was quite clear that it was aimed more broadly at Red Hat and Intel with the reference to systemd, which is unfortunate, because while there was a recent controversy surrounding Intel, I don’t think they are really the “tea party” that Mark wants to refer to.

For my non-Western or at least non-U.S based readers, the “tea party” is a far-right, decentralized and loosely associated political group that exists within the Republican party. They are (arguably) backed by some large and anonymous vested interests and have only performed the function of skewing debate about, and obstructing the implementation of some socially and economically “liberal” policies such as the Patient Protection and Affordable Care Act. Their actions are largely the reason why the non-essential administrative services associated with the normal running of government went unfunded for the past two weeks and why we almost had the United States default on its foreign debt on Thursday. The “Tea Party” uses its small political power to its maximum extent in order to manufacture crisis after crisis in order to destabilize a democratically elected government and house.

It is no surprise then that calling a group of people within a community a “tea party” is a very deliberate attempt to categorize them as a group of people whose sole purpose is to destabilize, and no doubt insulting to that group of people.

There is a tea party, but just not who you expect

I think Mark did pick up on something though, which is that a tea party does exist. I’m not going to name them or imply who they are, because they are a disparate group of loosely-affiliated people, so it would be a rather pointless exercise.

One thing I think Mark got confused in his comments was the difference between those who are heavily involved in the development of open source software projects on one hand and their vocal proponents (or opponents) on the other. I tend to find that these groups are usually mutually exclusive and it is only on limited occasions that you’ll find some overlap between the two. The reality is that the vast majority of us are far too busy working on our projects, whether that be Wayland, Weston, Mutter, KWin, Mir or something else to really care all that much about coming out on top of a political crisis with words. And we know all too well that its very difficult to change taken direction with words alone.

The “tea party” does exist though. It exists in a loosely-associated group of people I like to call “armchair free software developers”. These people feel a kind of emotional closeness to the community and to particular projects but are not involved in the day to day development of those projects, either because they don’t feel that have the skills required to do so, don’t want to make the effort to do so, or simply don’t have time (because they are busy with other and probably more important things). In order to resolve this relevant inability to contribute, they create political debate and drum up support and opposition to things as an ostensible “contribution” in order to maintain their self-deception about their closeness to a community or project.

Now in jest I say – the irony in Mark’s statements is that he’s just fulfilled this exact criterion of being an “armchair free software developer”, and the irony in my statements is that I’ve also fulfilled this exact criterion of being such an armchair developer. Mark can’t be involved with the day to day running of these projects because he’s too busy trying to run a venture with a lot of external influences. I can’t be involved with the day to day running of these projects because I’m too busy with my Law degree.

If you watch the logs of the #wayland or #ubuntu-mir IRC channels on freenode or their respective mailing lists, you’ll notice that its very rarely that anybody there makes political commentary on, or let alone even talks about, the other project. It is also very rare that they talk about Windows’ Desktop Window Manager, or OS X’ Quartz Extreme, or Android’s SurfaceFlinger, or any other “competing” project. Usually whenever that happens it is some new person trying to start an argument and they usually get ignored.

On the other hand, if you watch the Phoronix forums, Reddit or various blog posts and their comments, lots of people who have hardly anything to do with these projects seem to have a lot to say about them as well as the constant capacity to talk about them. The political commentary is typically quite shallow, makes reference to blog posts published months ago and is usually just spreading misinformation and hearsay. It does appear to make up the vast majority of the debate.

At this point, you might be asking what my credibility is on this. After all, this is just a blog post. I’ll be the first to say that I’m not involved in the day to day running of either project. After all – I have more pressing things to do with my time. I speak here as a developer who has been somewhat involved with both projects. I’ve made (very limited) contributions to both Mir and Weston, wrote the client code to support both display servers for a large media center project. I helped to write the backend for an important graphical toolkit for one of the display servers. More importantly, I speak as a person who is a personal friend of developers from both projects. Which leads me nicely on to the next section.

Why do “tea parties” and “armchair developers” matter?

One fallacy is to equate a project’s proponents and opponents with its developers. Another fallacy is to assume that because somebody does not contribute to discussion and debate, then they are not engaged or affected by its contents. There’s a really depressing feeling that comes with scrolling through comment after comment about your project, the fruit of your hard labor by the politically insatiable minds of Reddit or one of Mark’s blog posts about your “hidden agenda”.

Seeing hateful comment after hateful comment about compiz in an attempt to “contribute” to the debate by comparing it with other similar projects is what made me effectively give up on it and move on to other completely different projects in the middle of this year. In fact, I almost did so at the end of 2011 and didn’t because I was being paid to work on it. Why put all your effort into maintaining something when people constantly rag on about how its “garbage” or “tainted” or “CLA riddled” or “a corporation’s hidden agenda” or … you get the picture. The positive comments sprinkled here and there in a sea of partisan criticism manifesting itself as “contribution to the debate” only made my imposter syndrome worse. Those who manage the affairs of Intel, Red Hat and Canonical should take particular note of this. You are demotivating your employees by engaging in constant crisis and debate.

Every project that is somehow involved in all of this each has its own unique and important merits. Both Wayland and Mir have substantially innovated in the protocol space, development methodology and flexibility of display servers. They are both good things. Let their developers create and refine the fruits of their labor and don’t punish them with partisan criticism and never-ending political debate and crisis.

XBMC Wayland merged for next release

Support for wayland compositors is something that Cory Fields and I worked on over the past few months, and it has finally landed in xbmc git master. As of today and by the next release, you’ll be able to use xbmc as a fullscreen application directly on any wayland compositor (gnome-shell, weston, hawaii).

There’s some other interesting fixes and hidden functionality that might be of interest to more technically minded users as well since the last update.

Testing Framework

XBMC is probably one of the first standalone wayland clients that comes with a full unit and acceptance automated testing framework for its wayland backend. This means that we can use continuous integration systems, like team-xbmc’s jenkins instance, to get early warnings and ensure that the backend remains functional across different wayland client library versions (more on that later) and also ensure that other people don’t accidentally break a backend they most likely won’t be using for a while. Getting the backend under a high level of test coverage at both a unit and acceptance level was a key deliverable for my GSoC project – X11 alternatives like Wayland/Weston and Mir are only just coming to fruition now and there is a very small subset of users who will be using them on a day-to-day basis. This general lack of usage (for the time being) presents a substantial risk that the backend will be accidentally broken by a seemingly unrelated as very few people will be testing it. This all depends of course, on enabling, building and running-in-continuous-integration the tests by default, which is going to be part of some follow up work. The tests also provide a mechanism for people working on the wayland backend to quickly verify if they accidentally broke anything.

Implementation of the Testing Framework

The main test driver for XBMC is Google Test, and knowing gtest quite well, it was very straightforward to bootstrap some tests with that.

The amount of coverage we can achieve with the unit tests more or less depends on the architecture of the code. I made sure to write the wayland backend such that the architecture forms a directed acyclic graph with loose coupling in between the nodes. That is basically jargon for saying that each class or function in the system can generally be used with mock classes or data at test time and that data flows throughout the system in a single direction with no externally unobservable side effects. Take for example, the way that keyboard events are processed:



PollThread runs in the background for us and gathers up events from the event queue to be dispatched later. Once we want to process some events, CWinEvents::MessagePump (a static function that we have registered at build time) calls into our internal xbmc::wayland::Loop::Dispatch which, depending on the wayland client library version, delegates to a version specific event dispatch mechanism (more on that later). Through some magic in the wayland client library, this results in our registered proxy object (in this instance, Keyboard) having its callback function for the particular event we’re interested in (a key press or release on a surface) being called. Once Keyboard knows about this, it sends raw data about that event over to KeyboardProcessor which looks up a scancode and keysym in the registered Keymap (in this case, CXKBKeymap, implemented using libxkbcommon) and then converts it into an XBMC_Event, which is something that XBMC understands. It then sends that to a registered EventListener, for which the production implementation calls CApplication::OnEvent where control leaves our subsystem.

Two things to note about this description:

  1. Data flows from the top of the chart to the bottom of the chart. The same data in process never re-enters any other part of the system its already been into. This means that we can observe all effects in the system by looking at the outputs of each node.
  2. Each node knows very little about the node before it or after it. KeyboardProcessor receives raw data which is looked up in registered Keymap. So long as KeyboardProcessor gets some raw data and and Keymap can turn that into a keysym, then KeyboardProcessor won’t be affected by the operation of either.

This makes KeyboardProcessor, which probably does the bulk of the work, very easy to test. You register a fake keymap (which you control) with it and give it some fake keycodes and then expect the right XBMC_Events to come out the other end when it calls the registered EventListener::OnEvent function (which you also control). Indeed, this is exactly how the PointerProcessor tests work!

using ::testing::Values;
using ::testing::WithParamInterfacee

namespace xw = xbmc::wayland;

class WaylandPointerProcessor :
public ::testing::Test
StubCursorManager cursorManager;
StubEventListener listener;
xbmc::PointerProcessor processor;
WaylandPointerProcessor::WaylandPointerProcessor() :
processor(listener, cursorManager)
TEST_F(WaylandPointerProcessor, Motion)
const float x = 5.0;
const float y = 5.0;
xw::IPointerReceiver &receiver =
static_cast<xw::IPointerReceiver &>(processor);
receiver.Motion(0, x, y);

XBMC_Event event(listener.FetchLastEvent());
EXPECT_EQ(::round(x), event.motion.xrel);
EXPECT_EQ(::round(y), event.motion.yrel);


We’re able to call directly into the PointerProcessor::Motion event above and make sure that the registered listener gets an XBMC_MOUSEMOTION event as we expect.

This this effectively tests that we’re able to convert some raw data provided to a hypothetical wayland callback into an XBMC_Event, and runs very quickly (in microseconds), but it doesn’t test that the entire system works the way we expect. More importantly, it doesn’t test that if we actually receive a mouse motion event from a wayland compositor that it is going to be dispatched properly and then turned into the XBMC_Event that we’re looking for. For that, we will want something more substantial and less fine-grained.

Acceptance Tests Driver

If we really want to ensure that the whole system works for a particular usecase, you need to test the entire process from providing real external input to watching the output of your system at its boundary. For this, I took some inspiration from a project called wayland-fits which runs a series of automated test cases against real instances of weston and libraries like Gtk+, Qt and EFL.

For the purposes of testing a client, simulating things like keyboard and mouse interaction is a matter of sending those events to the client. Weston provides a fairly flexible extension API and amongst other things, allows you to easily define new private protocols to talk to clients that understand that protocol.

In XBMC’s acceptance tests then, we launch an instance of weston on a private socket name (effectively, just a randomly generated filename in /tmp/) with the “headless backend”, which does not use OpenGL at all and does not need access to the hardware mouse and keyboard or the underlying display framework. It allows clients to connect to it and to manipulate some state on the compositor side, but it doesn’t actually have any real outputs other than the events it sends back to clients. We then also tell Weston to load our private “xbmcwayland” module, which in turn registers a global protocol for clients that understand it to talk to.

This global protocol allows clients to make Weston “deliver” input events to themselves or manipulate state on the compositor side. For instance:

<request name="move_pointer_to_on_surface">
<arg name="surface" type="object" interface="wl_surface"/>
<arg name="x" type="fixed"/>
<arg name="y" type="fixed"/>
<request name="send_button_to_surface">
<arg name="surface" type="object" interface="wl_surface"/>
<arg name="button" type="uint"/>
<arg name="state" type="uint"/>

And used like this:

const unsigned int oKeycode = LookupKeycodeForKeysym(*Base::keymap,

On the module side, we’re going to get that request on the private protocol, and send back a real key event in response.

xtw::XBMCWayland::SendKeyToKeyboard(struct wl_client *client,
struct wl_resource *resource,
struct wl_resource *surface,
uint32_t key,
uint32_t state)
struct wl_client *surfaceClient = wl_resource_get_client(surface);
struct wl_resource *keyboard = m_compositor.KeyboardResource(surfaceClient);
struct wl_display *display = wl_client_get_display(surfaceClient);
uint32_t serial, time; GetSerialAndTime(display, serial, time);

wl_keyboard_send_key(keyboard, serial, time, key, state);

In the test, the only thing that we customized was the EventListener for KeyboardProcessor. Once we receive the event from Weston instance, we can test to make sure it got converted back into what we expected.

/* Inserts a "synchronization point" in the event stream so that we
* keep calling CWinEvents::MessagePump() until all requests that
* would have generated an event have finished processing their
* corresponding events on our side */

XBMC_Event event(Base::listener.FetchLastEvent());
EXPECT_EQ(oKeycode, event.key.keysym.scancode);
EXPECT_EQ(XBMCK_o, event.key.keysym.sym);
EXPECT_EQ(XBMCK_o, event.key.keysym.unicode);

And the test runs as expected:

[ RUN      ] EventQueues/InputEventQueueWestonTest/0.KeyEvent
[       OK ] EventQueues/InputEventQueueWestonTest/0.KeyEvent (340 ms)

Lots of other tests run this way too:

[----------] 1 test from CompatibleEGLNativeTypeWaylandWestonTest
[ RUN      ] CompatibleEGLNativeTypeWaylandWestonTest.TestConnection
[       OK ] CompatibleEGLNativeTypeWaylandWestonTest.TestConnection (343 ms)
[----------] 1 test from CompatibleEGLNativeTypeWaylandWestonTest (343 ms total)

[----------] 5 tests from ConnectedEGLNativeTypeWaylandWestonTest
[ RUN      ] ConnectedEGLNativeTypeWaylandWestonTest.CreateNativeWindowSuccess
[       OK ] ConnectedEGLNativeTypeWaylandWestonTest.CreateNativeWindowSuccess (339 ms)
[ RUN      ] ConnectedEGLNativeTypeWaylandWestonTest.ProbeResolutionsSuccess
[       OK ] ConnectedEGLNativeTypeWaylandWestonTest.ProbeResolutionsSuccess (336 ms)
[ RUN      ] ConnectedEGLNativeTypeWaylandWestonTest.PreferredResolutionSuccess
[       OK ] ConnectedEGLNativeTypeWaylandWestonTest.PreferredResolutionSuccess (344 ms)
[ RUN      ] ConnectedEGLNativeTypeWaylandWestonTest.CurrentNativeSuccess
[       OK ] ConnectedEGLNativeTypeWaylandWestonTest.CurrentNativeSuccess (340 ms)
[ RUN      ] ConnectedEGLNativeTypeWaylandWestonTest.GetMostRecentSurface
[       OK ] ConnectedEGLNativeTypeWaylandWestonTest.GetMostRecentSurface (340 ms)
[----------] 5 tests from ConnectedEGLNativeTypeWaylandWestonTest (1700 ms total)

[----------] 4 tests from AssistedEGLNativeTypeWaylandTest
[ RUN      ] AssistedEGLNativeTypeWaylandTest.TestGotXBMCWayland
[       OK ] AssistedEGLNativeTypeWaylandTest.TestGotXBMCWayland (348 ms)
[ RUN      ] AssistedEGLNativeTypeWaylandTest.AdditionalResolutions
[       OK ] AssistedEGLNativeTypeWaylandTest.AdditionalResolutions (341 ms)
[ RUN      ] AssistedEGLNativeTypeWaylandTest.PreferredResolutionChange
[       OK ] AssistedEGLNativeTypeWaylandTest.PreferredResolutionChange (350 ms)
[ RUN      ] AssistedEGLNativeTypeWaylandTest.CurrentResolutionChange
[       OK ] AssistedEGLNativeTypeWaylandTest.CurrentResolutionChange (331 ms)
[----------] 4 tests from AssistedEGLNativeTypeWaylandTest (1371 ms total)

[----------] 2 tests from WaylandPointerProcessor
[ RUN      ] WaylandPointerProcessor.Motion
[       OK ] WaylandPointerProcessor.Motion (0 ms)
[ RUN      ] WaylandPointerProcessor.MotionThenButton
[       OK ] WaylandPointerProcessor.MotionThenButton (0 ms)
[----------] 2 tests from WaylandPointerProcessor (0 ms total)

[----------] 8 tests from EventQueues/InputEventQueueWestonTest/0, where TypeParam = (anonymous namespace)::SingleThreadedEventQueue
[ RUN      ] EventQueues/InputEventQueueWestonTest/0.Construction
[       OK ] EventQueues/InputEventQueueWestonTest/0.Construction (330 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/0.MotionEvent
[       OK ] EventQueues/InputEventQueueWestonTest/0.MotionEvent (338 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/0.ButtonEvent
[       OK ] EventQueues/InputEventQueueWestonTest/0.ButtonEvent (351 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/0.AxisEvent
[       OK ] EventQueues/InputEventQueueWestonTest/0.AxisEvent (328 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/0.KeyEvent
[       OK ] EventQueues/InputEventQueueWestonTest/0.KeyEvent (340 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/0.RepeatAfter1000Ms
[       OK ] EventQueues/InputEventQueueWestonTest/0.RepeatAfter1000Ms (1442 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/0.NoRepeatAfterRelease
[       OK ] EventQueues/InputEventQueueWestonTest/0.NoRepeatAfterRelease (2543 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/0.Modifiers
[       OK ] EventQueues/InputEventQueueWestonTest/0.Modifiers (354 ms)
[----------] 8 tests from EventQueues/InputEventQueueWestonTest/0 (6028 ms total)

[----------] 8 tests from EventQueues/InputEventQueueWestonTest/1, where TypeParam = xbmc::wayland::version_11::EventQueueStrategy
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.Construction
[       OK ] EventQueues/InputEventQueueWestonTest/1.Construction (343 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.MotionEvent
[       OK ] EventQueues/InputEventQueueWestonTest/1.MotionEvent (353 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.ButtonEvent
[       OK ] EventQueues/InputEventQueueWestonTest/1.ButtonEvent (361 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.AxisEvent
[       OK ] EventQueues/InputEventQueueWestonTest/1.AxisEvent (367 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.KeyEvent
[       OK ] EventQueues/InputEventQueueWestonTest/1.KeyEvent (367 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.RepeatAfter1000Ms
[       OK ] EventQueues/InputEventQueueWestonTest/1.RepeatAfter1000Ms (1450 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.NoRepeatAfterRelease
[       OK ] EventQueues/InputEventQueueWestonTest/1.NoRepeatAfterRelease (2551 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.Modifiers
[       OK ] EventQueues/InputEventQueueWestonTest/1.Modifiers (371 ms)
[----------] 8 tests from EventQueues/InputEventQueueWestonTest/1 (6165 ms total)

[----------] 8 tests from EventQueues/InputEventQueueWestonTest/2, where TypeParam = xbmc::wayland::version_12::EventQueueStrategy
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.Construction
[       OK ] EventQueues/InputEventQueueWestonTest/2.Construction (350 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.MotionEvent
[       OK ] EventQueues/InputEventQueueWestonTest/2.MotionEvent (350 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.ButtonEvent
[       OK ] EventQueues/InputEventQueueWestonTest/2.ButtonEvent (356 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.AxisEvent
[       OK ] EventQueues/InputEventQueueWestonTest/2.AxisEvent (343 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.KeyEvent
[       OK ] EventQueues/InputEventQueueWestonTest/2.KeyEvent (353 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.RepeatAfter1000Ms
[       OK ] EventQueues/InputEventQueueWestonTest/2.RepeatAfter1000Ms (1444 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.NoRepeatAfterRelease
[       OK ] EventQueues/InputEventQueueWestonTest/2.NoRepeatAfterRelease (2581 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.Modifiers
[       OK ] EventQueues/InputEventQueueWestonTest/2.Modifiers (354 ms)
[----------] 8 tests from EventQueues/InputEventQueueWestonTest/2 (6131 ms total)

With all those tests, we’re able to get some fairly comprehensive functionality coverage and ensure that the backend isn’t accidentally stepped on (as long as people run the tests!).

Support for multiple wayland client library versions

Usually, this isn’t a problem for the vast majority of wayland clients. It is only a problem for xbmc because xbmc uses a game loop, a design going back to its Xbox origins. The game loop structure means that any external function that could cause the program to stop and wait indefinitely for external input of a certain kind (blocking) cannot happen in the main thread and must be done in a separate thread.

The relevant blocking operation here is reading the wayland socket to see if there are any events, or at least, going to sleep until there are events. We can’t just fcntl (s, F_SETFL, O_NONBLOCK) the socket’s file descriptor, because it is the client library is the one reading the socket and we can’t really guarantee that it will handle -EWOULDBLOCK in a consistent manner across versions. A quick code audit showed that at present, it isn’t handled and there are probably a number of assumptions in place that read() from the socket will block if there is nothing to read. As such, if we want to read the socket without potentially blocking xbmc forever, we need to put that into a different thread and then set a flag to notify the main thread when there are events to be dispatched.

In wayland versions 1.2 and above, the new functions wl_display_prepare_read and wl_display_read_events were added. These functions read the client socket without calling any of the registered callback handling functions until wl_display_dispatch_pending is called. This means that we can read in one thread and dispatch read events in the main thread. That’s great, but we need to support older wayland versions as well.

In wayland 1.1 and lower, the only function which would read the event queue was wl_display_dispatch, which would also dispatch events once they were read. This would have the effect of dispatching events in a non-main thread, which is undesirable for a number of reasons and would result in numerous data races or other weird behaviour.

The former implementation is the superior one for a number of reasons, namely that we can keep all the event handling on the main thread and allow ourselves more room to move in terms of adding small side effects to functions. At least for the latter, we need to get all the events back on to the main thread before they leave the wayland backend and go back into xbmc.

As such, we have two event dispatching implementations, which each have three purposes:

  1. Creating a thread capable of reading the wayland socket
  2. Dispatching events from the wayland socket to our registered wayland object listeners
  3. Getting the processed XBMC_Events back on to the main thread for later processing outside the backend.

The implementation is chosen at runtime based on the functions available in the wayland client library. The version 1.1 and lower implementation reads and dispatches the initial events internally on a second thread, which then sends them to the main thread for dispatch into the rest of xbmc. The version 1.2 and higher implementation reads the event queue on a second thread, and then dispatches the initial events on the main thread and forwards them on to xbmc from there.

Because we’re able to replace this strategy at runtime, we can run all of our functional tests over these two implementations to check that the resulting output is the same:

[----------] 8 tests from EventQueues/InputEventQueueWestonTest/1, where TypeParam = xbmc::wayland::version_11::EventQueueStrategy
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.Construction
[       OK ] EventQueues/InputEventQueueWestonTest/1.Construction (343 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/1.MotionEvent
[       OK ] EventQueues/InputEventQueueWestonTest/1.MotionEvent (353 ms)
[----------] 8 tests from EventQueues/InputEventQueueWestonTest/2, where TypeParam = xbmc::wayland::version_12::EventQueueStrategy
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.Construction
[       OK ] EventQueues/InputEventQueueWestonTest/2.Construction (350 ms)
[ RUN      ] EventQueues/InputEventQueueWestonTest/2.MotionEvent
[       OK ] EventQueues/InputEventQueueWestonTest/2.MotionEvent (350 ms)

GSoC Mini-Promotion

You may or may not be aware that this project was the result of my participation in Google Summer of Code 2013. Google Summer of Code (or GSoC for short), is a programme run annually by Google which incentivises currently enrolled university students to participate in Open Source projects. Students work directly with nominated mentors from participating projects to implement a localized feature or substantial change that would take around two months to complete. Students receive a modest stipend from Google so that they can support themselves and devote their attention to the project instead of taking up summer jobs.

Even if you’re already quite involved in Open Source project work already, GSoC is a highly rewarding experience. It provides a decent incentive to participate in a project that you might not have otherwise participated in and in this way allows you to expand the breadth of your knowledge in a particular area. It is running for its tenth year in 2014, so I highly recommend you check out the schedule and register to participate next year!


I tend not to comment all that much on goings-on these days since I’ve lost friends by doing so in the past. However, I feel as though it would be dishonest for me to refrain from saying this:

Playing politics with source code repositories is an incredibly immature and unprofessional way of staging a protest against you don’t like. Every time this happens, greater numbers of the silent majority of FOSS developers become increasingly disillusioned and choose to emigrate to other communities, and other platforms.

A culture of negativity and exclusivity will ultimately drive your most impassioned developers away.

Sharing the love

Tim Penhey made an excellent post this morning where he took some of his time to thank those who contributed to who he was an his professional development.  I think that his post makes a very good post that us software engineers don’t become who we are in isolation – we are part of a much broader network of other people and our development is often shaped by the activities and advice of those around us. I wanted to take this post to share a few thank-you’s of my own to former mentors and coworkers who showed me a lot.

First of all, thank you to the original compiz team – Danny Baumann, David Reveman, Dennis Kaspyrzk, Jigish Gohil, Erkin Bahceci, Guillaume Seguin. You were all my original mentors who got me into programming in the first place and its been a wonderful ride ever since. Thanks for putting up with fifteen-year-old me and some of my dramatic behavior at times! But thanks even more for believing in me and giving me a chance to go on and make further contributions to compiz and other projects. It was something I really needed at the time and gave me a real head start.

Further thanks go out to some of the people I worked with on some of the hobby projects I did. Rodolfo Granata – thanks for writing the original Freewins plugin and thanks for taking my (massive) patches to make it do even more. Kevin Lange – thanks for working on the original wiitrack Wii-remote headtracking plugin and thanks even-more for working with me to get it working with actual hardware.

To my buddies at Canonical – thanks Jason Smith for being an all-round awesome dude and putting yourself on the line to recommend me for working at Canonical. I enjoyed working with you on every single damn project we worked on together, whether that be Compiz, Nux, Unity or whatever. Our late-night debugging sessions were awesome and you were supportive all the way. I always loved geeking out and talking about OpenGL and such with you.

To Neil Patel – thanks for being an all-round awesome technical lead. Neil was the one who stood up for us when we had a “massive axe placed over our heads” to deliver a newly rewritten based-on-compiz-and-nux Unity for Ubuntu 11.04.

To Didier Roche – thanks for showing me the ins and outs of Ubuntu and Debian packaging, a world that I totally would not have been able to understand without you. Thanks for always giving me another chance, especially when there was a last minute bug or tarball that wouldn’t build or whatever.

Robert Carr – thanks for being a general all-round awesome hacker who showed me that you could always do something in novel ways, and furthermore, thanks for being an awesome friend.

Tim Penhey – you didn’t think I was going to leave you out did you? Tim was an awesome line manager. Not only did he stand up for his engineering team (which was massive), he also worked with all of us to turn us into really smart engineers. I’ve got about 12 books (and counting!) on software in my library now and its all thanks to Tim’s recommendations. Tim taught me how to go from a hacker to a software craftsman by writing good, clean and lean code, test driven development and consistent (i.e. non chaotic) development process. I could always rely on Tim for advice on how to tackle a particular problem or improve myself.

Alan Griffiths – I was incredibly privileged to work with you, and the people whom you work with at Canonical still are. Alan taught me the fine details of getting tests in place around almost everything and his “you touch it, you test it” mentality really changed the way I thought about writing code. Alan also helped me to write a lean C-based object system for compizconfig, which is the very reason why things like the compizconfig gsettings backend (along with its desktop integration code and others) has 98% test coverage, which is remarkable for software written in C.

Daniel van Vugt – thanks for showing me that there was always another way of doing things. We may not have agreed on everything, but that is certainly a good thing. Daniel showed me that there are ways to write good, clean, testable software without it becoming enormously complex and hard to read.

Marco Trevisan, Andrea Azzarone, Brandon Schafer – thanks for being awesome colleagues and working with me to produce great stuff. We were in the same boat for a lot of things and all learnt new stuff together.

Of course, this list doesn’t encompass everyone. There are people whom I worked with from time to time both at Canonical and on compiz which definitely had an impact upon my professional development as an engineer and if I listed all of you I’d end up with a blog post several pages long!

The end result of years of engineering process isn’t just a final product that you see on a screen. Its also strongly-forged friendships of all the people who were there every step of the way.

Experiencing the Mir client API

Now that XBMC has display, mouse and keyboard support on Mir, I feel somewhat qualified to write about using the Mir client API, from the perspective of porting an existing application.

From a high level, the Mir client API (affectionately known as mir_toolkit in libmirclient) follows some of the basic constructs that you would find in Wayland. Namely, you have a MirConnection which is responsible for application-level communication with the display server, and a MirSurface which is responsible for managing some remote surface, drawing into it and handling events. Aside from MirWaitHandle, those are the only two objects that you’ll be concerned with, so its a little unlike Wayland which splits its “Surface” concept up into separate interfaces like wl_surface, wl_shell_surface and wl_egl_window(*) and its “Connection” concept up into wl_displaywl_registrywl_compositorwl_shell etc.

(*) Technically, wl_egl_window is not an interface but just a wrapper around wl_surface and wl_buffer which uses wl_drm under the hood.

Synchronously opening a connection and creating an OpenGL(ES)-renderable surface in Mir is a simple-enough affair:

MirConnection *connection =
  mir_connect_sync (path_to_socket, app_name);

MirSurfaceParameters parameters =

MirSurface *surface =
  mir_connection_create_surface_sync(connection, &parameters);

EGLNativeDisplayType nativeDisplay =
EGLNativeWindowType nativeWindow =

From there, all you need to do is just initialize EGL, create an EGLDisplay with eglGetDisplay, create an EGLContext with eglCreateContext, create an EGLSurface with eglCreateWindowSurface, and then call eglMakeCurrent and start using OpenGL(ES) to render to your surface.

Usually Asynchronous

Earlier I mentioned that a MirConnection and MirSurface can be created synchronously, but they can also be created asynchronously too. A callback can be registered on mir_connect and mir_connection_create_surface which will be called as soon as the connection or surface is ready. Your application can do other initialization tasks which don’t depend on a MirConnection or MirSurface while it waits for the server to return them.

Pervasively Multithreaded

One of the really interesting things about these callbacks and other callbacks, for example, the callback provided through MirEventDelegate to mir_surface_set_event_handler, is that your callback gets called in a completely independent thread. This means that there’s no main loop to spin or poll on – the display server will wake up the thread that’s interested in the particular data that its sending.

I believe that Mir is unique here in that no other display server has been designed to work this way. Doing it this way has some immediate benefits – handling input in applications can be somewhat expensive, because your application will often need to go through some complicated process to look up what to do in response to some particular combination of input. Instead of blocking the main thread, all this can happen in a worker thread which then tells the main thread what to do.

Unfortunately for legacy applications, the vast majority of which are single threaded and pretty much all of which do input and display handling in one thread, it means that the author of the application needs to lock a queue, push the incoming data on to that queue and then unlock the queue (and vice-versa in the main-thread) in order to preserve compatibility with other windowing system backends. The client library’s documentation provides that you as the author are responsible for thread-safety.

 * Set the event handler to be called when events arrive for a surface.
 * \warning event_handler could be called from another thread. You must do
 * and locking appropriate to protect your data accessed in the
 * callback. There is also a chance that different events will be
 * called back in different threads, for the same surface,
 * simultaneously.
 * \param [in] surface The surface
 * \param [in] event_handler The event handler to call
void mir_surface_set_event_handler(MirSurface *surface,
 MirEventDelegate const *event_handler);

Thankfully, there aren’t going to be too many consumers of mir_toolkit other than application toolkits or other special-case cross platform applications like XBMC. This design also opens up some interesting future possibilities. I’m looking into doing some of the (somewhat time-consuming) keysym-to-internal-keysym processing inside of the worker thread before handing it off to the XBMC main thread as a kind of demonstration of how that would work.