Opening seams for testing
While testing Rhino Service Bus, I run into several pretty annoying issues. The most consistent one is that the actual work done by the bus is done on another thread, so we have to have some synchronization mechanisms build into the bus just so we would be able to get consistent tests.
In some tests, this is not really needed, because I can utilize the existing synchronization primitives in the platform. Here is a good example of that:
1: [Fact]
2: public void when_start_load_balancer_that_has_secondary_will_start_sending_heartbeats_to_secondary()3: {
4: using (var loadBalancer = container.Resolve<MsmqLoadBalancer>())5: {
6: loadBalancer.Start();
7:
8: Message peek = testQueue2.Peek();
9: object[] msgs = container.Resolve<IMessageSerializer>().Deserialize(peek.BodyStream);10:
11: Assert.IsType<HeartBeat>(msgs[0]);
12: var beat = (HeartBeat)msgs[0];
13: Assert.Equal(loadBalancer.Endpoint.Uri, beat.From);
14: }
15: }
Here, the synchronization is happening in line 8, Peek() will wait until a message arrive in the queue, so we don’t need to manage that ourselves.
This is not always possible, however, and this actually breaks down for more complex cases. For example, let us inspect this test:
1: [Fact]
2: public void Can_ReRoute_messages()3: {
4: using (var bus = container.Resolve<IStartableServiceBus>())5: {
6: bus.Start();
7: var endpointRouter = container.Resolve<IEndpointRouter>();
8: var original = new Uri("msmq://foo/original");9:
10: var routedEndpoint = endpointRouter.GetRoutedEndpoint(original);
11: Assert.Equal(original, routedEndpoint.Uri);
12:
13: var wait = new ManualResetEvent(false);14: bus.ReroutedEndpoint += x => wait.Set();
15:
16: var newEndPoint = new Uri("msmq://new/endpoint");17: bus.Send(bus.Endpoint,
18: new Reroute19: {
20: OriginalEndPoint = original,
21: NewEndPoint = newEndPoint
22: });
23:
24: wait.WaitOne();
25: routedEndpoint = endpointRouter.GetRoutedEndpoint(original);
26: Assert.Equal(newEndPoint, routedEndpoint.Uri);
27: }
28: }
Notice that we are making explicit synchronization in the tests, line 14 and line 24. ReroutedEndpoint is an event that we added for the express purpose of allowing us to write this test.
I remember several years ago the big debates on whatever it is okay to change your code to make it more testable. I haven’t heard this issue raised in a while, I guess that the argument was decided.
As a side note, in order to get rerouting to work, we had to change the way that Rhino Service Bus viewed endpoints. That was a very invasive change, and we did it in less than two hours, but simply making the change and fixing the tests where they broke.
Comments
I've been working on some code for a socket library we need here that sits on top of the .NET Async sockets. Testing that's been interesting because allowing compression, SSL encryption/decryption and packing/unpacking of our protocol units on background threads is a goal. Not the real issue though.
I noticed that you put no timeout on your WaitOne call.
I found that sometimes in a multi-threaded environment I could break stuff well enough to never have the wait.Set() called. In that instance the timeout was what stops the suite from stalling forever and allows an error to be logged. Just a thought.
Have you guys played with CHESS yet for unit testing concurrency situations? Microsoft demo'ed it at the PDC.
It only works with MSTest at the moment, or standalone exe's, but it's really quite amazing. It tests many likely interleavings of threads by taking control of the thread scheduler.
research.microsoft.com/en-us/projects/chess/
I second Matt's suggestion for timeout-ing your waits.
In our multihreaded tests we use our own Assert.EventWasSet(), and Assert.EventWasNotSet() with default or optional explicit timeouts.
Comment preview