I wrote similar thing for NMock2 some time ago:

http://blog.ashmind.com/index.php/2007/09/23/nmock-35/

But not for constraints.

The idea is good, the syntax does not look perfect yet.

Yup, love the idea of using lambdas for constraints.

I'm not quite clear from the post whether this is using expression trees to work out the expectations (how are they matched?) but it's certainly interesting.

And yes, calling it "LINQ to Mock" is very silly, as it has nothing to do with LINQ beyond using lambda expressions, as far as I can see. The term "LINQ" is vague enough already...

Jon

yes, the name is misleading. but "LINQ" sells! ;-)

LINQ is magic ... it makes everything warm and fuzzy and all nice 'n stuff!

Seriosuly - it does look interesting ... but at the end of the day - it is just a question of whether you prefer Moq syntax, Rhino syntax, or TypeMock syntax ... beyond that they do pretty much the same things ...

I think we shouldn't be stealing Daniel's thunder by any guise. He's done a great job.

Linq to Mock in my reading is more symbolic than semantic. And in a sense good humor. I think Ayande should lighten up.

The statement "You can do most of that with Rhino Mocks right now, I feel forced to point out" is not so collegial if not condescending, coming from the author of Rhino Mocks. Rhino Mock currently holds my recommendations right now and that's my preference for today. It was NMock 4 years ago. So, maybe it's Moq next ... who knows? A wise man said about 2000 year ago, "Let them grow together". The emphasis is on LET .. as in 'allow'!

Thanks for the feedback Ayande.

The reason we called it Moq (following LINQ conventions) is because all Moq does it does by using heavily System.Linq.Expressions namespace.

Yes, it might seem to some people that LINQ is just the syntatic sugar for querying, but in reality, that's just the tip of the iceberg. Lambdas and expression trees and lazy execution are the real meet. People will realize with time ;)

But other than that, it just seemed like a cool derivative name for a mock project that we were having a hard time naming :)

Finally, it's not clear to me from Rhino expectation what's going to happen:

Expect.Call( () => foo.DoInt(0) )

.Callback( (int i) => i % 2 == 0 )

.Return(1);

If I read that code, first thing I think is "so this is an expectation for a call to DoInt with a zero". But that doesn't seem to be the case, otherwise my test wouldn't pass on it. It's not clear to the non-Rhino user. I prefer Moq syntax better, as the intent is clear from the expectation itself and not a subsequent "callback"

Also, and I didn't mention it in the announcement, I'm not fond of the record/replay metaphor for setting expectations. And the fact that pretty much everything are static calls makes me somewhat uncomfortable.

And I'm not a Rhino Mocks user, nope. I used extensively NMock2 and read just a bit about the other frameworks around.

I think another important "feature" of Moq is that it doesn't support full interaction testing (at least explicitly, you can do it though callbacks and failed assertions in there). I think it's quite bad when using a mocking framework leads you to perform extensive interaction testing as opposed to state/classic testing. That's one critical drawback that no other mocking library tackles IMO.

I have more information on my concerns on that area at my blog at http://www.clariusconsulting.net/kzu

Thanks again, and I hope we all benefit from these experiments with the cool new features of C# 3.0!

Yup, agreed on it's more of a buzzword use ;).

I understand the record/replay model Rhino uses, I've seen that it's even introduced as a feature in the doc/wiki. And that's precisely what I feel is unnatural. If it gets time to get used to, then maybe it's not the right model after all. That's why I got into writing Moq: I wanted something that felt as close as possible to what I already do manually when I go through the progression explained in my post on "Mocks, Stubs and Fakes: it's a continuum" (http://www.clariusconsulting.net/blogs/kzu/archive/2007/12/21/47152.aspx)

When you go that route, you get used to passing delegates/lambdas as the implementations for your mocks. That's what I feel is most natural to any developer.

I agree about the parameter passing. Note that Moq also does explicit parameter passing, as all the It methods are generic and take the type of the constraint, doing compile-time resolution of the mocked interface overloads.

I think there's quite a bit more "magic" you could do with the compiler-generated translation between lambdas and expression trees. I have the feeling I've only scratched the surface. We'll see :)

I have experienced myself how hard a mocking framework ends up making your refactorings. I agree with every single concern Fowler expresses in his article on "Mocks aren't Stubs" (http://martinfowler.com/articles/mocksArentStubs.html) and not just after reading them, but after experimenting them in a real project. It's more of a psychological effect rather than a true limitation of the mocking framework you use: on the record/replay/verifyall model, you end up caring too much about the interactions, and subconsciously end up expecting/verifying everything, and then suffer the consequences.

That's why I prefer a simpler model, where verification is not built-in and is not "induced" by the framework, so that it will lead you to doing the right thing automatically, but allow you to do the weird things if you really want to (albeit a tad harder).

The verify can be implemented in Moq by performing assertions in callbacks to calls you don't want to happen. This more explicit model is better I think, and makes the test more readable. Such an explicit expectation shows precisely that there's something that should NOT happen because of an explicit design desicion that you have to honor when you refactor/redesign your code later.

An automatic/implicit failure from a reply/verify model is much weaker. Because you typically get a TON of failures upon refactoring because of the tight coupling between the expectations and the internal implementations, you will end up mechanically fixing all failing expectations, and maybe missing important by-design failures you wanted to avoid in the first place.

Daniel,

I have had given several talks and have a webcast that discuss proper use of mocking.

Like everything thing else, you can overuse it. I find that proper separation of mocks from stubs make it very easy to work.

Separating expectations from setup is also important.

I am not sure how you are going to verify that methods calls have actually been made.

You also need to take into account that there are styles of mocking. Some people like to use mocks as a last resort, and use state testing as much as possible. Other prefer the interaction approach.

It is a style & context issue.

Well, for us state/classic TDDers, it's not a matter of style. It's more a matter of a smell in your tests ;).

Verification with MoQ:

mock.Expect(x => x.Foo()).Callback(x => Assert.Fail("Shouldn't have been called"));

;)

The explicit lack of support for coarse-grained "VerifyAll" is a feature, as I see it :). I've written a bit more on the subject at my blog (http://www.clariusconsulting.net/blogs/kzu/archive/2007/12/26/48177.aspx)

It would be awesome to create a unified comparison using a common real-world scenario (maybe the new ASP.NET MVC framework?) so that people can see the various choices and their benefits...

MOCKFRAMEWORKS.COM is available :). Maybe we can setup a wiki where all mock framework owners can expose their examples based on a "standard" one... (just registered the domain... seems like a cool idea, hehe)

Expecting calls in specific orderings is an internal implementation detail of the object under test that should never surface on a unit test. I (and most classic TDDers) consider this one of the worst practices derived from using mock frameworks that allow such expectations to be set.

I've been there, setting highly detailed expectations that were nothing but a rewrite of the internal implementation in terms of expectations. Later down the road, every refactoring of the class internals that would perfectly preserve the semantics of the object under test cause a huge number of test failures because of that.

I do not want to go down that road, and that's why I think it's actually a feature not to support it. You can be sure that your developers aren't testing the wrong thing.

But it's very debatable topic and I respect people who think otherwise. As a team we've felt the pain (through NMock, and not precisely because of rename-refactorings and its use of strings) and stopped using mocking frameworks in later projects because of that tendency they cause.

We hope to be solving that with MoQ and its "limited" features.

A very simple sample, you must call the BeginUnitOfWork() before you can call the Persist() method.

It escapes me why you'd want to test such an internal implementation detail of (say) a Person.Save() method. Again, this is another example where you're thinking as a mockist and I'm thinking about the resulting state. My test would be to ensure that whatever got saved, can be retrieved. And that's it. If the class is internally using a UnitOfWork, ITransaction, IFooHelper, it's up to it, and a unit test should never dig into those internal details.

Your solution to that is to break the system entirely.

Our solution is not the result of pure speculation on what mock frameworks should be. It's the result of actual experience doing intensive TDD over 3+ years, most of them to the service of MS patterns & practices working on everything from CAB to EntLib to MCSF, etc. It's the result of trying various approaches that would help us in TDD, while at the same time avoiding the pitfalls of deep interaction testing. It's the result of using one such framework and seeing it cause bad practices (again, not because of the lack of strong-typing) because of the very same "features" all current mock frameworks support.

And it's the result of taking the natural progression I explain in "Mocks, Stubs and Fakes: it's a continuum" (http://www.clariusconsulting.net/blogs/kzu/archive/2007/12/21/47152.aspx) to the next level by turning the delegate passing into lambda passing with a better syntax for expectations, callbacks, returning results and throwing from methods.

I'm just now realizing how controversial the topic is. I think I'll propose to speak again at Teched US or Barcelona, this time about this :).

There are two problems in our discussion:

1 - You contradict yourself when you say I've misused the framework (and therefore I draw the wrong conclusions about the topic in general) by doing interaction testing (that's what I claimed, and which caused us pain) and later say it's perfectly OK (and therefore a GOOD use of the framework) to perform the kind of interaction test you showed as an example. I think you example is as bad for refactoring as any other using interaction testing. You claim that that kind of testing is OK. So I fail to see how you keep asserting that I've misused the framework.

2 - I think I understand why we're colliding: we have a semantic/definition problem. I think of mocks in a general sense, as testing aids, Test Doubles in Mezaros terms. I don't think regular developers are aware of the thin distinction between stubs, mocks, fakes, and dummies.

I believe (and according to Fowler's analysis of the two types of testing, most static/classic TDDers do too) that "dumber" mocks are actually better than "richer" mocks like what Rhino provides, because they are just helpers for classic/state testing which is more natural. I have the benefit over Fowler to actually have used a mock framework seriously for a project. But I wholeheartedly agree with his final thoughts on the topic:

"I don't see any compelling benefits for mockist TDD, and am concerned about the consequences of coupling tests to implementation.

This has particularly struck me when I've observed a mockist programmer. I really like the fact that while writing the test you focus on the result of the behavior, not how it's done. A mockist is constantly thinking about how the SUT is going to be implemented in order to write the expectations. This feels really unnatural to me."

avoiding the pitfalls of deep interaction testing.

Bias, anyone?

absolutely all opinions emitted by humans are biased IMO. so what? :p.

And I didn't bring CAB/EntLib for their technical merit (or lack thereof). In general I agree with your sentiment that we didn't dogfood it early enough... we only got to do it late in the project, and we only had enough time to refactor a few things that were much worse than the v1.0 you all know. I brought them up as further context on the kind of non-trivial projects where we were applying TDD extensively. And where we never felt we were not properly testing something because we were not using a mocking framework. That's telling to me.

That was a quote from http://martinfowler.com/articles/mocksArentStubs.html

1 - I'm talking about the example you brought to the discussion: the begin unit of work + persist calls. The fact that an object is using a "unit of work" component to perform its internal operations should be transparent to the unit test. It's an implementation detail IMO, so your example is as bad for refactoring as all the ones we experienced (which were very much in that vein). If you later decide that you'll use some other internal mechanism to perform a Save (say, you start using entity framework or whatever) you'll have to substantially revamp your unit tests when in reality the observable behavior and resulting state of the object hasn't changed in the least.

2 - I wonder how do your "dynamic mocks" differ from what you can do with MoQ. They are much more like stubs returning specific values set through expectations than a true mock which verifies the entire interaction. As you can see, there's a fairly gray area (the very same I believe results in the continuum between mocks and stubs) which most mockists try to ignore. Fortunately Rhino doesn't and supports that very gray area. That's a big plus for it IMO. So just skipping the Verify on the mock, you're basically using mocks as stubs.

"Regular developer", "average Joe", that "80% of the industry", those who don't read blogs, talk about design patterns, use TDD or mock frameworks, etc... I'm sure you know about them, don't you? Try asking around for the difference between a stub, a mock and a fake and you'll find them pretty easily.

If I were absolutely against any kind of interaction testing, I wouldn't have written MoQ in the first place.

The kind of basic interaction testing you show as examples is very easily achievable with it too. For example, the SLA one:

[ExpectedException(typeof(SlaViolationException))]

[Test]

public void ShouldThrowSLAIfElapsed10Seconds()

{

var externalService = new Mock<IExternalService>();

externalService.Expect(x => x.DoSomething()).Callback(() => Thread.Sleep(10000));

var obj = new MyObject(externalService.Object);

obj.DoSomethingWithExternalService();

}

That's perfectly fine, and the fact that there's no "verify" on the mock is the good part of it, as I see it. You're still testing the observable behavior of the object under test, which is to throw. You don't care (from an assertion/verification point of view) if the mock was called at all or not. It may be that internally this MyObject later uses some other mechanism to determine availability of the external service (i.e. a DNS lookup, a ping, whatever) and this test might continue to pass without modifications. If I were verifying the mock, I'd have to add more and more expectations to it as the internal implementation changes. That's the part I feel is not good.

The other two examples seem more of a case of a cross-cutting concern that wouldn't be implemented in the tested class itself, but through some kind of configurable workflow (you don't want the "SMS" hardcoded on the "should notify" behavior I guess) which could be tested in isolation through more classic testing too.

Especially the first case (unauthenticated page access) is more of an underlying framework feature than an app-level feature (i.e. ASP.NET itself does this for you). Framework testing is somewhat different to app testing, and I agree it may require a bit more of interaction testing than typical, as you need to ensure that configurable components are called at the right times. So the example is relevant if you're writing such a framework, yes.

However, according to Mezaros, a stub that records the calls it received is still a stub. If all you're doing is checking that a method was called, how is that different than a simple stub that gives you a boolean if the method was called and optionally the values of the last call? So all a mock framework is doing in that case you brought is little more than syntatic sugar for stubs or "trying to get to stubs in the long way around" as you said.

So I guess your example in that case is still one about stubs being (mis?)used as mocks ;). As you see, more gray area.

This discussion is convincing me more and more that the distinction between stubs and mocks is a superfluous one which doesn't hold in practice.

http://www.ayende.com/Blog/archive/2007/03/09/Querying-is-a-business-concern.aspx

Daniel,

I don't know about you, but I don't throw on SLA violations. I report them and collect the money.

The test is to verify that reporting the SLA is done correctly. The SLA report goes to a collaborator, which is the mock in this test.

The user login example is simple, yes. Try doing it with a real example where you have business logic determining whatever you can/cannot access something.

In the SMS case, I would use an INotificationStrategy and ensure that it was called correctly.

Sounds familiar?

About recording calls: that is the least that a mock does. It validate that the correct calls where made, with the correct parameters, with the correct order, etc. It can spit out appropriate responses based on needs, etc.

It is more than a dumb recorder.

Try doing it with a real example where you have business logic determining whatever you can/cannot access something.

You can do that with classic testing. i.e.:

[ExpectedException(typeof(UnauthorizedException))]

[Test]

public void ShouldThrowIfNoPermissions()

{

//...

x.GetBalance(...);

}

You don't need to assert/verify anything on the mock itself. All the test should care about is the result of the call to the object under test: an exception if the user is not authorized. How that was calculated or performed is not something that you should care, as it's the internal implementation details of the method. If you need to stub stuff so that collaborators return something meaningful to the method under test, that's fine. But I still fail to see why I'd care about verifying the mocks.

SLA violation: just yet another way of implementing it, I wasn't proposing any particular implementation as that's not the target of this discussion. You could log exceptions with a special logging rule that notifies whoever and records your money wherever. Don't care really. Again, you're thinking that the only way to test anything is through interactions with collaborators. I'm thinking on ways to design APIs so that they can be tested "by themselves", by their return values or exceptions thrown, etc. It's a mind state, so I doubt we'll agree regardless of how many more comments we write.

I understand your point and why you think the way you do. I also understand the trade-offs of my own thinking. So far, I still haven't seen any reason why I'd default to interaction testing as opposed to classic testing (and only use interaction when absolutely unavoidable).

Specifically, the example that I had in mind was ensuring that a TransferMoney calls BeginTransaction() before doing the money transfers.

IMO, you're testing the wrong thing. Instead of testing how the state is affected by the operation (i.e. is it atomic in case of exceptions? is it isolated? etc..) you're testing an interaction with an internal component which can change over time (may even be completely replaced) without changing the external behavior of the object. In such a case, I ALWAYS favor a test that keeps passing no matter how much I change the internals.

This applies to any such interaction test, and it's the main (inadvertedly?) "mind shift" that happens when you use mock frameworks extensively and don't think about these consequences up-front. The frameworks lead you down that path, which I don't believe is a good one except for very specific exceptions.

Certainly not for a security violation.

Fancy enough, specially given that .NET already does so for CAS security :p. But that's the topic of an entirely different discussion. Redirecting or not, how to do it, etc. is definitely not the topic here. (you could redirect from the security exception, just as ASP.NET does when you use role-based security throughout, although I don't even recall anymore how that worked, it seems like decades since I wrote that article http://65.19.150.102/Content.aspx?id=1232).

As you see, depending on the implementation you chose, we can argue forever on the approach to testing.

Thanks for engaging in the discussion, though. It's been very fruitful for me.

Daniel,

"The other two examples seem more of a case of a cross-cutting concern that wouldn't be implemented in the tested class itself, but through some kind of configurable workflow (you don't want the "SMS" hardcoded on the "should notify" behavior I guess) which could be tested in isolation through more classic testing too."

How do you test that your crosscutting concerns have been correctly addressed? Mocking out your mixins and interceptors and verifying that they are being called seems pretty appropriate to me?

/Mats

Security exception is a means of last resort, since it robs you the ability to actually do things.

I don't know how we ended up talking about this in a post about mocking frameworks.

Anyway, there's nothing intrinsic to using exceptions that prevents you from actually doing things. You can do your thing before throwing. Maybe I'm missing something.

partially trusted views

I fail to see how that has any relationship with an escenario where an invocation is issued without the proper permissions. You can totally do that, but it's completely unrelated to exceptions or CAS, and you can definitely use .NET role-based security from the thread principal to achieve that.

Even if you block access, you don't do it via exceptions, you redirect to a nice page that has the context of the problem.

So a piece of business logic in a library has to know about the concept of "redirect to a nice page"? Not in my world... bz logic is 100% agnostic of the UI it will be driving.

You can still throw an exception and have the UI controller catch and do the redirect to a nice page with the appropriate context (maybe even retrieved from the exception itself?).

But again, I don't know why are we even talking about this. I fail to see how this is related to mocking, or what's the point you're trying to prove. This is hardly a post/discussion about "proper .NET architecture" or anything like that.

How do you test that your crosscutting concerns have been correctly addressed?

If you look at CAB/EntLib unit tests you'll realize that both are classic TDD-driven. We didn't use any mocking library, and we never felt the "lack" of it. And remain assured there are plenty of crosscutting functionality implemented both in ObjectBuilder and extensions to it (i.e. CAB event broker, command infrastructure, etc.). There may be an exception or two, but as I continue to hold: interaction testing should be your last resort.

So a piece of business logic in a library has to know about the concept of "redirect to a nice page"? Not in my world...

Nor in mine. But it has a collaborator, ISecurityViolationService, which makes those decisions.

Do you see the pattern?

About security, I have a fairly long post about that. Suffice to say that I find most approaches to security lacking in real world business scenarios. I also see no point in throwing just to have an upper layer catch that. I can use a collaborator and have a far richer semantic interface.

I see... so all you need is a stub that records a flag that it was called... precisely what the big majority of .NET developers are doing anyway. That's precisely the realization I wrote about in my post about the continuum from stubs to mocks and fakes, which you didn't agree to.

And in that progression (and in this particular example about the ISecurityViolationService), I still don't see why I'd want to Verify the Mock (in the strict sense).

So you see, there's no need for that, it's still an unnatural jump from the straight progression you go through while doing TDD (without a mocking library to "help" up front). The mock in the ISecurityViolationService case is still there just to assure the test passes, and it can do its task with a minimalist approach like MoQ.

Daniel,

Why would I want a flag? I am expecting something to happen, I can verify that it did.

I think that you are taking the insistence on stubs too far.

At any rate, this is getting a bit too long for a comment thread, I suggest we will continue this in the altdotnet @ yahoo.com mailing list

Nothing will happen from a call to an interface, unless you figured out a way to put some implementation there ;-)

According to Meszaros definition, a Stub may also record calls it received.

So:

class StubSecurityViolationService : ISecurityViolationService

{

public bool ViolationCalled;

public void ViolationHappenedDoWhateverRedirectToNicePageEtc(...some context info...)

{

 ViolationCalled = true;

}

}

[Test]

public void ShouldCallSecurityServiceForViolation()

{

var stub = new StubSecurityViolationService();

var obj = new MyBzLogic(stub);

obj.DoSomethingWIthoutPermissions();

Assert.IsTrue(stub.ViolationCalled);

}

No mock verification, as you can see. That's what everyone not using a mock framework is doing, and I don't see anything wrong with that. And it fits perfectly the definition of Stub (although most will call this a "MockSecurityService", but that's the topic of another post - http://www.clariusconsulting.net/blogs/kzu/archive/2007/12/27/48594.aspx)

At this point, I believe each of us is considering the other as someone with a hammer that sees everything as a nail.

It doesn't need to scale much as it should be mostly the exception rather than the rule. The rule is more about testing state, again. But if you create mocks following the progression I explained in my blog, you end up with mocks that receive delegates for the interface implementations (you only need one per type), and in those delegate callbacks you can do assertions, etc.

Any mock library makes this substantially easy, of course, and that was the whole point of MoQ: taking the manual (but generic) approach of passing anonymous delegates around and turning it into a lightweight mock library. But as you can see, there's nothing that is out of the ordinary stub+recording. Mocks (in their strict sense) are overrated I think...

Regarding the mailing list: too bad we've so much written already :(

And I've not used Yahoo for the last few years either...