Mocks Are A Sometimes Food
There's an interesting pattern when you start doing TDD and trying to make your tests as atomic as possible1. First of all you wonder how anyone could ever get far with completely standalone classes that don't interact with anything - obviously a program needs some level of communication between classes. Then you discover Mock objects. These wonderful little gems allow you to have communication between classes but still test them independently. Pretty soon you're going on a cookie monster style binge session with mocks. Everything can should and will be mocked out and there's no longer any need to worry about making your classes keep to themselves, all those external dependencies can just be mocked out.
At some point though, you get an error about the max permspace being reached. You've created so many mock classes that the JVM no longer has room to load class definitions. What's worse, your atomic tests which should be running lightning fast because they're so atomic, aren't. This is the point where you learn that Mocks are in fact a sometimes food. They're a nasty hack which allows you to pretend the world is all happy and isolated even though it's not.
Now, there are certainly architectures you could use which would completely avoid the need for mocks while still allowing inter-class communication but it can become mind-bendingly difficult to work out where the heck those messages are going and the performance implications of breaking things up at this level is pretty severe. A completely separated design is often beneficial at a component level, but it's usually impractical at the class level.
So mocks are a useful tool to help test things that need to interface with other classes but are largely independent. I've taken to the rule of thumb that if I need to use more than one mock in a test I'm probably doing something wrong - if I'm using more than three mocks I really need to rethink my approach.
How do other people approach the use of mocks and other related techniques?
1 - I'm not yet sure, but perhaps it would be better to focus on making them run as fast as possible, rather than as atomic as possible. They would still need to be focussed enough to make it clear where the problem is instead of just showing that this is a problem.↩
September 13th, 2006 at 2:09 am
I worked at an organization that went Mock crazy. Every class (and I mean every class) was an interface with both a concrete implementation and a mock implementation. We had good test coverage, but I couldn’t help but get the sense that creating mock classes was something of a distraction from “The Prime Directive” (addressing real requirements). And, that’s about the sense I get of unit testing in general, TDD is a nice idea, but to really commit to it, you’ll tend to focus on testing before you’ve even fleshed out what a system needs to look like. I know it is contrary, but I don’t think to think that tests can only come first in the most mature system. there’s a fair amount of tinkering necessary before you can really settle on the right interface (and by interface, I don’t necessarily mean “Interface”)
Unfortunately, “Tests are a Sometimes Food”, and that isn’t by choice, it is by necessity.
BTW, my solution to the mock issue (too many classes) is to use EasyMock - instead of creating a Mock class, you create something of a dynamic proxy object and then you train it. I’ve found that it is a great way to achieve Mock objects without having to go to the trouble of creating Mock classes.
And, the PermGen space - I think it is only 64M by default? (might be 256M, I forget). Seems to me that the default is too low these days, I’ve run up against it recently in a web application with too many JSPs.
September 13th, 2006 at 8:30 am
Hi Tim,
We actually use EasyMock for nearly all of our mocks and we use the class extension to avoid having to create interfaces for absolutely everything - on the whole it’s fairly efficient to write the tests. The problem is that every time you create a new mock instance, it defines and loads a new class for that mock so you very quickly run out of permgen space. We’ve since developed a simple wrapper that reuses mocks which sped up our tests a fair bit and solved the permgen space problem.
We’ve also developed a couple of custom, hand-crafted mock classes for some key datastructure type classes - the document itself and the elements that make it up. The custom mocks are far more flexible than EasyMock for this particular case and it’s made testing easier and avoided the problem of our code being forced to stay within the bounds of what the EasyMock makes easy for us.
September 13th, 2006 at 4:20 pm
It’s very easy for mocks to get out of hand. I like the following paper (hope this link works) that describes in better detail how to get the most out of mocks. I prefer the use of JMock myself, but I see that EasyMock is moving to the same sort of Domain Specific Language patterns of expectations.
On a personal note, I reserve mocks for where I understand the boundary to be relatively unchanging, or I expect it to be stable going forward. I dislike using it for areas where I don’t understand the real interactions (I’d prefer to stub or get more integration tests going) because at least it will be more realistic. I definitely don’t think we should ever be coming domain objects.
Hope this helps.