Creating Unit Tests (with mocks and shunts!)

This thread is dedicated to the mocks and shunts section of my unit testing article.

As time goes on, I see less value in encapsulation at the class level, and more at the unit level, a unit here being one or more classes that really need to be intimate with one another. That may or not include the test class -- there are legitimate arguments on both sides of that debate. But what I'm saying is that fretting about opening up a member because you lose encapsulation is misguided, because you gain more than you lose.

I'ld like to see the Mock section extended slightly to talk about the way some other tools can help reduce the burden/annoyances of Mocks. Your narration kinda criticizes the direct coding of mocks and shows places where you had to stray from nice OOP-principles, and I've never felt that I had to stray when using tools to generate the Mocks programmaticly. In fact, the task of creating places to inject constructed mocks always seemd to improve the design of the code.

I think its good that you show how one would hand-code/develop a mock or shunt, but I think many of the downsides are eliminated through tools.

EasyMock has it's ups and downs. I think it is a smart tool, but I worry a little about ramp (all those little ramps add up a to a big one) and I worry about readability (the unit test code is about the same size, but I think the easy mock stuff is not as easy to understand - of cuorse, folks that use easymock all the time would probably differ with me on this).

One thing that I think could be mighty interesting, but I've never tried it, is to use reflection in a unit test to exercise private methods. I think that this might be able to be made simple enough to be okay ...

I wasn't suggesting highlighting any particular mock creator; but I think that mentioning that they exist and remove some of the drudgery from using mocks would be worthwhile in that part of your exposition. Having some tool whether its EasyMock or one of the others, or even a home-grown code generator, etc I feel helps you focus on "using" mocks to help you test instead of "writing" mocks to help you test. Under the former case, while a mock is slightly more complex than a "plain' unit test, a tester is still highly likely to use the mock. In the latter case, I've seen testers go to great ends to avoid having to write a mock and/or abandoning their test all together.

One thing that I think could be mighty interesting, but I've never tried it, is to use reflection in a unit test to exercise private methods. I think that this might be able to be made simple enough to be okay ...

In my project team, we've been using PrivilegedAccessor to test private methods by means of reflection.

Eric,

I think it's good that you bring up the mock tools that are out there. While I've yet to be too wowed by them, and tend to still roll my own, they do have a strong following. I think it is one of those things where there are up sides and down sides.

I think that the article, as is, is pretty huge. The presentation I stole all this from does have a blurb about using easy mock ... but something had to go.

In fact, I think if people walk away from the article knowing that there is a difference between unit tests and functional tests our world will be a much nicer place!

Originally posted by Roger Chung-Wee:

In my project team, we've been using PrivilegedAccessor to test private methods by means of reflection.

Roger,

Can you throw us together a wee code sample?

Paul,

I'm on leave at the moment, but I've made a note to reply after I get back to the office.

Originally posted by Paul Wheaton:
In fact, I think if people walk away from the article knowing that there is a difference between unit tests and functional tests our world will be a much nicer place![/QB]

Agreed. However perhaps the answer is to separate the article to separte essays -- one for each of the threads you opened here for feedback.

If the/a main goal is the the help clarify the vocabulary/characterization of tests then the stuff about directory structure, mocks/shunts, and event test creation is superfluous. Given the amount of discussion about alternate types of tests, and the amount of talk about why a unit test should/shouldn't/can/can't use certain features in the other thread it would seem worthwhile to expand that part of the essay to accomodate that.

It would seem that the mock treatment should be expanded and the directory area is minimal at best but could be fleshed out, etc

Eric,

I think that right here, right now, in this very thread is an excellent place to flesh out these issues.

If you think that all of the thoughts that result from such a discussion could be consolidated into one short, easy to digest article for the masses, you could write one up and submit it to the journal editor, Ernest Friedman-Hill. Annnnnndddd ... we have one of them thar wiki things ....

I agree - there is soooo much more to say. But I think my article conveyed the most important 95% - at least from the perspective of the problems I see in most shops.

When I get to the point that I have a rich suite of tests, and my test class names all end with "Test", then my IDE makes twice as many suggestions as I want it to. When my test class names all start with "Test", my IDE makes exactly the right number of suggestions.

I suppose it depends on whether you want to see all the Test... classes that you've written or whether you want to see if you've written a test for MyClass called MyClassTest.

Concerning your article, I've read it in its entirety. Thank you for writing it. What follows is very very wordy and I am sorry about that. But I really want to understand better. Please take these comments, not as critisisms - I mean absolutely no insult, it's just that it isn't clear to me. If you like, would you elaborate a bit? Thanks in advance.

ARTICLE:
"Does this class make you .... uncomfortable? It should. There are two things about this sort of class that bugged me the first time I was exposed to it: The class attributes are not private, and they have underscores in them. The first time I saw a mock object like this I was told "Your unit test code doesn't go into production, so it can cut a few corners." I dunno ... I want to only write first class code all the time! Not even an hour had passed and I needed to mock java.sql.Connection. 40 methods! Getters and setters for every parameter, return value and counter for every method? .... hmmmm .... thinking this through a little .... the reason we make the attributes private is for the sake of encapsulation - to hide how things are done on the inside so that we can change our business logic later without breaking a bunch of stuff that decided to tap into our innards. But that doesn't really apply to a mock, does it? By definition, a mock has zero business logic. Further, it doesn't really have anything that it didn't just copy from somebody else. All mock objects everywhere could easily be 100% generated at build time! ... So I sometimes still feel a little queasy about this, but in the end I always end up re-convincing myself that this is the best way. So it is still "first class code all the time" - it just smells a little off. But it smells better than if I did it the other way."

QUES/COMMENT:
This entire paragraph confused me, because in the end - after reading it several times - I could not get whether you were advocating or not the use of overriding private methods for testing. It sounds like yes, but not sure.

ARTICLE:
"If you are a good OO engineer, you should be hopping mad right about now! Oh sure, violating encapsulation in unit test code is mighty uncomfortable, but violating encapsulation in production code JUST AIN'T DONE! (in case you missed it, I took out the keyword "private", making the method "package private" - now, anything in the same package can see that method) Again, a long winded explanation might help smooth things over a bit. I'm going to save that for the forums and say for now: be ever vigilant about first class encapsulation in your production code ... but ... once in a while ... you might consider trading a dollar's worth of encapsulation for twenty dollars worth of testability. And to salve your pain/shame a bit, you might add a comment:

public class FarmServlet extends ActionServlet
{

//exposed for unit testing purposes only!
FarmEJBRemote getRemote()
{
return FarmEJBUtil.getHome().create();
}

public void doAction( ServletData servletData ) throws Exception
{
String species = servletData.getParameter("species");
String buildingID = servletData.getParameter("buildingID");
if ( Str.usable( species ) && Str.usable( buildingID ) )
{
FarmEJBRemote remote = getRemote()
remote.addAnimal( species , buildingID );
}
}

}
"

QUES/COMMENT:
Why are you overriding this in the read production class when you override it later on in the extended class 'FarmServletShunt'?

ARTICLE:
"Now for the grand finale! The actual unit test code!

public class TestFarmServlet extends TestCase
{

static class FarmServletShunt extends FarmServlet
{

FarmEJBRemote getRemote_return = null;
FarmEJBRemote getRemote()
{
return getRemote_return;
}

}

public void testAddAnimal() throws Exception
{
MockRemote mockRemote = new MockRemote();
FarmServletShunt shunt = new FarmServletShunt();
shunt.getRemote_return = mockRemote();

// just another mock to make
MockServletData mockServletData = new MockServletData();
mockServletData.getParameter_returns.put("species","dog");
mockServletData.getParameter_returns.put("buildingID","27");

shunt.doAction( mockServletData );
assertEquals( 1 , mockRemote.addAnimal_calls );
assertEquals( "dog" , mockRemote.addAnimal_species );
assertEquals( 27 , mockRemote.addAnimal_buildingID );
}

}
"

QUES/COMMENT:
Where is the class/interface 'TestCase'? Where is 'MockServletData'? Thanks and best regards,

~Bill

This entire paragraph confused me, because in the end - after reading it several times - I could not get whether you were advocating or not the use of overriding private methods for testing. It sounds like yes, but not sure.

That particular paragraph had to do with the mocks having non-private attributes and underscores. It does not address changing the scope of production code.

Why are you overriding this in the read production class when you override it later on in the extended class 'FarmServletShunt'?

If it is private, I cannot override it! I want to make sure that when the production code calls that method, it calls my shunt code instead!

Where is the class/interface 'TestCase'?

That is part of the JUnit framework. I guess I assume that you know about that already.

Where is 'MockServletData'?

I took a quickie shortcut by using the comment "// just another mock to make"

Repost - sorry, but no preview

Thanks you your reply. As you noted, I have no prior knowledge of JUnit, Though of course I am familiar with the very general term "unit testing" - I was trying to expand my horizons.

This entire paragraph confused me, because in the end - after reading it several times - I could not get whether you were advocating or not the use of overriding private methods for testing. It sounds like yes, but not sure.

That particular paragraph had to do with the mocks having non-private attributes and underscores. It does not address changing the scope of production code.

Why are you overriding this in the read production class when you override it later on in the extended class 'FarmServletShunt'?

If it is private, I cannot override it! I want to make sure that when the production code calls that method, it calls my shunt code instead!

Hiding the method with a friendly one won't due?

Where is the class/interface 'TestCase'?

That is part of the JUnit framework. I guess I assume that you know about that already.

No, I don't
Where is 'MockServletData'?

I took a quickie shortcut by using the comment "// just another mock to make"

Sorry, missed that

Not only is this calling other business logic, it's calling an application server! Possibly across a network! Thousands of these are gonna take more than ten seconds. Plus, changes in the EJB stuff could break my tests here! So a mock object needs to be introduced.

If I could just mock out all of the EJB stuff, I'd be sitting pretty. Hmmmm .... If the code were to somehow get my mock FarmEJBRemote, I'd be in fat city.

First, to create the mock. If FarmEJBRemote were a class, I would extend it and override all the methods. But since it happens to be an interface, I'll just make a fresh class and implement all the methods:

Part of the problem is that your code isn't written for maximum testability.

From its name, I'm guessing FarmEJBRemote extends EJBObject. Your code would be better off not knowing about EJBs or their create() methods at all. You could easily achieve this by using that J2EE design pattern (whose name escapes me right now) whereby you declare a business interface that has no relation to EJBs, and your EJB's remote interface extends from both it and EJBObject. You've now decoupled the controller logic in your servlet from the implementation of your business layer. Decoupling always improves testability (as well as buying you other benefits like maintainability). So now you have two interfaces that look like this:



Now your servlet only needs access to some implementation of FarmService and it's happy. No need for JNDI lookups (or asking a Service Locator to do one) and no need to call create() or handle the resulting CreateExceptions.

The next logical step towards more testable code is to inject the FarmService into your servlet, rather than having the servlet go out and find it. The servlet's job is handling requests, not looking things up. So now your servlet looks like this:



Now when you want to test your servlet, you can pass a mock FarmService to the setter. In production you would pass your real FarmService to the setter. You've used Inversion of Control (IoC, a.k.a. Dependency Injection, or DI), see http://www.theserverside.com/articles/article.tss?l=IOCBeginners for more details. There's no need to break the encapsulation of your servlet by exposing its properties or methods unnecessarily, and no need to subclass it just for tests. Apart from making your servlet more testable, you've also decoupled your web tier from the implementation of your business tier, so if you decide to switch from EJBs to some other approach, you can do so without touching your web classes.

My next suggestion would be to use a mocking tool to mock the FarmService. I used to write mocks by hand as you advocate, but it creates too much work, both in the initial writing and then in maintenance when a method signature changes. Packages like EasyMock really are easy to use, and since 2.0, the resulting code is a lot more readable, e.g. "expect()" this and "return()" that.

In case you're wondering who calls the servlet's setFarmService() method, this is where an IoC/DI framework really comes into its own. Personally I use Spring (www.springframework.org), but the ServerSide article I mentioned above provides links to some others.

I love writing testable code, and when I have trouble writing a test, I usually find it's because components are too closely coupled to each other (e.g. static method calls or insufficient use of interfaces). Observing proper separation of concerns and using DI usually solves the problem.

HTH,

Andrew

I like the fact that you distinguish between unit tests and functional tests. I think this is a key distinction.

However, I disagree about what is more important. Unit tests can help get the code working more easily, but it's the functional test suite that's really important.

Think about it for a minute ... suppose you have a beautiful suite of 2000 unit tests, all with nice mocks to insulate them from the other tests, that runs in 20 seconds. It tests every single method using mock objects. You run the whole thing frequently, every time you compile. What are you getting out of it?

Well, you're getting something out of the one test that tests the unit you're working on at the moment. With all the tests made fully independent through the use of mocks and such, though, the other 1999 tests are doing exactly nothing. You're not touching the code they test, so their results cannot change based on the work you're doing. You might as well not run them.

The time would better be spent running one integration test for the unit that you're working on. One integration test won't take any more time than a full suite of unit tests, and it does you a lot more good. It not only tests your own code, but also the interfaces with the code that your code uses. It's in those interfaces that the more difficult bugs generally hide; without integration tests, you'd never catch them at all. Then run the half hour integration suite before you check the code in, over lunch or something.

As for the case where one bug fix broke 47 tests - that can only happen if the 47 tests, or the units they were testing, were dependent on the bug that was fixed. If the tests are written correctly, those 47 units will need to be changed to reflect the fact that the bug they were depending on no longer exists. If the tests are written incorrectly, the tests themselves should be fixed. Making integration tests into independent unit tests just sweeps the problem under the rug, where the bugs can breed and come back to bite you harder later. Better to fix the problem now, even if it means fixing 47 more files.
[ March 30, 2006: Message edited by: Warren Dew ]

If you are a good OO engineer, you should be hopping mad right about now! Oh sure, violating encapsulation in unit test code is mighty uncomfortable, but violating encapsulation in production code JUST AIN'T DONE!

Generally speaking, unit tests are first-class users of the runtime code and deserve the same consideration as any other user. If your code is too inflexible for the tests to use, then you should correct the code.

But of course, in this case it'd be much much better to use IoC patterns (i.e. inject a mock or the real thing) to make the class easier to test as someone pointed out previously in the thread.

It should be mockRemote; not mockRemote():