14

I am implementing some tests for an existing Java Swing application, so that I can safely refactor and extend the code without breaking anything. I started with some unit tests in JUnit, since that seems the simplest way to get started, but now my priority is to create some end-to-end tests to exercise the application as a whole.

I am starting the application afresh in each test by putting each test method in a separate test case, and using the fork="yes" option in Ant's junit task. However, some of the use cases I would like to implement as tests involve the user exiting the application, which results in one of the methods calling System.exit(0). This is regarded by JUnit as an error: junit.framework.AssertionFailedError: Forked Java VM exited abnormally.

Is there a way to tell JUnit that exiting with a return code of zero is actually OK?

Improve this question
1

4 Answers 4

Reset to default
33

The library System Rules has a JUnit rule called ExpectedSystemExit. With this rule you are able to test code, that calls System.exit(...):

public class MyTest { @Rule public final ExpectedSystemExit exit = ExpectedSystemExit.none(); @Test public void systemExitWithArbitraryStatusCode() { exit.expectSystemExit(); /* the code under test, which calls System.exit(...) * with an arbitrary status */ } @Test public void systemExitWithSelectedStatusCode0() { exit.expectSystemExitWithStatus(0); //the code under test, which calls System.exit(0) } }

System Rules needs at least JUnit 4.9.

Full disclosure: I'm the author of System Rules.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Very nice! System Rules is great for testing command-line tools that often do things like System.exit(), write to System.out and so on.
Cool stuff. I misleed by the imports (org.junit.contrib.java.lang.system.ExpectedSystemExit;) in some examples. It's not in mvnrepository.com/artifact/org.junit.contrib but in the mvnrepository.com/artifact/com.github.stefanbirkner/…
8

How I deal with that is to install a security manager that throws an exception when System.exit is called. Then there is code that catches the exception and doesn't fail the test.

public class NoExitSecurityManager extends java.rmi.RMISecurityManager { private final SecurityManager parent; public NoExitSecurityManager(final SecurityManager manager) { parent = manager; } public void checkExit(int status) { throw new AttemptToExitException(status); } public void checkPermission(Permission perm) { } }

And then in the code, something like:

catch(final Throwable ex) { final Throwable cause; if(ex.getCause() == null) { cause = ex; } else { cause = ex.getCause(); } if(cause instanceof AttemptToExitException) { status = ((AttemptToExitException)cause).getStatus(); } else { throw cause; } } assertEquals("System.exit must be called with the value of " + expectedStatus, expectedStatus, status);
Improve this answer

1 Comment

That works, and FEST-Swing even provides its own NoExitSecurityManager. I think this will be my approach for now, since it involves no changes to existing application code. The only downside is that the application has an UncaughtExceptionHandler that pops up a bug-reporting dialog when the security manager blocks the exit call. The test still passes, but anyone watching the test might get the impression the application has crashed.
6

Could you abstract out the "system exiting" into a new dependency, so that in your tests you could just have a fake which records the fact that exit has been called (and the value), but use an implementation which calls System.exit in the real application?

Improve this answer

4 Comments

That is a possibility. I would like to get as much as possible of the test suite in place before making any changes, but I guess sometimes you just have to dive in and make a few careful changes to get the tests working.
@Ben: Hopefully in this case it would be a relatively small change - and I'd definitely prefer to do this than mess around with the security manager. Of course, you'll also need to make sure that the app's Swing thread terminates appropriately etc.
Looking at the code, it might be a bit more involved than I have time for just yet, but undoubtedly a good exercise in applying the lessons from "Working Effectively with Legacy Code". I'm not sure how to get the Swing thread to terminate appropriately: just dispose of the open Swing windows? It seems safer at this stage to fork a new VM for each test, which is why I am restricted to one test per class. I'm glad to receive better suggestions though.
@Ben: I believe that closing all the open windows will do it, yes. A long time ago it didn't, but I believe it does now.
5

If anybody needs this functionality for JUnit 5, I've written an extension to do this. This is a simple annotation you can use to tell your test case to expect and exit status code or a specific exit status code.

For example, any exit code will do:

public class MyTestCases { @Test @ExpectSystemExit public void thatSystemExitIsCalled() { System.exit(1); } }

If we want to look for a specific code:

public class MyTestCases { @Test @ExpectSystemExitWithStatus(1) public void thatSystemExitIsCalled() { System.exit(1); } }
Improve this answer

1 Comment

The instructions for maven at github are not working. Not able to make it work...

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.