205

I've a Maven build in which I use the SureFire plugin to run some unit tests, and the FailSafe plugin to run some integration tests. I would like a way to run just the FailSafe plugin's tests.

It's not a good solution for me to add different profiles or anything in the pom, because it's a multimodule build and I don't want to have to edit every module's pom.

There are skip.tests and maven.test.skip and skipTests which stop all tests, and skipITs, which stops only the failsafe plugin.

So, is there a command-line flag for Maven like skipITs, but instead with the functionality of "onlyITs"?

Improve this question
9
  • 1
    @khmarbaise in theory, yes. But in most projects I have worked in, the "unit tests" where actually integration tests with an in-memory db (if you were lucky) Commented Jul 7, 2011 at 14:43
  • 9
    @khmarbaise Lots of unit tests. They take a couple of minutes to run and we don't need them to run in this circumstance. Specifically, we run unit tests before building the artifact (of course), but we want to run the ITs in multiple environments. No point re-running the unit tests at this point. Commented Jul 7, 2011 at 14:45
  • 2
    Hi @khmarbaise, in my set up skipTests skips only the surefire tests, not the failsafe test! Maybe it's a new feature ? Commented May 15, 2017 at 15:36
  • 2
    FYI : skipTests is now deprecated in Failsafe Plugin 3.0.0-M3 (SUREFIRE-1611) Commented Feb 1, 2019 at 10:49
  • 1
    Have Failsafe Plugin 3.0.0-M5 and it still skip integration tests with skipTests. According to the comment in source code skipTests will be removed in Failsafe 3.0.0 Commented Sep 11, 2020 at 18:10

7 Answers 7

Reset to default
214

I found the simplest way to skip only surefire tests is to configure surefire (but not failsafe) as follows:

<plugin> <artifactId>maven-surefire-plugin</artifactId> <version>2.14</version> <configuration> <!-- skips surefire tests without skipping failsafe tests. Property value seems to magically default to false --> <skipTests>${skip.surefire.tests}</skipTests> </configuration> </plugin>

This allows you to run mvn verify -Dskip.surefire.tests and only surefire, not failsafe, tests will be skipped; it will also run all other necessary phases including pre-integration and post-integration, and will also run the verify goal which is required to actually fail your maven build if your integration tests fail.

Note that this redefines the property used to specify that tests should be skipped, so if you supply the canonical -DskipTests=true, surefire will ignore it but failsafe will respect it, which may be unexpected, especially if you have existing builds/users specifying that flag already. A simple workaround seems to be to default skip.surefire.tests to the value of skipTests in your <properties> section of the pom:

<properties> <skip.surefire.tests>${skipTests}</skip.surefire.tests> </properties>

If you need to, you could provide an analagous parameter called skip.failsafe.tests for failsafe, however I haven't found it necessary - because unit tests usually run in an earlier phase, and if I want to run unit tests but not integration tests, I would run the test phase instead of the verify phase. Your experiences may vary!

These skip.(surefire|failsafe).tests properties should probably be integrated into surefire/failsafe code itself, but I'm not sure how much it would violate the "they're exactly the same plugin except for 1 tiny difference" ethos.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Using this solution I've been able to set up my framework so -DskipUnitTests skips over the surefire plugin, -DskipIntegrationTests skips over the failsafe plugin, and DskipTests skips over both. Exactly what was needed!
my IDE is complaining about "cannot resolve symbol 'skipTests'" the solution was to add a line <skipTests>false</skipTests> still works with any combination of -DskipTests or -Dskip.surefire.tests as command line args seem to overwrite properties stackoverflow.com/questions/13708738/… you may want to add that to your solution
<skipTests>${skip.surefire.tests}</skipTests> is not working with maven-surefire-plugin version 3.0.0-M3. All the surefire tests are still running. Anyone else found this? Sean Patrick Floyd's below solution is however working.
About "however I haven't found it necessary - I would run the test phase instead of the verify phase.", this would disable a test coverate analysis run by Jacoco in the "verify" phase, and u may want to see your Unit test coverage without integration tests.
136

A workaround would be to call:

mvn clean test-compile failsafe:integration-test

Admittedly, this is ugly, but it may solve your problem.

Or (another hack):

mvn clean integration-test -Dtest=SomePatternThatDoesntMatchAnything -DfailIfNoTests=false

Reference:

Improve this answer

10 Comments

Is this a good idea? Won't this result in your build succeeding even if your integration tests fail? That's the whole essence of failsafe, if you don't also run the 'verify' goal. Quote: "The Failsafe Plugin will not fail the build during the integration-test phase". You need run the verify goal to actually tell you whether the integration tests succeeded or not!
@bacar is right, but just use verify instead of integration-test in the 2nd solution.
If you add failsafe:verify to the end of the first hack (mvn clean test-compile failsafe:integration-test failsafe:verify) it will fail the build if one of the integration tests fails.
The first solution also works with running a single test: mvn -Dit.test=<testName> failsafe:integration-test
One reason to skip (huge piles of) unit tests and ignore verify is that one is debugging an integration test or a bug that it has uncovered.
|
120

I am using the code from Antonio Goncalves Blog , which works perfect.

You can use the following properties:

-DskipUTs=true for skipping surefire tests.

-DskipITs=true for skipping failsafe tests.

-DskipTests=true for skipping all tests.

The pom.xml is as follows:

<properties> <skipTests>false</skipTests> <skipITs>${skipTests}</skipITs> <skipUTs>${skipTests}</skipUTs> </properties> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.19.1</version> <configuration> <skipTests>${skipUTs}</skipTests> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>2.19.1</version> <executions> <execution> <id>run-integration-tests</id> <phase>integration-test</phase> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> </execution> </executions> <configuration> <skipTests>${skipTests}</skipTests> <skipITs>${skipITs}</skipITs> </configuration> </plugin> </plugins> </build>
Improve this answer

4 Comments

thanks, great approach! it also useful to override default execution of surefire (default-test) see: stackoverflow.com/questions/11935181/…
Really a nice solution. However, the failsafe configuration is redundant as skipITs is the default.
SkipITs is a custom option in this config. You right in the default implementation, but that was not the OPs question.
In my opinion this should be the accepted answer.
32

Hope this helps!

Try to run test only with FailSafe (plugin for integration-test - it will allow you to run only integration tests with this kind naming, by default: */IT.java, **/IT.java, */*ITCase.java; , but you can easily change that from pom file)

mvn failsafe:integration-test

And when you want to use only SureFire (plugin for unit-testing)

mvn surefire:test

or one test at a time with:

mvn -Dtest=MyUnitlTest
Improve this answer

2 Comments

This should have more votes! Simple and works!
This works but disables other plugins who run only in the "verify" phase like test coverage analysis with Jacoco.
15

I do like this so every phases are normally executed:

 mvn -Dtest=foo -DfailIfNoTests=false verify
Improve this answer

1 Comment

doesn't require adding anything in the pom as the OP asked and runs all phases as indicated. great answer.
2

This skips the UnitTests triggered by SureFire and tests only one test:

mvn failsafe:integration-test -Dit.test=YourTestName

You can also provide a pattern: mvn failsafe:integration-test -Dit.test=*

(Maven 3.6.3)

Improve this answer

Comments

-2

Try running your integration or unit tests in a separate profile. Then you can just enable/disable the profile.

Improve this answer

1 Comment

Maybe add parent pom in which you define profile that is running only ITs? all project sub modules could inherit form that pom, so you wouldn't need to change every pom or run modules with special switches (as you can activate profile on property absence).

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.