When I create a test suite the tests are runned twice, once independenly and in suite. [mvn test]

[sveridovd]

How to solve that problem, that tests that are grouped in suite, only will be runned in suite, not independently?

[marcphilipp]

One common solution is to use a naming pattern. For example, call all suites *Suite and all tests *Tests and configure Maven Surefire to only include the former.

[laeubi]

I like to add something here and even as what @marcphilipp suggests here "solves" the issue this is not very straight forward/convenient from a users point of view:

1. This naming patterns are a constant confusing for users, e.g. if I run tests usually from the IDE there is no such restriction, then one (maybe even much later) notice that tests are never executed on the CI and regressions slipped in unnoticed
2. That junit can scann the class-path is really great and that's again something similar to whats from the IDE is well know, just select a folder or project chose run and your are done
3. Its a bit strange that a tool should not be able to make a simple filtering of duplicates, as these classes are literally the same, the are from the same project and even referencing each other (at least uni-directional)

So to summarize, it would really be great if JUnit would sort that out without any extra configuration and naming schemes.

[mpkorstanje]

I don't think anything on the user facing side suggests Suites work that way. Granted, the documentation is rather terse and phrased in JUnit terminology, but that's why its in the advanced section of the user guide. 😉

Aside of your preconceived idea about what suites are supposed to be, what convinced you suites work that way in JUnit?

[laeubi]

Almost every tutorial about junit suite talks about its for grouping tests, so at least I'm not alone with my assumptions :-)

Also here https://docs.junit.org/current/user-guide/#junit-platform-suite-engine it says

The JUnit Platform supports the declarative definition and execution of suites of tests
..
By annotating a class with @suite it is marked as a test suite on the JUnit Platform. As seen in the following example, selector and filter annotations can then be used to control the contents of the suite.

So one might argue, that if there is any suite discovered this is the definition of tests to run so the autodiscovery should probably ignore any "non suites" or at least not discover them twice. But again this is only an intuition from what I see when people recognize that behavior they are confused and the next is to filter for only suites to be included so I never seen anyone that could actually benefit from the behavior of discovering the same test with two pathes. e.g. if I really want to execute the test twice with different settings I would just define two suites...

[mpkorstanje]

Ah. Thanks for pointing that out. I wrote the Suite Engine, so I haven't read any of the tutorials about it. 😅

Fortunately, they do all mention that you have to change which tests are included. I think we can add that to the documentation and also reference to tags as an alternative.

[laeubi]

Ah. Thanks for pointing that out. I wrote the Suite Engine, so I haven't read any of the tutorials about it. 😅

No problem, I just wanted to note that it is at least how people interpret that thing :-)

It might stem from the history of JUnit3/4 as well https://github.com/junit-team/junit4/wiki/aggregating-tests-in-suites

Fortunately, they do all mention that you have to change which tests are included.

I think none of them see this as a feature, but a restriction, that why I mentioned it here :-)

So JUnit has a lot of smartness, the handling of classpath scanning when suites are used feels rather dumb (no offense ;-))

[mpkorstanje]

None taken. I originally wrote the the Suite Engine to support Cucumber. And then it works really well because nobody asks JUnit to discover tests in feature files.

It might stem from the history of JUnit3/4 as well https://github.com/junit-team/junit4/wiki/aggregating-tests-in-suites

Can't help but notice that the classes in the Suite have suspiciously test free names.