Return to Answer

This is a huge topic, so I will only write a moderate amount of text here, and refer you to external sources for further reading.

Disclosure: I am the author of the external sources.

The way Automated Software Testing is practiced in the industry today involves Unit Testing with Mock Objects. By established practice, these three concepts are regarded as inextricably tied together:

• If you want to be doing Automated Software Testing, you have to be doing Unit Testing.
• If you want to be doing Unit Testing, you have to be using Mock Objects.

That is what the vast majority of programmers all over the world do.

And it is all deeply misguided.

The sole reason of existence of Unit Testing is to achieve Defect Localization: when the system malfunctions, we want to know exactly which component causes the malfunction.

Defect localization is in fact a huge issue in Hardware Testing, where the concept of Unit Testing originates from. It is also of some importance in Software Testing, it but is not nearly as important. Nonetheless, Unit Testing is the only tool that people know, so that is what people do.

In order to achieve Defect Localization, Unit Testing demands that each component must be tested in strict isolation from the rest of the system. To achieve this, people use Mock Objects. The use of Mock Objects is misguided because they are laborious, over-specified, presumptuous, and lead to tests that are fragile and non-reusable. For more information, see michael.gr - If you are using mock objects you are doing it wrong.

Here is the abstract of that paper:

The practice of using Mock Objects in automated software testing is examined from a critical point of view and found to be highly problematic. Opinions of some well known industry speakers are cited. The supposed benefits of Mock Objects are shown to be either no real benefits, or achievable via alternative means.

(Note: the well known industry speakers that are cited include Martin Fowler and Kent Beck.)

So, the "higher level unit testing that tests multiple objects at a time" that you are speaking of in your question does not exist, because it would not be Unit Testing. If you are testing multiple objects at a time then you are doing some form of integration testing, and if you are testing all objects at the same time then you are doing end-to-end testing.

I have come up with an alternative technique, which goes quite a long way towards achieving Defect Localization; it just does not achieve it perfectly, in exchange for other benefits. I call this technique Incremental Integration Testing. Perhaps it is close to your notion of testing multiple objects at a time. You can read about it here: michael.gr - Incremental Integration Testing

Here is the abstract of that paper:

A new method for Automated Software Testing is presented as an alternative to Unit Testing. The new method retains the benefit of Unit Testing, which is Defect Localization, but eliminates the need for mocking, thus greatly lessening the effort of writing and maintaining tests.

It is hard to tell why people are stuck with Unit testing. Maybe it is adherence to tradition, maybe it is cargo cult engineering, maybe it is reluctance to admit the magnitude of their prior fallacy and reluctance to write off the monumental effort that they have already put into it.

This is a huge topic, so I will only write a moderate amount of text here, and refer you to external sources for further reading.

Disclosure: I am the author of the external sources.

The way Automated Software Testing is practiced in the industry today involves Unit Testing with Mock Objects. By established practice, these three concepts are regarded as inextricably tied together:

• If you want to be doing Testing, you have to be doing Unit Testing.
• If you want to be doing Unit Testing, you have to be using Mock Objects.

That is what the vast majority of programmers all over the world do.

And it is all deeply misguided.

The sole reason of existence of Unit Testing is to achieve Defect Localization: when the system malfunctions, we want to know exactly which component causes the malfunction.

Defect localization is in fact a huge issue in Hardware Testing, where the concept of Unit Testing originates from. It is also of some importance in Software Testing, it but is not nearly as important. Nonetheless, Unit Testing is the only tool that people know, so that is what people do.

In order to achieve Defect Localization, Unit Testing demands that each component must be tested in strict isolation from the rest of the system. To achieve this, people use Mock Objects. The use of Mock Objects is misguided because they are laborious, and they constitute testing against the implementation instead of testing against the interface, so they are over-specified, presumptuous, and lead to tests that are fragile and non-reusable. For more information, see michael.gr - If you are using mock objects you are doing it wrong.

Here is the abstract of that paper:

The practice of using Mock Objects in automated software testing is examined from a critical point of view and found to be highly problematic. Opinions of some well known industry speakers are cited. The supposed benefits of Mock Objects are shown to be either no real benefits, or achievable via alternative means.

(Note: the well known industry speakers that are cited include Martin Fowler and Kent Beck.)

So, the "higher level unit testing that tests multiple objects at a time" that you are speaking of in your question does not exist, because it would not be Unit Testing. If you are testing multiple objects at a time then you are doing some form of integration testing, and if you are testing all objects at the same time then you are doing end-to-end testing.

I have come up with an alternative technique, which goes quite a long way towards achieving Defect Localization; it just does not achieve it perfectly, in exchange for other benefits. I call this technique Incremental Integration Testing. Perhaps it is close to your notion of testing multiple objects at a time. You can read about it here: michael.gr - Incremental Integration Testing

Here is the abstract of that paper:

A new method for Automated Software Testing is presented as an alternative to Unit Testing. The new method retains the benefit of Unit Testing, which is Defect Localization, but eliminates the need for mocking, thus greatly lessening the effort of writing and maintaining tests.

It is hard to tell why people are stuck with Unit testing. Maybe it is adherence to tradition, maybe it is cargo cult engineering, maybe it is reluctance to admit the magnitude of their prior fallacy and reluctance to write off the monumental effort that they have already put into it.

This is a huge topic, so I will only write a moderate amount of text here, and refer you to external sources for further reading.

Disclosure: I am the author of the external sources.

The way Automated Software Testing is practiced in the industry today involves Unit Testing with Mock Objects. By established practice, these three concepts are regarded as inextricably tied together:

• If you want to be doing Automated Software Testing, you have to be doing Unit Testing.
• If you want to be doing Unit Testing, you have to be using Mock Objects.

That is what the vast majority of programmers all over the world do.

And it is all deeply misguided.

The sole reason of existence of Unit Testing is to achieve Defect Localization: when the system malfunctions, we want to know exactly which component causes the malfunction.

Defect localization is in fact a huge issue in Hardware Testing, where the concept of Unit Testing originates from. It is also of some importance in Software Testing, it but is not nearly as important. Nonetheless, Unit Testing is the only tool that people know, so that is what people do.

According to Unit Testing, in order to achieve Defect Localization you have to test each component in strict isolation from the rest of the system. To achieve this, people use Mock Objects. The use of Mock Objects is misguided because they are laborious, over-specified, presumptuous, and lead to tests that are fragile and non-reusable. For more information, see michael.gr - If you are using mock objects you are doing it wrong.

Here is the abstract of that paper:

The practice of using Mock Objects in automated software testing is examined from a critical point of view and found to be highly problematic. Opinions of some well known industry speakers are cited. The supposed benefits of Mock Objects are shown to be either no real benefits, or achievable via alternative means.

(Note: the well known industry speakers that are cited include Martin Fowler and Kent Beck.)

So, the "higher level unit testing that tests multiple objects at a time" that you are speaking of in your question does not exist, because it would not be Unit Testing. If you are testing multiple objects at a time then you are doing some form of integration testing, and if you are testing all objects at the same time then you are doing end-to-end testing.

I have come up with an alternative technique, which goes quite a long way towards achieving Defect Localization, it just does not achieve it perfectly, in exchange for other benefits. I call this technique Incremental Integration Testing. You can read about it here: michael.gr - Incremental Integration Testing

Here is the abstract of that paper:

A new method for Automated Software Testing is presented as an alternative to Unit Testing. The new method retains the benefit of Unit Testing, which is Defect Localization, but eliminates the need for mocking, thus greatly lessening the effort of writing and maintaining tests.

It is hard to tell why people are stuck with Unit testing. Maybe it is adherence to tradition, maybe it is cargo cult engineering, maybe it is reluctance to admit the magnitude of their prior fallacy and reluctance to write off the monumental effort that they have already put into it.

This is a huge topic, so I will only write a moderate amount of text here, and refer you to external sources for further reading.

Disclosure: I am the author of the external sources.

The way Automated Software Testing is practiced in the industry today involves Unit Testing with Mock Objects. By established practice, these three concepts are regarded as inextricably tied together:

• If you want to be doing Automated Software Testing, you have to be doing Unit Testing.
• If you want to be doing Unit Testing, you have to be using Mock Objects.

That is what the vast majority of programmers all over the world do.

And it is all deeply misguided.

The sole reason of existence of Unit Testing is to achieve Defect Localization: when the system malfunctions, we want to know exactly which component causes the malfunction.

Defect localization is in fact a huge issue in Hardware Testing, where the concept of Unit Testing originates from. It is also of some importance in Software Testing, it but is not nearly as important. Nonetheless, Unit Testing is the only tool that people know, so that is what people do.

In order to achieve Defect Localization, Unit Testing demands that each component must be tested in strict isolation from the rest of the system. To achieve this, people use Mock Objects. The use of Mock Objects is misguided because they are laborious, over-specified, presumptuous, and lead to tests that are fragile and non-reusable. For more information, see michael.gr - If you are using mock objects you are doing it wrong.

Here is the abstract of that paper:

The practice of using Mock Objects in automated software testing is examined from a critical point of view and found to be highly problematic. Opinions of some well known industry speakers are cited. The supposed benefits of Mock Objects are shown to be either no real benefits, or achievable via alternative means.

(Note: the well known industry speakers that are cited include Martin Fowler and Kent Beck.)

So, the "higher level unit testing that tests multiple objects at a time" that you are speaking of in your question does not exist, because it would not be Unit Testing. If you are testing multiple objects at a time then you are doing some form of integration testing, and if you are testing all objects at the same time then you are doing end-to-end testing.

I have come up with an alternative technique, which goes quite a long way towards achieving Defect Localization; it just does not achieve it perfectly, in exchange for other benefits. I call this technique Incremental Integration Testing. Perhaps it is close to your notion of testing multiple objects at a time. You can read about it here: michael.gr - Incremental Integration Testing

Here is the abstract of that paper:

A new method for Automated Software Testing is presented as an alternative to Unit Testing. The new method retains the benefit of Unit Testing, which is Defect Localization, but eliminates the need for mocking, thus greatly lessening the effort of writing and maintaining tests.

It is hard to tell why people are stuck with Unit testing. Maybe it is adherence to tradition, maybe it is cargo cult engineering, maybe it is reluctance to admit the magnitude of their prior fallacy and reluctance to write off the monumental effort that they have already put into it.

This is a huge topic, so I will only write a moderate amount of text here, and refer you to external sources for further reading.

Disclosure: I am the author of the external sources.

The way Automated Software Testing is practiced in the industry today involves Unit Testing with Mock Objects. By established practice, these three concepts are regarded as inextricably tied together:

• If you want to be doing Automated Software Testing, you have to be doing Unit Testing.
• If you want to be doing Unit Testing, you have to be using Mock Objects.

That is what the vast majority of programmers all over the world do.

And it is all deeply misguided.

The sole reason of existence of Unit Testing is to achieve Defect Localization: when the system malfunctions, we want to know exactly which component causes the malfunction.

Defect localization is in fact a huge issue in Hardware Testing, where the concept of Unit Testing originates from. It is also of some importance in Software Testing, it but is not nearly as important. Nonetheless, Unit Testing is the only tool that people know, so that is what people do.

According to Unit Testing, in order to achieve Defect Localization you have to test each component in strict isolation from the rest of the system. To achieve this, people use Mock Objects. The use of Mock Objects is misguided because they are laborious, over-specified, presumptuous, and lead to tests that are fragile and non-reusable. For more information, see michael.gr - If you are using mock objects you are doing it wrong.

Here is the abstract of that paper:

The practice of using Mock Objects in automated software testing is examined from a critical point of view and found to be highly problematic. Opinions of some well known industry speakers are cited. The supposed benefits of Mock Objects are shown to be either no real benefits, or achievable via alternative means.

(Note: the well known industry speakers that are cited include Martin Fowler and Kent Beck.)

So, the "higher level unit testing that tests multiple objects at a time" that you are speaking of in your question does not exist, because it would not be Unit Testing. If you are testing multiple objects at a time then you are doing some form of integration testing, and if you are testing all objects at the same time then you are doing end-to-end testing.

I have come up with an alternative technique, which goes quite a long way towards achieving Defect Localization, it just does not achieve it perfectly, in exchange for other benefits. I call this technique Incremental Integration Testing. You can read about it here: michael.gr - Incremental Integration Testing

Here is the abstract of that paper:

A new method for Automated Software Testing is presented as an alternative to Unit Testing. The new method retains the benefit of Unit Testing, which is Defect Localization, but eliminates the need for mocking, thus greatly lessening the effort of writing and maintaining tests.

It is hard to tell why people are stuck with Unit testing. Maybe it is adherence to tradition, maybe it is cargo cult engineering, maybe it is reluctance to admit the magnitude of their prior fallacy and to write off the monumental amount of work that they have already been putting into it.

This is a huge topic, so I will only write a moderate amount of text here, and refer you to external sources for further reading.

Disclosure: I am the author of the external sources.

The way Automated Software Testing is practiced in the industry today involves Unit Testing with Mock Objects. By established practice, these three concepts are regarded as inextricably tied together:

• If you want to be doing Automated Software Testing, you have to be doing Unit Testing.
• If you want to be doing Unit Testing, you have to be using Mock Objects.

That is what the vast majority of programmers all over the world do.

And it is all deeply misguided.

The sole reason of existence of Unit Testing is to achieve Defect Localization: when the system malfunctions, we want to know exactly which component causes the malfunction.

Defect localization is in fact a huge issue in Hardware Testing, where the concept of Unit Testing originates from. It is also of some importance in Software Testing, it but is not nearly as important. Nonetheless, Unit Testing is the only tool that people know, so that is what people do.

According to Unit Testing, in order to achieve Defect Localization you have to test each component in strict isolation from the rest of the system. To achieve this, people use Mock Objects. The use of Mock Objects is misguided because they are laborious, over-specified, presumptuous, and lead to tests that are fragile and non-reusable. For more information, see michael.gr - If you are using mock objects you are doing it wrong.

Here is the abstract of that paper:

The practice of using Mock Objects in automated software testing is examined from a critical point of view and found to be highly problematic. Opinions of some well known industry speakers are cited. The supposed benefits of Mock Objects are shown to be either no real benefits, or achievable via alternative means.

(Note: the well known industry speakers that are cited include Martin Fowler and Kent Beck.)

So, the "higher level unit testing that tests multiple objects at a time" that you are speaking of in your question does not exist, because it would not be Unit Testing. If you are testing multiple objects at a time then you are doing some form of integration testing, and if you are testing all objects at the same time then you are doing end-to-end testing.

I have come up with an alternative technique, which goes quite a long way towards achieving Defect Localization, it just does not achieve it perfectly, in exchange for other benefits. I call this technique Incremental Integration Testing. You can read about it here: michael.gr - Incremental Integration Testing

Here is the abstract of that paper:

A new method for Automated Software Testing is presented as an alternative to Unit Testing. The new method retains the benefit of Unit Testing, which is Defect Localization, but eliminates the need for mocking, thus greatly lessening the effort of writing and maintaining tests.

It is hard to tell why people are stuck with Unit testing. Maybe it is adherence to tradition, maybe it is cargo cult engineering, maybe it is reluctance to admit the magnitude of their prior fallacy and reluctance to write off the monumental effort that they have already put into it.