Skip to main content
added 124 characters in body
Source Link
user1430
user1430

Testing is an investment in your future. While an individual test might duplicate some aspect of manual testing you are about to do in any given run of the game, a robust suite of tests can in the long run cover far more scenarios than that small bit of overlapping work (manual testing can also more effectively test more complex scenarios where writing automatic tests might be too difficult).

Writing tests is an overhead, yes. So is maintaining them when the things you are testing change. It's also much harder to retrofit tests onto a big project not built from scratch with that goal in mind, as there are some technical design choices that are less compatible with easy, isolated testing than others.

Whether or not that overhead is worth the benefit depends on you and the scale and scope of your project.

The payoff is that automated tests can be run automatically and require far, far less work on your part that equivalent manual testing coverage would. In your example, sure, you manually run the game and ad-hoc test your change.

But do you do it on every platform you are shipping on? On every build configuration? Automated tests can do so easily, which means you can catch errors that only manifest on one or two platforms/configurations that you don't regularly test more quickly.

Similarly, while you may ad-hoc test that one ability change, do you manually go back and ad-hoc test every other ability? Probably not, because you're operating under the assumption that you didn't make a change that could impact those other abilities. But you are a programmer, and therefore, you make mistakes ("bugs") and therefor you could have accidentally made your change in a such a way as to have unintentionally broke something you were not expecting. An automated test suite could catch that.

Granted, the value of the test suite is dependent on what you test and how, and there is a point of diminishing returns in the investment (100% test coverage is often impractical, for example).

For example, it's probably worthwhile to write a test to validate the results of a math function that presumes a left- or right-handed convention, as change in that function to prefer the other convention will probably destabilize a lot and should be flagged. But a test for the outcome of some damage calculation may not be a good candidate for a test, at least not early on when you are iterating rapidly on game balance. That sort of test is perhaps best added later, once balance has hardened a little, to alert you that you may have made a change that has balance implications.

Testing is an investment in your future. While an individual test might duplicate some aspect of manual testing you are about to do in any given run of the game, a robust suite of tests can in the long run cover far more scenarios than that small bit of overlapping work.

Writing tests is an overhead, yes. So is maintaining them when the things you are testing change. It's also much harder to retrofit tests onto a big project not built from scratch with that goal in mind, as there are some technical design choices that are less compatible with easy, isolated testing than others.

Whether or not that overhead is worth the benefit depends on you and the scale and scope of your project.

The payoff is that automated tests can be run automatically and require far, far less work on your part that equivalent manual testing coverage would. In your example, sure, you manually run the game and ad-hoc test your change.

But do you do it on every platform you are shipping on? On every build configuration? Automated tests can do so easily, which means you can catch errors that only manifest on one or two platforms/configurations that you don't regularly test more quickly.

Similarly, while you may ad-hoc test that one ability change, do you manually go back and ad-hoc test every other ability? Probably not, because you're operating under the assumption that you didn't make a change that could impact those other abilities. But you are a programmer, and therefore, you make mistakes ("bugs") and therefor you could have accidentally made your change in a such a way as to have unintentionally broke something you were not expecting. An automated test suite could catch that.

Granted, the value of the test suite is dependent on what you test and how, and there is a point of diminishing returns in the investment (100% test coverage is often impractical, for example).

For example, it's probably worthwhile to write a test to validate the results of a math function that presumes a left- or right-handed convention, as change in that function to prefer the other convention will probably destabilize a lot and should be flagged. But a test for the outcome of some damage calculation may not be a good candidate for a test, at least not early on when you are iterating rapidly on game balance. That sort of test is perhaps best added later, once balance has hardened a little, to alert you that you may have made a change that has balance implications.

Testing is an investment in your future. While an individual test might duplicate some aspect of manual testing you are about to do in any given run of the game, a robust suite of tests can in the long run cover far more scenarios than that small bit of overlapping work (manual testing can also more effectively test more complex scenarios where writing automatic tests might be too difficult).

Writing tests is an overhead, yes. So is maintaining them when the things you are testing change. It's also much harder to retrofit tests onto a big project not built from scratch with that goal in mind, as there are some technical design choices that are less compatible with easy, isolated testing than others.

Whether or not that overhead is worth the benefit depends on you and the scale and scope of your project.

The payoff is that automated tests can be run automatically and require far, far less work on your part that equivalent manual testing coverage would. In your example, sure, you manually run the game and ad-hoc test your change.

But do you do it on every platform you are shipping on? On every build configuration? Automated tests can do so easily, which means you can catch errors that only manifest on one or two platforms/configurations that you don't regularly test more quickly.

Similarly, while you may ad-hoc test that one ability change, do you manually go back and ad-hoc test every other ability? Probably not, because you're operating under the assumption that you didn't make a change that could impact those other abilities. But you are a programmer, and therefore, you make mistakes ("bugs") and therefor you could have accidentally made your change in a such a way as to have unintentionally broke something you were not expecting. An automated test suite could catch that.

Granted, the value of the test suite is dependent on what you test and how, and there is a point of diminishing returns in the investment (100% test coverage is often impractical, for example).

For example, it's probably worthwhile to write a test to validate the results of a math function that presumes a left- or right-handed convention, as change in that function to prefer the other convention will probably destabilize a lot and should be flagged. But a test for the outcome of some damage calculation may not be a good candidate for a test, at least not early on when you are iterating rapidly on game balance. That sort of test is perhaps best added later, once balance has hardened a little, to alert you that you may have made a change that has balance implications.

Source Link
user1430
user1430

Testing is an investment in your future. While an individual test might duplicate some aspect of manual testing you are about to do in any given run of the game, a robust suite of tests can in the long run cover far more scenarios than that small bit of overlapping work.

Writing tests is an overhead, yes. So is maintaining them when the things you are testing change. It's also much harder to retrofit tests onto a big project not built from scratch with that goal in mind, as there are some technical design choices that are less compatible with easy, isolated testing than others.

Whether or not that overhead is worth the benefit depends on you and the scale and scope of your project.

The payoff is that automated tests can be run automatically and require far, far less work on your part that equivalent manual testing coverage would. In your example, sure, you manually run the game and ad-hoc test your change.

But do you do it on every platform you are shipping on? On every build configuration? Automated tests can do so easily, which means you can catch errors that only manifest on one or two platforms/configurations that you don't regularly test more quickly.

Similarly, while you may ad-hoc test that one ability change, do you manually go back and ad-hoc test every other ability? Probably not, because you're operating under the assumption that you didn't make a change that could impact those other abilities. But you are a programmer, and therefore, you make mistakes ("bugs") and therefor you could have accidentally made your change in a such a way as to have unintentionally broke something you were not expecting. An automated test suite could catch that.

Granted, the value of the test suite is dependent on what you test and how, and there is a point of diminishing returns in the investment (100% test coverage is often impractical, for example).

For example, it's probably worthwhile to write a test to validate the results of a math function that presumes a left- or right-handed convention, as change in that function to prefer the other convention will probably destabilize a lot and should be flagged. But a test for the outcome of some damage calculation may not be a good candidate for a test, at least not early on when you are iterating rapidly on game balance. That sort of test is perhaps best added later, once balance has hardened a little, to alert you that you may have made a change that has balance implications.