2

We have a simple CRUD application. QA has a test script where they are doing the following:

  1. Call Create Customer service
  2. Query the database for the newly created Customer
  3. Verify the Customer in the database matches the expected result
  4. Call the Get Customer service
  5. Verify that the Customer returned from the services matches the expected result

My question is: should be QA even be running steps 2 and 3? Or should they be agnostic to how the data is stored and just validate the application output (run steps 1, 4, 5).

On the one hand, it would be good to check that we are saving data correctly to the database (especially if we are saving in calculations, foreign keys, generating timestamps, etc.).

On the other hand, QA shouldn't care how/where we are storing data, as long as it is returned correctly from the application.

Improve this question
2
  • Who owns the customer data? If you were the owner, would you be happy with Q&A not checking how the data gets stored? Commented Mar 21, 2017 at 9:24
  • Who owns the database? Can you make schema changes (e.g. renaming tables/columns) without affecting other applications? Commented Mar 21, 2017 at 9:58

2 Answers 2

Reset to default
4

In my experience, it isn't unusual for QA to carry out these kinds of tests. The priority of course is to check basic behaviour (black box tests). As an extra sense check, you may want to check the system internals (white box tests). Steps 2 & 3 would fall into the white box testing category.

I don't know how your system is structured but it is possible to imagine a scenario where data is cached in some kind of mid-tier. If the database wasn't checked, tests 1, 4 & 5 would pass with flying colours but if the mid-tier were re-started, those tests could fail. Also, if the test script contained static data to be submitted, you couldn't be sure this wasn't hard coded into the application somewhere.

You can also spot potential sources of bugs by comparing the submitted data to the saved and retrieved data. If the submitted and retrieved data are the same but the saved data in the database is different, it may be worth doing some additional testing to ensure all conversions for the value in question are carried out correctly.

A system I worked on had an issue handling gender. A dropdown in the application had M(ale), F(emale) & U(nknown), but only M, F or space was stored in the database. On further inspection, other parts of the system used a variety of strings to represent blank such as "unspecified", "not known" etc. This is something we wouldn't have found if we hadn't checked the database.

Improve this answer
3

On the other hand, QA shouldn't care how/where we are storing data, as long as it is returned correctly from the application.

I strongly disagree.

If what's being tested is the storage of data, then you absolutely have to test that the data is being stored correctly.
You cannot rely on the Application under test to tell you that it's stored the data correctly without some external agent verifying that it has actually done so.

It's like building a brand new thermometer and then trying to calibrate it using nothing but the thermometer itself to do so. You might get lucky and it displays every temperature correctly or, more likely, you'll get burned.

Improve this answer
2
  • This is very true if your system makes use of triggers to modify data not in the original (your 'Customer') table. Commented Mar 21, 2017 at 18:05
  • Agree with Phil. This is especially important when your application data is enterprise data that is being used by other systems and reports. Imagine if your data was used for compliance of Sarbanes-Oxley or for Tax Reporting. Commented Oct 11, 2017 at 12:51