0

I have an API that has the following logic:

  1. Consume from Kafka.
  2. Process the record.
  3. Update the database, if the processing was successful.
  4. If the processing fails, then push it to a Kafka topic.
  5. If pushing to Kafka topic failed, then commit.
  6. If the record was processed successfully, then commit.
  7. If the commit fails, then log and move ahead with consuming the next event.

I am writing BDDs for this API. Currently, I feel like I am testing too many scenarios:

  1. ProcessingFailed -> Database is unchanged -> Event should be pushed to Kafka -> Should be committed.
  2. Kafka push failed -> Should be committed.
  3. Commit failed -> (what to do? Should I check if the log is printed correctly?)
  4. Happy path -> Database updated -> Kafka topic does not contained the event -> Commit was successful.

My question is, what's the proper way to test for such side effects?

Now suppose my process step is made of three steps:

  1. Fetch from the database.
  2. Make an HTTP call.

Now supposing that I am simulating a 'processing failed' by bringing my database down. Now do I also need to test that the HTTP call was not made?

Improve this question

2 Answers 2

Reset to default
2

A good general rule for bdd tests is each test should only have one reason to fail. For cucumber this translates to only one Then step in each scenario.

With this as guidance I would recommend writing one scenario per step of the process.

# Consume from Kafka Given a certain thing has happened # Process the record When some action is performed successfully # Update database if processed successfully Then some result exists in the database

Then your next scenario starts where the first one left off:

Given a certain thing happened When the action is performed unsuccessfully # Push failed message to Kafka queue Then a failed message is sent

The third scenario picks up where the second one leaves off:

Given a certain thing happened And the action was performed unsuccessfully When a failure message is sent Then a thing should not exist in the database

Each scenario builds off the steps verified in the previous scenarios, being careful to ensure scenarios do not share data, or depend on the success of previously executed scenarios.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

I usually consider "one reason to fail" to mean only one "when" rather than one "then" (and even then there are exceptions). Consider for instance taking money from an ATM. You get the money, and also your bank account is debited; both of those have to happen at the same time, so should IMO appear in the same scenario. This is pretty typical of transactions. If it's a separate outcome for another stakeholder (deliverable independently), then yes, put them in a separate scenario as it's a different capability that happens to have the same trigger for its behaviour.
@Lunivore: I can see what you mean, but I've always thought of BDD tests in respect to the change in state you expect. If a single change of state causes your test to fail for multiple reasons, people will think that test is flaky or unstable. Same rule applies to unit tests as well. You want the failure to be: A) Singular; and B) Easy to identify. Having a single Then gives you that single point of expected failure. If you think about it, a Given is a When that has already happened, so most BDD tests already have more than one When anyhow.
You're thinking of BDD's primary purpose as testing. Try thinking of them as automated scenarios and living documentation. If it's done right, it helps people to understand how/why the software works and should prevent bugs. Catching them should be a rare thing. If you're primarily using BDD for regression tests, use more class-level TDD / BDD to help separate concerns and improve quality. And Givens aren't always Whens that have happened; we've frequently for instance set up data without going through the UI or API. It shouldn't matter how a Given is set up (or who did it!).
Short version: I value making automated scenarios and unit tests easy to understand when reading them, more than I value making them easy to understand when they fail. They fail less that way.
1

Currently, I feel like I am testing too many scenarios My question is, what's the proper way to test for such side effects?

Well, it sounds to me like you are describing a state machine; where the transitions are driven by representations of different effects in the protocol.

Given that, I would normally expect to see tests for each target state.

Depending on your evaluation of the risks, it might make sense to run your automated checks at a number of different grains -- lots of decoupled tests exploring the different corner cases of the state machine itself, some checks to make sure that the orchestration of the different effects is correct, a few tests to make sure the whole mess works when you wire it all together.

Now do I also need to test that the HTTP call was not made?

There are probably two important questions to ask yourself here:

  1. What are the risks of not having an automated test?
  2. Why is just adding tests not effortless?

If the test subject is "so simple that there are obviously no deficiencies", then the investment odds tell us that investing time and money into extra testing is not a favorable play.

On the other hand, if you are looking for an excuse not to test the thing, then you might want to turn a critical eye toward your design. That's especially true if you are adding/changing code in a module that "already works". A big payoff for test investment comes from having many easy accurate tests for the code we are changing on a regular basis, so reluctance to add a new test for code that you are changing is a Big Red Flag[tm] that something Is Not According To Plan.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.