4

I'm starting to work with TPL right now. I have seen a simple version of the producer/consumer model utilizing TPL in this video.

Here is the problem:

The following code:

BlockingCollection<Double> bc = new BlockingCollection<Double>(100); IEnumerable<Double> d = bc.GetConsumingEnumerable();

returns an IEnumerable<Double> which can be iterated (and automatically consumed) using a foreach:

foreach (var item in d) { // do anything with item // in the end of this foreach, // will there be any items left in d or bc? Why? }

My questions are:

  1. if I get the IEnumerator<Double> dEnum = d.GetEnumerator() from d (to iterate over d with a while loop, for instance) would the d.MoveNext() consume the list as well? (My answer: I don't think so, because the the dEnum is not linked with d, if you know what I mean. So it would consume dEnum, but not d, nor even bc)
  2. May I loop through bc (or d) in a way other than the foreach loop, consuming the items? (the while cycles much faster than the foreach loop and I'm worried with performance issues for scientific computation problems)
  3. What does exactly consume mean in the BlockingCollection<T> type?

E.g., code:

IEnumerator<Double> dEnum = d.GetEnumerator(); while (dEnum.MoveNext()) { // do the same with dEnum.Current as // I would with item in the foreach above... }

Thank you all in advance!

Improve this question
2
  • Why do you think foreach would have performance issues? Commented Oct 5, 2012 at 18:47
  • Well, I've done some really dummy tests (like iterating through an array with millions elements and performing exactly the same conditional checking over each item of the array) and measured the time it took for while, for for and for foreach to finish the task... In the, the while was considerably faster... Commented Oct 5, 2012 at 18:52

2 Answers 2

Reset to default
4

If I get the IEnumerator<Double> dEnum = d.GetEnumerator() from d (to iterate over d with a while loop, for instance) would the d.MoveNext() consume the list as well?

Absolutely. That's all that the foreach loop will do anyway.

May I loop through bc (or d) in a way other than the foreach loop, consuming the items? (the while cycles much faster than the foreach loop and I'm worried with performance issues for scientific computation problems)

If your while loop is faster, that suggests you're doing something wrong. They should be exactly the same - except the foreach loop will dispose of the iterator too, which you should do...

If you can post a short but complete program demonstrating this discrepancy, we can look at it in more detail.

An alternative is to use Take (and similar methods).

What does exactly consume mean in the BlockingCollection type?

"Remove the next item from the collection" effectively.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

As I responded to the comment by @Reed, I've done some dummy tests and while seems faster... At least when dealing with non-parallalel problems...
@Girardi: Your comment talks about iterating over arrays. Have you actually iterated over a BlockingCollection<T>? Also, it's very easy to get benchmarking like this wrong. That's why I've requested a short but complete program demonstrating the issue - there's no point in going off on a completely different route based on faulty assumptions.
Well, as I'm a complete ignorant in TPL, I didn't test the BlockingCollection<T>. So I may have made faulty assumptions after all...
@Girardi: As ever - get code working in a simple way first, then measure performance.
@Girardi: It removes it from the blocking collection. That's why it's a "consuming enumerable". From the docs for GetConsumingEnumerable: "An IEnumerable<T> that removes and returns items from the collection."
|
4

There is no "performance issue" with foreach. Using the enumerator directly is not likely to give you any measurable improvement in performance compared to just using a foreach loop directly.

That being said, GetConsumingEnumerable() returns a standard IEnumerable<T>, so you can enumerate it any way you choose. Getting the IEnumerator<T> and enumerating through it directly will still work the same way.

Note that, if you don't want to use GetConsumingEnumerable(), you could just use ConcurrentQueue<T> directly. By default, BlockingCollection<T> wraps a ConcurrentQueue<T>, and really just provides a simpler API (GetConsumingEnumerable()) to make Producer/Consumer scenarios simpler to write. Using a ConcurrentQueue<T> directly would be closer to using BlockingCollection<T> without using the enumerable.

Improve this answer

4 Comments

As I responded to the comment by @Reed, I've done some dummy tests and while seems faster... At least when dealing with non-parallalel problems...
@Girardi Typically, the performance difference of the language construct itself will pale in comparison to any computation being performed. This is especially true if you're working with something like BlockingCollection<T> - the while/for/foreach differences would be microscopic compared to just the overhead of putting items in and out of the collection. As such, I wouldn't focus on that in your optimization - focus on what makes the cleanest algorithm, so you can find and fix the "real" performance issues (in your code that uses the items)
BlockingCollection is more than a simpler API for ConcurrentQueue (or other concurrent collections). Its point is that its methods block until an operation can be performed. For example, if you wanted to implement alternative to BlockingCollection.Take() using just ConcurrentQueue, you would need to manually use some synchronization construct (like ManualResetEvent).
@svick Yes - sorry, I realize my wording wasn't perfectly clear there, either. BC adds the "blocking" portion, including .Take and .GetConsumingEnumerable to the ConcurrentQueue. If you want the blocking nature, then yes, you'd need that. If you want that, though, GetConsumingEnumerable is likely the option you'd want anyways.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.