C# Parallel Foreach repeat approach

Question

So basically, what I'm trying do is I want the parallel.foreach to repeat the log if an error is caught to the specific thread.

Parallel.ForEach(concurrentLogs, parallelOptions, log => try{ //Do work Console.WriteLine(log); } catch(Exception ex) { concurrentLogs.Enqueue(log); //repeat this log } });

Because when I'm debugging it, if a thread catches an error (ex. io exception) it won't repeat the same log anymore.

What could be a possible approach to this?

It depends on the type of concurrentLogs If it is a BlockingCollection you need to use GetConsumingEnumerable() — Scott Chamberlain
– Scott Chamberlain, Commented Nov 30, 2019 at 19:14
Hello @ScottChamberlain the concurrentLogs is a ConcurrentQueue. — Anna P.
– Anna P., Commented Nov 30, 2019 at 19:17

Stuart · Accepted Answer · 2019-11-30 19:49:50Z

You are enumerating a ConcurrentQueue, which will be a snapshot of the collection and not reflect your later Enqueue items.

A quick solution would be to simply retry within the "foreach":

Parallel.ForEach(concurrentLogs, parallelOptions, log => { void DoWork(string log) { //Do work Console.WriteLine(log); } try { DoWork(log); } catch(Exception ex) { // or loop and keep count DoWork(log); } });

aside: as noted in the comments, this isn't how best to handle retrying, you need to decide a strategy. Polly is great for this sort of thing.

However this suggests you don't want a ConcurrentQueue, or you aren't using ineffectively. You might want to look at a BlockingCollection, Channel, or ActionBlock (TPL Dataflow).

Then you'd write it as a loop, keep count, or whatever strategy you want. I'm here to answer the question, not write production ready code 🙂
@Eldar "if it fails again..." is kind of beside the point. The OP can simply add a counter, and bail out after max retries. The key issue is that the design is probably "flawed"; the OP ought to consider alternatives. Stuart suggested a couple of good choices :)
@AnnaP. +1 Also, I can recommend you to use Polly for retry logic. Install it via nuget. And then declare that policy outside of the loop: var retryPolicy = Policy.Handle<Exception>().RetryForever() And use so inside loop: retryPolicy.Execute(() => DoWork(log)))

Amadeu Antunes · Accepted Answer · 2019-11-30 19:31:05Z

1

 Parallel.ForEach(concurrentLogs, parallelOptions, log => { bool sucess = true; do { try { //Do work Console.WriteLine(log); } catch (Exception ex) { sucess = false; } }while(!sucess) });

edited Nov 30, 2019 at 19:31

answered Nov 30, 2019 at 19:24

Amadeu Antunes

7361 gold badge8 silver badges28 bronze badges

2 Comments

Yeldar Kurmangaliyev Over a year ago

@Eldar What's wrong with the memory though? You just repeat the operation. It probably misses a retry policy (like "how much time I need to wait between attempts", "how I report errors" and "after how many fails do I need to stop trying"), but in general it looks like a correct way to achieve this.

Eldar Over a year ago

No first version of your code was enqueueing another log every time it fails while trying to handle that log. When while loop completed log will be already processed and logs enqueued in catch block will be redundant. But this version is ok but again as you stated while can be dangerous and i like this answer more. You should add your comments content in to your answer.

Collectives™ on Stack Overflow

C# Parallel Foreach repeat approach

2 Answers 2

7 Comments

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

2 Comments

Related