3

The following code is a program.

  • The payloadList contains json-objects like {"id": 1, "foo": "one" }.
  • Each object of payloadList should be sent to a server with httpClient.SendAsync()
  • The response for each request should be stored in responseList

The code below does partly work. But i do not understand why some parts are not working. I assume that the responses are not completed when responseList.Add(foo) is executed.

This request shall be send for each json-object {"id": 1, "foo": "one" }

public static async Task<string> DoRequest(HttpClient client, string payload) { var request = new HttpRequestMessage(HttpMethod.Post, "http://httpbin.org/anything"); request.Content = new StringContent(payload , Encoding.UTF8, "application/json"); var response = await client.SendAsync(request); string responseContent = await response.Content.ReadAsStringAsync(); return responseContent; }

The DoRequest()-method wraps the request and can be used inside main like this

static async Task Main() { var responseList = new List<string>(); var payloadList = new List<string>{"{ 'id': 1, 'bar': 'One'}", "{ 'id': 2, 'bar': 'Two'}", "{ 'id': 3, 'bar': 'Three'}"}; var client = new HttpClient(); payloadList.ForEach(async (payload) => { var responseFoo = await DoRequest(client, payload); responseFoo.Dump("response"); // contains responseFoo responseList.Add(responseFoo); // adding responseFoo to list fails }); responseList.Dump(); // is empty }

The responseList is empty.

  • Expected responseList.Dump() contains all responses responseFoo.
  • Actual responseList is empty.

Linqpad-demo

Questions

  • How can each response for await client.SendAsync(request) be added to a responseList?
  • Why is responseList empty despite that foo.Dump() works?
  • How to confirm or check if every client.SendAsync is finished?
  • Would you write the code above different - why?
Improve this question
7
  • ask one question at a time please. See also How to Ask Commented Dec 8, 2022 at 14:14
  • responseList.Add(foo); // this not ...in what way does it "not work"? Is there an error? Commented Dec 8, 2022 at 14:14
  • The responseList is empty. Expected responseList.Dump() contains all responses foo. Actual responseList is empty. Commented Dec 8, 2022 at 14:16
  • P.S. putting responseList.Dump(); outside the foreach loop obviously won't work because that'll run before all the async stuff is complete. Commented Dec 8, 2022 at 14:16
  • stackoverflow.com/questions/25009437/… may be helpful Commented Dec 8, 2022 at 14:17

1 Answer 1

Reset to default
4

List.ForEach is not Task-aware and will execute everything in parallel without waiting for the results (i.e. it will create tasks for all items in payloadList and will continue to the next statement responseList.Dump(); without awaiting them).

  1. In newer versions of .NET you can use Parallel.ForEachAsync(for example as in this answer) combined with use of appropriate collection from System.Collections.Concurrent, for example ConcurrentBag<T>. List is not thread-safe and modifying it concurrently can lead to a lot of problems.

    Code can look like the following:

    var responseList = new ConcurrentBag<string>(); await Parallel.ForEachAsync(payloadList, async (payload, ct) => { var foo = await DoRequest(client, payload, ct); responseList.Add(foo); }); static async Task<string> DoRequest(HttpClient client, string payload, CancellationToken ct) { var request = new HttpRequestMessage(HttpMethod.Post, "http://httpbin.org/anything"); request.Content = new StringContent(payload, Encoding.UTF8, "application/json"); var response = await client.SendAsync(request, ct); string responseContent = await response.Content.ReadAsStringAsync(ct); return responseContent; }

  2. If you are fine with all requests running in parallel - just create a enumerable of tasks and use Task.WhenAll<T>():

    var tsks = payloadList .Select(payload => DoRequest(client, payload)); string[] result = await Task.WhenAll(tsks);

  3. If you want to execute requests one after another - just switch to ordinary foreach:

    var responseList = new List<string>(); // no need for concurrent collection in this case foreach (var payload in payloadList) { var foo = await DoRequest(client, payload); responseList.Add(foo); }

If you want to dive deeper here some links:

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

Make sure to await Parallel.ForEachAsync
Could you add some code how to use it?
Thanks for the code. One more thing. I would expect that responseList.Add(foo) should work. But it stays empty. Why is this the case?
@surfmuggle Potentially you app finishes before the requests are completed. And responseList.Dump(); is executed before any request is completed with very high chance. I.e. async-await inside List.ForEach has no effect.
@surfmuggle added some links which can be useful if you will want to go a bit deeper into the topic.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.