Synchronous Concurrent HttpClient Usage

Question

I am trying to use HttpClient synchronously but when I make many concurrent requests it stops working. I wrote two tests, one for asynchronous usage and one for synchronous usage. TestMethod always returns response after 4 seconds. Asynchronous test works fine. Almost all requests are timed out in synchronous test, only ~20 last requests are successful. I tried both using a single HttpClient for requests and creating new HttpClient instance for each new requests. No difference. Maybe it has something to do with this deadlock.

I am using VS2013 with .NET Framework 4.5.1 targeting Framework 4.5. I get HttpClient via NuGet: <package id="Microsoft.Net.Http" version="2.2.15" targetFramework="net45" />

I don't want to use HttpClient asynchronously just yet because it means that I have to rewrite my whole app. Any ideas what am I doing wrong here?

// all 800 requests complete successfully [Test] public async void Asyncmethod() { var sw = new Stopwatch(); sw.Start(); var client = new HttpClient(); client.Timeout = TimeSpan.FromSeconds(15); var tasks = Enumerable.Range(0, 800).Select(x => Task.Run(async () => { try { var swx = new Stopwatch(); swx.Start(); var response = await client.GetStringAsync("http://localhost:7002/TestMethod").ConfigureAwait(false); swx.Stop(); Console.WriteLine(x + " " + response + " in " + swx.ElapsedMilliseconds + " ms."); } catch (Exception e) { Console.WriteLine(x + " Exception: " + e.Message); } })).ToArray(); await Task.WhenAll(tasks); sw.Stop(); Console.WriteLine(sw.ElapsedMilliseconds); } // almost all of 800 requests time out [Test] public void Syncmethod() { var sw = new Stopwatch(); sw.Start(); var client = new HttpClient(); var tasks = Enumerable.Range(0, 800).Select(x => Task.Run(() => { try { var swx = new Stopwatch(); swx.Start(); var response = client.GetStringAsync("http://localhost:7002/TestMethod"); if (response.Wait(15000)) { swx.Stop(); Console.WriteLine(x + " " + response.Result + " in " + swx.ElapsedMilliseconds + " ms."); } else { swx.Stop(); Console.WriteLine(x + " timed out."); } } catch (Exception e) { Console.WriteLine(x + " Exception: " + e.Message); } })).ToArray(); foreach (var task in tasks) task.Wait(60000); sw.Stop(); Console.WriteLine(sw.ElapsedMilliseconds); }

Stephen Cleary · Accepted Answer · 2013-10-22 12:53:49Z

8

I do recommend that you use await for HttpClient instead of Wait. If you truly want synchronous requests, consider using WebClient which supports synchronous methods.

That said, I believe the reason you're seeing that behavior is due to ServicePointManager.DefaultConnectionLimit, which will throttle the number of requests to a given server. If you want to have large numbers of concurrent requests to the same server (synchronous or asynchronous), then you'll need to increase that limit (it is 2 by default for UI applications).

answered Oct 22, 2013 at 12:53

Stephen Cleary

460k78 gold badges721 silver badges858 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Paulo Morgado Over a year ago

await - after all the question is tagged C#-5.0

Thresh Over a year ago

The problem is, if there is no timeout, none of the requests is completed. I get around 20 completed requests only after 780 are canceled due to timeout.

Thresh Over a year ago

Using await in not an option, I'll have to rewrite everything else too. Yes, WebClient works perfectly. But I still don't understand why HttpClient hangs.

Panagiotis Kanavos Over a year ago

The DefaultConnectionLimit is a VERY good reason to hang. The default is 2 which means that getting even 20 completed requests is not bad. Just increase the limit

Community · Accepted Answer · 2017-05-23 11:46:37Z

Yes, It's seems like deadlock.

This code works perfect

public static void Syncmethod() { var sw = new Stopwatch(); sw.Start(); var client = new HttpClient(); var tasks = Enumerable.Range(0, 800).Select(x => Task.Run(() => { var swx = new Stopwatch(); swx.Start(); var result = client.GetStringAsync("http://yandex.ru").ContinueWith(task => { try { swx.Stop(); Console.WriteLine(x + " " + task.Result + " in " + swx.ElapsedMilliseconds + " ms."); return task.Result; } catch (Exception e) { swx.Stop(); Console.WriteLine(x + " Exception: " + e.Message); throw e; } }, TaskContinuationOptions.AttachedToParent); })).ToArray(); foreach (var task in tasks) task.Wait(60000); sw.Stop(); Console.WriteLine(sw.ElapsedMilliseconds); }

I guess you need additional logic to handle timeout. May be one more task that completes in 15 sec, for example.

Look to this post HttpClient.GetAsync(...) never returns when using await/async

And as described above you may hang your async example by simple change

var response = client.GetStringAsync("http://yandex.ru").GetAwaiter().GetResult();

When you write async code never ever use block execution and await result.

First of all, in your example the task will be completed before the request is completed if request is long enough. Second, how would I write a function that returns something from response using ContinueWith?
Updated. But you have to rewrite you code to use Task<string> without waiting.

Collectives™ on Stack Overflow

Synchronous Concurrent HttpClient Usage

2 Answers 2

4 Comments

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

2 Comments

Linked

Related