I'm making a simple c# program to establish if server side compression is available/enabled on various servers. This is my request code below.
using (var client = new HttpClient()) { client.Timeout = new TimeSpan(0, 0, 5); client.DefaultRequestHeaders.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"); client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36"); client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate, sdch, br"); var httpClientResponse = client.GetAsync(websiteURL).Result; string header = httpClientResponse.Headers.GetValues("Content-Encoding").First(); } I can see by watching the request in fiddler that compression is enabled for this request however I can't seem to extract that information from the response headers in code.
These are the full headers for request and response.
GET https://www.dobbies.com/ HTTP/1.1 Accept: text/html, application/xhtml+xml, application/xml; q=0.9, image/webp, */*; q=0.8 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36 Accept-Encoding: gzip, deflate, sdch, br Host: www.dobbies.com HTTP/1.1 200 OK Cache-Control: private Content-Type: text/html; charset=utf-8 Content-Encoding: gzip Vary: Accept-Encoding Server: Set-Cookie: ASP.NET_SessionId=hx1rb34ottgfritgt3rciql4; path=/; secure; HttpOnly X-Frame-Options: SAMEORIGIN X-Content-Type-Options: nosniff Strict-Transport-Security: max-age=31536000 X-Xss-Protection: 1; mode=block Date: Fri, 07 Apr 2017 08:06:17 GMT Content-Length: 16836 This is what I get when I use httpClientResponse.Headers
{ Vary: Accept-Encoding X-Frame-Options: SAMEORIGIN X-Content-Type-Options: nosniff Strict-Transport-Security: max-age=31536000 X-Xss-Protection: 1; mode=block Cache-Control: private Date: Fri, 07 Apr 2017 08:06:17 GMT Set-Cookie: ASP.NET_SessionId=hx1rb34ottgfritgt3rciql4; path=/; secure; HttpOnly Server: } As you can see header of Content-Encoding: gzip is missing in the response.
Why would this header be missing? Along with others. Give me back my headers!