Section 14.13 of the HTTP/1.1 spec detailed the Content-Length header, and says this:
Applications SHOULD use this field to indicate the transfer-length of the message-body, unless this is prohibited by the rules in section 4.4.
The word 'SHOULD' has a very specific meaning in RFCs:
- SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
So, you may not always see a Content-Length. Typically you might not see it for any content which is dynamically generated, since that might be too expensive to service an exploratory HEAD request. For example, a HEAD request to Apache for a static file will have a Content-Length, but a request for a PHP script may not.
For example, try this very website...
telnet stackoverflow.com 80 HEAD / HTTP/1.0 Host:stackoverflow.com HTTP/1.1 200 OK Date: Mon, 11 Jan 2016 10:58:25 GMT Content-Type: text/html; charset=utf-8 Connection: close Set-Cookie: __cfduid=c2eb4742a1e02d89cab0402220736c0bd1452509905; expires=Tue, 10-Jan-17 10:58:25 GMT; path=/; domain=.stackoverflow.com; HttpOnly Cache-Control: public, no-cache="Set-Cookie", max-age=36 Expires: Mon, 11 Jan 2016 10:59:02 GMT Last-Modified: Mon, 11 Jan 2016 10:58:02 GMT Vary: * X-Frame-Options: SAMEORIGIN X-Request-Guid: 487e80bc-3783-4cfd-d883-a3bc84253234 Set-Cookie: prov=8dc24306-c067-45eb-bf5d-cffa855c2b03; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly Server: cloudflare-nginx CF-RAY: 26303c15f8e035a2-LHR
No content-length there.