TL;DR - Checking the existence of a non-standard header like "X-Requested-By" should be sufficient to guard against CSRF attacks without checking the value of the header.
Non-standard headers cannot be set in a CSRF attack
The Play framework site breaks it down really well:
Simply put, an attacker can coerce a victims browser to make the following types of requests:
- All GET requests
- POST requests with bodies of type application/x-www-form-urlencoded, multipart/form-data and text/plain
An attacker can not:
- Coerce the browser to use other request methods such as PUT and DELETE
- Coerce the browser to post other content types, such as application/json
- Coerce the browser to send new cookies, other than those that the server has already set
- Coerce the browser to set arbitrary headers, other than the normal headers the browser adds to requests
This makes sense if you consider the attack vectors for CSRF:
- GET requests (e.g. <img>, <iframe>) - which cannot set headers.
- <form> submitted by a user click - which are limited to a few specific headers.
- <form> submitted by JavaScript (HTMLFormElement.submit()) - which are limited to a few specific headers.
JavaScript is subject to the same-origin policy, so it can only add non-standard headers if one of the following conditions hold:
- it is "in-domain" (i.e. loaded from the same domain as the target of the request).
- it is allowed to do so through CORS.
XSS attacks are out of scope for this question
Non-standard headers can be set in an XSS attack. Using a non-standard header to prevent CSRF attacks does not make a site any more (or any less) vulnerable to XSS attacks regardless of the value of the header. Both non-standard headers and CSRF tokens are vulnerable to XSS attacks. If the XSS attacker can set a non-standard header on a request (e.g. in-domain XHR), he/she can certainly gain access to a CSRF token set in a cookie or embedded in DOM or in a JavaScript variable.
Reference
There is a similar SO question herehere which is confusing but came to the same conclusion.
Some examples of such non-standard headers in the wild:
- "X-Requested-By" (mentioned by OP) recognized by Jersey/others
- "X-Requested-With" set by jQuery
- "X-XSRF-TOKEN" set by Angular
- "X-CSRF-TOKEN" recognized by the Play framework