This is not possible. Once you expose an endpoint publicly, you can always try to connect to it with whatever software supporting the protocol (or an analyzer which will reverse-engineer the protocol).
This said, you can make it difficult by obfuscation.
One solution is to use information provided by the client (browser in your case). This is usually the agent header (as you mention) but can also be some fingerprinting. You could for instance detect the size of the screen and make some decisions from there on.
Another solution is to use some secrets (better call them "secrets") to show the backend application (or API) that some specific code is calling. I can imagine that you could bundle the HTML/JS/CSS code in your application, together with a key so only the application owners will be able to send a recognizable traffic (recognized by the backednd).
All of this is more or less difficult to crack but with some layers you will get rid of at least some population which would like to access the site via other means than an app.