You can't; it is linked with the X and Gtk+ client libraries, and I don't think it's even possible to build it without them.

Even in the Firefox headless version, [GTK+ is still needed][1].


 [1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1372998