8

I followed this post on Stackoverflow to disable Firefox WebDriver detection.

Launch Geckodriver:

System.setProperty("webdriver.gecko.driver", geckdriverExecutableFilePath); File firefoxProfileFile = new File(fullPathOfFirefoxInstallationFolder); FirefoxProfile firefoxProfile = null; try { firefoxProfile = new FirefoxProfile(firefoxProfileFile); } catch (Exception e) { e.printStackTrace(); }

I disabled WebDriver:

WebDriver Disabled

FirefoxOptions firefoxOptions = new FirefoxOptions(); firefoxOptions.setProfile(firefoxProfile); // Disables WebRTC firefoxProfile.setPreference("media.peerconnection.enabled", false);

I disabled Automation Extensions:

Automation Extension Disabled

// Disables Automation Extension firefoxProfile.setPreference("useAutomationExtension", false);

I added Proxy:

 DesiredCapabilities dc = DesiredCapabilities.firefox(); Proxy proxy = new Proxy(); proxy.setHttpProxy(ipAddress + ":" + port); proxy.setFtpProxy(ipAddress + ":" + port); proxy.setSslProxy(ipAddress + ":" + port); dc.setCapability(CapabilityType.PROXY, proxy); firefoxOptions.merge(dc); driver = new FirefoxDriver(firefoxOptions);

Yet BotD still detects my browser as being controlled by automation tool.

BotD Detection

How can I solve this?

Improve this question
3
  • Update the question with your code trials. Commented Nov 14, 2021 at 18:43
  • Would you like to evaluate an answer in Python? Commented Nov 21, 2021 at 20:56
  • 1
    @DebanjanB python is OK IF the code can be easily ported to Java. Meaning, if code is using some special function that is only available in python then it won't be of use to me as my entire code base is in Java. Thanks in advance for your response! You seem to be the most knowledgeable user on this forum regarding Selenium :) Commented Nov 22, 2021 at 6:18

3 Answers 3

Reset to default
8
+50

When using Selenium driven GeckoDriver initiated Browsing Context

The webdriver-active flag is set to true when the user agent is under remote control. It is initially false.

webdriver-active flag

where, webdriver returns true if webdriver-active flag is set, false otherwise.

As:

navigator.webdriver Defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, for example so that alternate code paths can be triggered during automation.

Further @whimboo in his comments confirmed:

This implementation have to be conformant to this requirement. As such we will not provide a way to circumvent that.

Conclusion

So, the bottom line is:

Selenium identifies itself

and there is no way to conceal the fact that the browser is WebDriver driven.

Recommendations

However some pundits have suggested some different approaches which can conceal the fact that the Mozilla Firefox browser is WebDriver controled through the usage of Firefox Profiles and Proxies as follows:

compatible code

from selenium.webdriver import Firefox from selenium import webdriver from selenium.webdriver.firefox.service import Service from selenium.webdriver.firefox.options import Options profile_path = r'C:\Users\Admin\AppData\Roaming\Mozilla\Firefox\Profiles\s8543x41.default-release' options=Options() options.set_preference('profile', profile_path) options.set_preference('network.proxy.type', 1) options.set_preference('network.proxy.socks', '127.0.0.1') options.set_preference('network.proxy.socks_port', 9050) options.set_preference('network.proxy.socks_remote_dns', False) service = Service('C:\\BrowserDrivers\\geckodriver.exe') driver = Firefox(service=service, options=options) driver.get("https://www.google.com") driver.quit()

Potential Solution

A potential solution would be to use the browser as follows:

compatible code

from selenium.webdriver import Firefox from selenium import webdriver from selenium.webdriver.firefox.service import Service from selenium.webdriver.firefox.options import Options import os torexe = os.popen(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe') profile_path = r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default' firefox_options=Options() firefox_options.set_preference('profile', profile_path) firefox_options.set_preference('network.proxy.type', 1) firefox_options.set_preference('network.proxy.socks', '127.0.0.1') firefox_options.set_preference('network.proxy.socks_port', 9050) firefox_options.set_preference("network.proxy.socks_remote_dns", False) firefox_options.binary_location = r'C:\Users\username\Desktop\Tor Browser\Browser\firefox.exe' service = Service('C:\\BrowserDrivers\\geckodriver.exe') driver = webdriver.Firefox(service=service, options=firefox_options) driver.get("https://www.tiktok.com/")

References

You can find a couple of relevant detailed discussions in

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Upvoted for adding solution with TOR and for thinking out of the box..
+1 thoroughly impressed with your suggestion to bundle TOR with Firefox. A user on SO suggested that it is possible modify chromedriver source code to remove any bot-identifying attributes. Is it possible to achieve a similar outcome w/ Firefox by either modifying Geckodriver or Firefox WebDriver source code? We can then remove bot-identification without needing to bundle TOR with Firefox? You mentioned Selenium identifies itself but surely we can modify source code to remove identification similar to how it's done in chrome driver?
@BradfordGriggs If you can raise a question I'll be happy to post an answer.
@DebanjanB I awarded your response with +50 bounty. I raised another question about removing bot identifying features from Geckodriver / WebDriver source code. I will add another +50 bounty if you can think of a solution to that question!
1

Because my entire code base (thousands of lines of code) is written for Firefox. In the future I hope to add support for Chrome but in either case I do not want to give up using Firefox just because I can't solve one problem. I am hoping to find a way around this.. There must be a way..

Because the OP really wants an answer, I'm going to give an answer.

  • Install Violentmonkey addon to your geckodriver
  • Install this script:
// ==UserScript== // @name Webdriver Get Rekt // @author noname // @namespace http://www.example.url/to/your-web-site/ // @description Put a good description in here // @license Creative Commons Attribution License // @version 0.1 // @include http*://*/* // @run-at document-start // @compatible Greasemonkey // ==/UserScript== Object.defineProperty(navigator, 'webdriver', {get: () => undefined})

Should be able to override navigator.webdriver BEFORE page load, which eliminate driver detection.

Improve this answer

Comments

0

BotD detects you because you do not override navigator.webdriver attribute.

I was able to override it with this code:

((JavascriptExecutor)driver).executeScript("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})");

Re-run your code with this line after driver.get("BotD url") and click on 'Start detect' on the BotD page.

It will no longer show that webdriver is detected.

I understand you are looking for a way to make it work before the initial page load.

But here are 2 things to consider:

  1. Webdriver developers want their tool to be detected by browsers.
  2. Gecko driver developers are not going to implement an option to disable navigator.webdriver attribute. (This is the official reply from gecko developer.)
Improve this answer

3 Comments

I understand you are looking for a way to make it work before the initial page load Correct. Overriding attribute after the page already loaded is of no value as the browser will already be detected as a bot. I know gecko driver developers did not natively incorporate this option but I believe it's still possible to achieve this if we tinker with the underlying code - similar to how it can be achieved with Chrome.
So you are looking for some advice on how to hack the driver? Why you don't want to switch to Chrome if it has, what you need?
Because my entire code base (thousands of lines of code) is written for Firefox. In the future I hope to add support for Chrome but in either case I do not want to give up using Firefox just because I can't solve one problem. I am hoping to find a way around this.. There must be a way..

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.