All I had to do iswas:
my_options = webdriver.ChromeOptions() my_options.add_argument( '--disable-blink-features=AutomationControlled' ) Some more infoinformation to this: This relates to website skyscanner.com. In the past I have been able to scrape it. Yes, it did detect the browser automation and it gave me a captcha to press and hold a button. I used to be able to complete the captcha manually, then search flights and then scrape. But this time around after completing the captcha I get the same captcha again and again, just can't seem to escape from it. I tried some of the most popular suggestions to avoid automation being detected, but they didn't work. Then I found this article which did work, and by process of elimination I found out it only took the option above to get around their browser automation detection. Now I don't even get the captcha and everything else seems to be working normally.
Versions I am running currently:
- OS: Windows 7 64bitWindows 7 64 bit
- Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50-10-14) [MSC(MSC v.1916 64 bit (AMD64)]) on win32
- Browser: Chrome Version 100.0.4896.60 (Official Build) (64-bit)
- Selenium 4.1.3
- ChromeDriver 100.0.4896.60 chromedriver_win32.zip 930ff33ae8babeaa74e0dd1ce1dae7ff