10

I am trying to create a script using python and selenium to automate the checkout process at bestbuy.ca.

I get all the way to the final stage where you click to review the final order, but get the following 403 forbidden message (as seen in the network response) when I try to click through to the final step.

Is there something server side that has detected that I am using selenium and preventing me to proceed?

How can I hide the fact that it is selenium being used?

These are the options I am using for selenium:

options = Options() options.add_argument('--disable-blink-features=AutomationControlled') options.add_argument("start-maximized") options.add_argument("--disable-extensions") driver = webdriver.Chrome(options=options)

I currently have 10 second delays after each action(ie open page, wait, click add to cart, wait, click checkout, wait)

I have implemented a random useragent to be used on each run:

import fake_useragent ua = UserAgent() userAgent = ua.random options.add_argument(f'user-agent={userAgent}')

I have also modified my chromedriver binary as per the comments in THIS THREAD

Error seen when proceeding to order review page: Error seen when proceeding to order review page

Improve this question
9
  • 1
    Have you tried with user-agent? Commented Apr 7, 2021 at 16:09
  • 2
    Best guess is this actually due to timing issues - sending in selenium actions faster than the page can react to them. Try inserting some delays. Commented Apr 7, 2021 at 16:09
  • 1
    Does the same happen in debug mode too? Commented Apr 7, 2021 at 17:04
  • 1
    @DavidK.Hess I currently have 10 second delays after each action (ie open page, wait, click add to cart, wait, click checkout, wait) Commented Apr 7, 2021 at 17:18
  • 1
    Are you logged-in to bestbuy.ca or processing the order as a guest? Commented Apr 12, 2021 at 3:06

3 Answers 3

Reset to default
15

After much testing the last few days, here are the options that have allowed me to bypass the restrictions I was facing.

  1. Modified cdc_ string in my chromedriver

  2. Chromedriver options:

     options.add_argument('--disable-blink-features=AutomationControlled') options.add_argument("--disable-extensions") options.add_experimental_option('useAutomationExtension', False) options.add_experimental_option("excludeSwitches", ["enable-automation"]) chrome_driver = webdriver.Chrome(options=options)

  3. Change the property value of the navigator for webdriver to undefined:

chrome_driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")

After all three of these were implemented I no longer faced any 403 error when navigating the site and the cart/checkout process.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks for the question and answer. I had been facing a similar issue for some time. The "--disable-blink-features=AutomationControlled" did the trick for me. Thank you!
1

In my case, either using code to control the browser, or simply starting Chrome through python and manually using the browser always leads to the 403 error, even just adding a product to the cart.

As you said, I think that this site someway knows that the user is using Selenium or some sort of automation tool and the server is blocking API requests.

Searching in stackoverflow I found this https://stackoverflow.com/a/52108199/3228768 but editing the chromedriver results anyway in a failure.

The only way I completed the flow is settings this options:

u = 'https://www.bestbuy.ca/en-ca/category/appliances/26517' # relevant part start here options = webdriver.ChromeOptions() options.add_argument("--disable-blink-features") options.add_argument("--disable-blink-features=AutomationControlled") # relevant part ends here driver = webdriver.Chrome(executable_path=r"chromedriver.exe", options=options) driver.maximize_window() driver.get(u)

In this way I managed to add a product to the cart. I think you could use it to proceed the flow until checkout. Let me know.

Improve this answer

3 Comments

Hey thanks for the reply - Yes, that option was indeed what I had used in my original post to enable me to add it to the cart (as well as the modified chrome driver). My issue unfortunately lies when browsing to to the final checkout process page to order.
Really sorry, I was so happy that my tests were (at firts) successfull I forgot to check if you already had implemented my solution. Glad to see you resolved it anyway.
all good, thanks! Yes, glad I was able to figure it out!
0

Try this one: https://github.com/ultrafunkamsterdam/undetected-chromedriver

It avoids the selenium detection quite well, I've been having good result with it so far. Headless is not guaranteed though.

import undetected_chromedriver as uc driver = uc.Chrome() driver.get('https://www.bestbuy.ca/')
Improve this answer

1 Comment

Thanks for the reply, I might give this a shot if all else fails!

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.