3

I use Python Beautiful Soup to move through DOM elements and Selenium to open page in chrome and ActionChains to scroll in page. It worked fine but the website changed something and now I run into two kinds of error on the way it worked before and on a new way as a possible solution.

Old solution:

submit_button = driver.find_element_by_name('quantity') elementList = submit_button.find_elements_by_tag_name("option") elementList[int(column)-1].click()

Old Error:

Traceback (most recent call last): File "C:/Users/Felix_P/PycharmProjects/Price_Tool/Combination_14_6_2016.py", line 205, in <module> elementList[int(column)-1].click() File "C:\Python35-32\lib\site-packages\selenium\webdriver\remote\webelement.py", line 75, in click self._execute(Command.CLICK_ELEMENT) File "C:\Python35-32\lib\site-packages\selenium\webdriver\remote\webelement.py", line 469, in _execute return self._parent.execute(command, params) File "C:\Python35-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 201, in execute self.error_handler.check_response(response) File "C:\Python35-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 193, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.ElementNotVisibleException: Message: element not visible: Element is not currently visible and may not be manipulated (Session info: chrome=51.0.2704.103) (Driver info: chromedriver=2.21.371459 (36d3d07f660ff2bc1bf28a75d1cdabed0983e7c4),platform=Windows NT 6.1 SP1 x86_64)

New Solution:

submit_button = driver.find_element_by_name('quantity') elementList = submit_button.find_elements_by_tag_name("option") actions.move_to_element(elementList[int(column)-1]).click().perform()

New Error:

Traceback (most recent call last): File "C:/Users/Felix_P/PycharmProjects/Price_Tool/Combination_14_6_2016.py", line 201, in <module> actions.move_to_element(elementList[int(column)-1]).click().perform() File "C:\Python35-32\lib\site-packages\selenium\webdriver\common\action_chains.py", line 72, in perform action() File "C:\Python35-32\lib\site-packages\selenium\webdriver\common\action_chains.py", line 217, in <lambda> self._driver.execute(Command.MOVE_TO, {'element': to_element.id})) File "C:\Python35-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 201, in execute self.error_handler.check_response(response) File "C:\Python35-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 193, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document (Session info: chrome=51.0.2704.103) (Driver info: chromedriver=2.21.371459 (36d3d07f660ff2bc1bf28a75d1cdabed0983e7c4),platform=Windows NT 6.1 SP1 x86_64)
Improve this question
2
  • My guess is that there's a timing issue. has the script taken some action prior to these lines that would cause ANY <option> element to disappear? You might want to use a more specific path to find the options that are really of interest to you. Commented Jul 19, 2016 at 11:23
  • If I run the script on two product page versions of the same kind. The first attempt with "similar" website works. Second try does not work. Does that maybe lead to a more clear problem understanding? Commented Jul 19, 2016 at 11:44

2 Answers 2

Reset to default
3

From you error stack, it shows that your reserved reference were stale. This usually is caused by page refresh or enven a page event.

So you have to get the element again before access it.

For example,

elements = driver.find_elements_by_tag_name('tagname')) for element in elements: driver.get(element.get_attribute('url'))

will cause

selenium.common.exceptions.StaleElementReferenceException

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

I think I see the problem. If you show all of your code, it will probably show that last line is within a for loop that is navigating away from the page, then coming back to the same page (or some ajax has refreshed that set of options on the page). The problem is that the page has reloaded, so the target element from elementList doesn't exist any longer.

You'll need to define your elementList by keeping track of a list of something like value on the option, then use find_element within the loop to find each unique option element again.

Improve this answer

2 Comments

I don't think that is the reason. I debugged it and element list is filled correctly. It has to be something else probably
Your element list will still be filled, the problem is that those objects in the list are no longer valid once you navigate away from the page then come back again.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.