Python Selenium: Scrolling not working

Question

I am trying to automate this Instagram link. I need to scroll and scroll and fetch all links. I am trying following but not working.

def fetch_links_by_hashtag(hash_tag): url = 'https://www.instagram.com/explore/tags/marketing/' driver.get(url) driver.implicitly_wait(20) is_more = False try: elem_more = wait.until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "Load more"))) elem_more.click() is_more = True except Exception as ex: print(str(ex)) pop = driver.find_element_by_tag_name('footer') #pop = driver.find_element_by_link_text('About us') # pop = driver.find_element_by_class_name('_4gt3b') if pop is not None: for i in range(10): print('Calling scrolling script') # It scolls till end driver.execute_script('arguments[0].scrollTop = arguments[0].scrollHeight', pop) sleep(4) html = pop.get_attribute('innerHTML') print(html)

宏杰李 · Accepted Answer · 2017-01-20 07:28:07Z

1

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

How to scroll down to the bottom of a page ?

answered Jan 20, 2017 at 7:28

宏杰李

12.2k2 gold badges32 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Yonatan Kiron · Accepted Answer · 2017-01-22 15:15:00Z

In addition to 宏杰李 answer

driver.execute_script("return arguments[0].scrollIntoView();", element_obj)

Also, if you want to make an extra scroll:

driver.execute_script("return arguments[0].parentNode.scrollTop = " "arguments[0].parentNode.scrollTop + {extra_scroll}" .format(extra_scroll=extra_scroll_pixels), element_obj)

My entire code:

def _scroll_to_element(driver, element, extra_scroll=None): # Scroll to element driver.execute_script("return arguments[0].scrollIntoView();", element) # Scroll parentNode with the extra pixels (If provided) if extra_scroll: driver.execute_script( "return arguments[0].parentNode.scrollTop = " "arguments[0].parentNode.scrollTop + {extra_scroll}".format( extra_scroll=str(extra_scroll)), element)

His it different than what I did? also what is element_obj?
@Volatil3 It's a bit different; 1. see the return (might make a different) 2. I'm calling parentNode 3. element_obj is what you called pop (The element you want to scroll to). See the method that I use (Never had any problems)

Collectives™ on Stack Overflow

Python Selenium: Scrolling not working

2 Answers 2

Comments

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Related