Selenium Unable to process further after executing script

Question

This question is actually continued from here I have following link I got help from community regarding using script in selenium but main part here is I want to collect atleast 2-3 news and store in some json which can further be manipulated. I am not able to click the get those news section which is in bottom part of page any way I can scrape those news at least 1-2 new?

 stocks_data=["AKPL","NICA"] for stock_data in stocks_data: self.driver.get(f'https://merolagani.com/CompanyDetail.aspx?symbol={stock_data[0].lower()}') self.driver.execute_script( "document.getElementById('ctl00_ContentPlaceHolder1_CompanyDetail1_lnkNewsTab').click()")

Md. Fazlul Hoque · Accepted Answer · 2022-06-27 15:43:30Z

1

Use stocks_data=["AKPL","NICA"] instead of stocks_data="AKPL","NICA"]

answered Jun 27, 2022 at 15:43

Md. Fazlul Hoque

16.2k5 gold badges15 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

EvgenyKolyakov · Accepted Answer · 2022-06-27 17:22:55Z

As for the multiple news, just use a loop over document.querySelectorAll('[id*="_lnkNewsTab"]') or some similar selector if that one is too wide.

As for the json, try to populate some imaginary div (that you can create in Javascript by document.body.innerHTML += '<div id="my_json"></div>') with the content of the elements you need (document.getElementById('my_json').innerHTML += document.getElementById('the id of the element you want the data from').innerHTML;) and finally get that div from python (see How to get text with Selenium WebDriver in Python)

Then you can manipulate however you like in python.

Something like:

 stocks_data=["AKPL","NICA"] for stock_data in stocks_data: self.driver.get(f'https://merolagani.com/CompanyDetail.aspx?symbol={stock_data[0].lower()}') self.driver.execute_script("document.getElementById('ctl00_ContentPlaceHolder1_CompanyDetail1_lnkNewsTab').click()") self.driver.execute_script(""" document.body.innerHTML += '<div id="my_json"></div>'; var x = document.querySelectorAll('[id*="_lnkNewsTab"]'); for (let i = x.length - 1; i >= 0; --i) { document.getElementById('my_json').innerHTML += x[i].innerHTML + '--DELIMITER--' }); """) content = self.driver.find_element_by_id("my_json") content = content.text.split('--DELIMITER--')[:-1]

` ``self.driver.execute_script("document.querySelectorAll('[id*=\"_lnkNewsTab\"]')") for z in self.driver.find_elements_by_xpath("//table[@class='table table-striped table-hover']//tr//a"): print(z.click())``` I did upto here but no result
I dont know js but I got this errro ` TypeError: x.map is not a function`

Collectives™ on Stack Overflow

Selenium Unable to process further after executing script

2 Answers 2

Comments

8 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

8 Comments

Linked

Related