Selenium, python dynamic table

Question

I'm creating a robot with selenium that get all info from agencies in Brasil, i've alredy done the permutation click between all States and counties, all i have to do now's click in all agencies and get infos like: "telefone","endereço","HORÁRIO DE ATENDIMENTO". How can I do that?? That's my code till now:

 # -*- coding: UTF-8 -*- from selenium import webdriver from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support.select import Select import time import urllib import urllib.request from bs4 import BeautifulSoup import request def make_soup(url): thepage = urllib.request.urlopen(url) soupdata = BeautifulSoup(thepage,'html.parser') return soupdata chrome_path = r"D:\Users\gbenites\Desktop\Inovação\arquivos py\WebDriver\chromedriver.exe" url = 'http://www2.correios.com.br/sistemas/agencias/' driver = webdriver.Chrome(chrome_path) driver.get("http://www2.correios.com.br/sistemas/agencias/") #WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.ID, 'municipioAgencia'))) driver.maximize_window() time.sleep(3) estadoList = Select(driver.find_element_by_name('estadoAgencia')) for index in range(3,len(estadoList.options)): select = Select(driver.find_element_by_name('estadoAgencia')) print("selecting Estado: ",select.options[index].text) print("selecting Index: ",index) select.select_by_index(index) time.sleep(2) municípioList = Select(driver.find_element_by_name('municipioAgencia')) for index_b in range(1,len(municípioList.options)): select_b = Select(driver.find_element_by_name('municipioAgencia')) print("...selecting Municipio",select_b.options[index_b].text.replace("'","")) print("...selecting Municipio Index: ",index_b) select_b.select_by_index(index_b) time.sleep(3) get_info_list = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, "//table[@class = 'dadosAgencia']//td"))) info_list = driver.find_elements_by_xpath("//table[@class = 'dadosAgencia']//span") print(range(len(info_list))) for record in range(0,len(info_list)): print(record.get_attribute('innerText'))

And that's my output:

selecting Estado: AMAPÁ selecting Index: 3 ...selecting Municipio AMAPA ...selecting Municipio Index: 1 range(0, 27) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Please do not link only image. Edit your question and write in text what do you want. — Andrej Kesely
– Andrej Kesely, Commented Jul 10, 2018 at 21:18
Please read why a screenshot of HTML or code or error is a bad idea. Consider updating the Question with formatted text based relevant HTML, code trials and error stack trace. — undetected Selenium
– undetected Selenium, Commented Jul 11, 2018 at 8:10

Andrei · Accepted Answer · 2018-07-11 06:31:57Z

Try this xPath:

//table[@class = 'dadosAgencia']//tr

It would be like this:

elements = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located( (By.XPATH, "//table[@class = 'dadosAgencia']//tr")))

it gives you a list of all elements located. To print the text of each element you can use this:

for element in elements: print(element.text)

Note: you have to add some imports:

from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By from selenium.webdriver.support.wait import WebDriverWait

that will only get to detalheAgencia 1, not all detalheAgencia[i]

Monika · Accepted Answer · 2018-07-11 04:51:49Z

Try this XPath for the table:

//table[@class = 'dadosAgencia']

First, find table elements:

table_elements = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located( (By.XPATH, "//table[@class = 'dadosAgencia']")))

Iterate through the table:

for table_element in table_elements: for row in table_element.find_elements_by_xpath(".//tr"): print(row.text)

Collectives™ on Stack Overflow

Selenium, python dynamic table

2 Answers 2

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Linked

Related