Beautifulsoup parse Selenium element

Question

I use soup = BeautifulSoup(driver.page_source) to parse the whole page from Selenium in BeautifulSoup.

But how to just parse one element of Selenium in BeautifulSoup.

Below code will throw

TypeError: object of type 'FirefoxWebElement' has no len()

element = driver.find_element_by_id(id_name) soup = BeautifulSoup(element)

@AhmedI.Elsayed element.text will just return the text, and lost all the tag. — LF-DevJourney
– LF-DevJourney, Commented Oct 30, 2019 at 2:23
You want to get an element say like <div> Hi </div> , right? — Ahmed I. Elsayed
– Ahmed I. Elsayed, Commented Oct 30, 2019 at 2:41

Ahmed I. Elsayed · Accepted Answer · 2019-10-30 03:34:00Z

I don't know if selenium does this out of the box, but I managed to find this workaround

element_html = f"<{element.tag_name}>{element.get_attribute('innerHTML')}</{element.tag_name}>"

you may want to replace innerHTML with innerTEXT if you want to get only the text, for example

<li>Hi <span> man </span> </li>

Getting the innerHTML will return all of what inside but the innerTEXT won't, try & see.

now create your Soup object

soup = BeautifulSoup(element_html) print(soup.WHATEVER)

using the above technique, just create a method parseElement(webElement) & use it whenever you want to parse an element.

Btw I only use lxml & when I forgot to type it, the script didn't work

1 Answer 1