Python - Selenium find_element_by_xpath("") displays "expected string or bytes-like object"

Question

Application versioning:
Python: v3.5.4

Hello and good day!

I'm trying to scrape Facebook posts and fetch the number of comments a post has with the following code:

from selenium import webdriver from bs4 import BeautifulSoup from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait # Omitted some codes here # To summarize, the codes above let me log in to my dummy account in Facebook # Codes above tested if there's an existing cookie, or a new log-in instance, then refresh to FB fb = webdriver.Firefox() fb.get("https://www.facebook.com/") fb.get(url) comments = fb.find_elements_by_xpath('//div[@data-testid="fbFeedStoryUFI/feedbackSummary"]') print(comments)

But when I run my python script it is telling me this:

expected string or bytes-like object

Formerly, I was able to scrape the number of reactions a post has with this code:

reactions = fb.find_element_by_xpath('//span[@data-testid="UFI2ReactionsCount/sentenceWithSocialContext"]').click() modal = fb.find_element_by_xpath('//ul[@defaultactivetabkey="all"]').get_attribute('innerHTML') soup = BeautifulSoup(modal, 'html.parser')

And there were no errors, am I doing something wrong here? I would like to thank you in advance for those who can help me :)

hello sir @Guy it's on the line where i print the comments variable — Suomynona
– Suomynona, Commented Sep 16, 2019 at 9:18

Misieq · Accepted Answer · 2019-09-16 06:31:25Z

1

comments = fb.find_elements_by_xpath('//div[@data-testid="fbFeedStoryUFI/feedbackSummary"]')

this line, return whole list of objects, and in next line you are trying to print it while python expects string or byte. What I would have done:

comments = [] number_of_comments = len(fb.find_elements_by_xpath('//div[@data-testid="fbFeedStoryUFI/feedbackSummary"]')) for x in range(1, number_of_comments): # indexes of div/li and other markers starts at 1 comment = fb.find_element_by_xpath(f'//div[@data-testid="fbFeedStoryUFI/feedbackSummary"]/li{x}') # something like that, you need to check xpath comments.append(comment)

answered Sep 16, 2019 at 6:31

Misieq

5552 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Suomynona Over a year ago

it's displaying

line 129 comment = fb.find_element_by_xpath(f'//div[@data-testid="fbFeedStoryUFI/feedbackSummary"]/li{x}') # something like that, you need to check xpath ^

error around the {x}

RegCent Over a year ago

@JueViole17 I think you have to separate the variable eg '//div[@data-testid="fbFeedStoryUFI/feedbackSummary"]/li{'+x+'}'

SuperSimplePimpleDimple Over a year ago

@JueViole17 When using a variable in a string statement, you should parse it. End the string, add the casted version of the variable to the string and then add the rest of the string.

SuperSimplePimpleDimple Over a year ago

Although Python will tolerate non casted variables with different types being added, it can be the cause for some buggs later on.

Misieq Over a year ago

if you are adding f before 'some text {f}' then it will work fine, here's sample of webscrapper that work perfectly in my current project: job_title_xpath = f'{xpath_base}[{i}]/offer-item/a/div[2]/div[1]/span'

|

frianH · Accepted Answer · 2019-09-16 09:47:50Z

To get number of comments you can use:

.find_elements_by_css_selector('._3hg-._42ft'), and use iteration to extract the text.

But it will return some top post, because the posted below will appear if you scroll first.

comments = driver.find_elements_by_css_selector('._3hg-._42ft') for number in comments: print(number.text)

Collectives™ on Stack Overflow

Python - Selenium find_element_by_xpath("") displays "expected string or bytes-like object"

2 Answers 2

6 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Related