Linked Questions
35 questions linked to/from python BeautifulSoup parsing table
0 votes
2 answers
384 views
I'm having trouble pulling data from a table using BeautifulSoup [duplicate]
I'm trying to scrape two tables using beautifulsoup and running into a brick wall. Website: https://bgp.he.net/country/US I'm trying to grab the header row from the table, but for some reason can't ...
0 votes
1 answer
122 views
Once done using BeautifulSoup4, how do I make soup object further filtered? [duplicate]
I am trying to learn how to parse HTML data so I've chosen this website (http://ets.aeso.ca/ets_web/ip/Market/Reports/CSMPriceReportServlet) which has real time data for electricity prices. from ...
40 votes
3 answers
150k views
BeautifulSoup: Get the contents of a specific table
My local airport disgracefully blocks users without IE, and looks awful. I want to write a Python scripts that would get the contents of the Arrival and Departures pages every few minutes, and show ...
2 votes
3 answers
3k views
Find <td> that "belongs" to <th> with BeautifulSoup
I want to find the value of a <td> that "belongs" to a <th>? I can search for the text in the <th> tag and find it, but I do not know the value and there is no class to ...
1 vote
3 answers
1k views
Beautiful Soup and scraping wikipedia entries:
Beginner to BeautifulSoup, I am trying to extract the Company Name, Rank, and Revenue from this wikipedia link. https://en.m.wikipedia.org/wiki/List_of_largest_Internet_companies The code I've used so ...
2 votes
3 answers
848 views
lxml returned me a list but it's empty
I was trying to make a list of all the top 1000 instagramer's acount from this website:'https://hypeauditor.com/top-instagram/'. The list that returns from lxml is empty for both lxml.html and lxml....
0 votes
2 answers
2k views
How can I convert the beautiful soup text to JSON object?
What I'm trying to do is to convert the scraped data I get from the URL to JSON object. import bs4 as bs from urllib.request import Request, urlopen import json req = Request('https://www....
1 vote
2 answers
2k views
get data from MarketWatch
Using BeautifulSoup I am trying to scrape MarketWatch from bs4 import BeautifulSoup import requests import pandas url = "https://www.marketwatch.com/investing/stock/khc/profile" # Make a GET request ...
0 votes
2 answers
2k views
Python: find the 2nd td child from all tr children
My goal is grab all of the court case numbers and put them into an Excel folder. The cases are in the 2nd column My code: courtCases = driver.find_elements_by_css_selector('body > table:nth-...
0 votes
2 answers
2k views
Parsing multiple tables with BeautifulSoup
I'm having problems parsing table data with BeautifulSoup, though I've tried many solutions found here, here, and here. I hate to re-ask but maybe my issue is unique and that is why the above ...
1 vote
1 answer
2k views
Web Scraping Table to JSON
I have a task that I think my python skills far slightly short at. I have a oil production data scraped from an open data website and looking to turn this into a json format. At present with some ...
0 votes
1 answer
2k views
Scraping a Government Website in Python with Beautiful Soup
I am trying to scrape the New Hampshire Secretary of State's website on registered voters. So far I have been able to get the text of the website in Beautiful soup with the following code: import ...
1 vote
3 answers
1k views
Extract user input to python from a table created in browser?
I want to create a table in a browser that's been created with python. That part can be done by using DataTable of the bokeh library. The problem is that I want to extract data from the table when a ...
1 vote
1 answer
756 views
Create a Dataframe from HTML
I am trying to read a table from a web-page. Generally, my company has strict authentication policies restricting us in the way we can scrape the data. But the following code is how I am trying to use ...
-1 votes
1 answer
1k views
How can I extract data from a web page and turn it into proper Pandas dataframe? [closed]
For example, here is an address: https://pesdb.net/pes2021/?id=44379 There seems to be no api call (I am pretty new to this but I checked XHR in network monitor and there are no relevant json calls).