Scraping Data from Table Python

Question

I'm trying to scrape data from a website's table using Python.

from bs4 import BeautifulSoup from mechanize import Browser BASE_URL = "http://www.ggp.com/properties/mall-directory" def main(): mech = Browser() url = "http://www.ggp.com/properties/mall-directory" page1 = mech.open(url) html1 = page1.read() soup1 = BeautifulSoup(html1) extract(soup1, 2007) def extract(soup,year): table = soup.find("table") for row in table.findAll('option'): print row main()

Row prints out:

<option value="184">Yakima, WA</option> <option value="896">Yankton, SD</option> <option value="851">Yazoo City, MS</option> <option value="113">York-Hanover, PA</option> <option value="87">Youngstown-Warren, OH-PA</option> <option value="235">Yuba City, CA</option> <option value="205">Yuma, AZ</option> <option value="424">Zanesville, OH</option>

But what I need is

Yakima, WA Yankton, SD Yazoo City, MS York-Hanover, PA etc...

I've tried row.findAll('option value') but this doesn't work...

gist.github.com/cobralibre/120191

Robert Harvey
– Robert Harvey

2014-06-09 16:14:12 +00:00
Commented Jun 9, 2014 at 16:14 — Robert Harvey
– Robert Harvey, Commented Jun 9, 2014 at 16:14
use print row.get_text()

Vipul
– Vipul

2014-06-09 16:19:01 +00:00
Commented Jun 9, 2014 at 16:19 — Vipul
– Vipul, Commented Jun 9, 2014 at 16:19

Mark · Accepted Answer · 2014-06-09 16:16:57Z

1

I think this should work

print row.getText()

answered Jun 9, 2014 at 16:16

Mark

20.2k9 gold badges113 silver badges137 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Scraping Data from Table Python

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related