2

I am fresh new to python so I tried with visual studio and windows 7 the following

import csv from bs4 import BeautifulSoup import requests contents = [] with open('websupplies.csv','r') as csvf: # Open file in read mode urls = csv.reader(csvf) for url in urls: contents.append(url) # Add each url to list contents for url in contents: # Parse through each url in the list. page = requests.get(url).content soup = BeautifulSoup(page, "html.parser") price = soup.find('span', attrs={'itemprop':'price'}) availability = soup.find('div', attrs={'class':'product-availability'})

but I get - No connection adapters were found for .. '['a url']'

why?

The structure of csv is like the following

https://www.websupplies.gr/epeksergastis-intel-core-i5-8400-9mb-2-80ghz-bx80684i58400 https://www.websupplies.gr/epeksergastis-intel-celeron-g3930-2mb-2-90ghz-bx80677g3930 https://www.websupplies.gr/epeksergastis-amd-a6-9500-bristol-ridge-dual-core-3-5ghz-socket-am4-65w-ad9500agabbox

they dont have at the end a semicolumn

Improve this question
3
  • Can you show the structure of your CSV? Commented Mar 12, 2019 at 14:24
  • 2
    Possible duplicate of Python Requests - No connection adapters Commented Mar 12, 2019 at 14:24
  • @Mad my urls they have http protocol .. Commented Mar 12, 2019 at 14:28

2 Answers 2

Reset to default
1

Your file is a flat list of URLs. It's not really a CSV.

The CSV reader reads each row into its own list. So the structure of the loaded data would be:

[ ["https://www.websupplies.gr/epeksergastis-intel-core-i5-8400-9mb-2-80ghz-bx80684i58400"], ["https://www.websupplies.gr/epeksergastis-intel-celeron-g3930-2mb-2-90ghz-bx80677g3930"], ["https://www.websupplies.gr/epeksergastis-amd-a6-9500-bristol-ridge-dual-core-3-5ghz-socket-am4-65w-ad9500agabbox"], ]

One way to fix this would be to use url[0] as the parameter to requests.get, but really the proper fix is not to use CSV at all. Since you have only one piece of data per line, you can just read the data directly and pass it to requests:

with open('websupplies.csv','r') as csvf: # Open file in read mode for line in csvf: contents.append(line.strip('\n')) # Add each url to list contents
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

as a txt file you mean?
1

In this question it says that requests needs the http scheme, maybe it's this the problem? You also have to remove the /n when you read the lines from the file

Improve this answer

2 Comments

in your answer has a question, it looks like a comment.
i know, but i don't have enough reputation, i would have written a comment. I didn't respond in fact