How to download file from web using request module?

Question

I am using python to download files from webpage, and after I submit the request, I get the headers like below:

>>> r.headers {'Content-Disposition': 'attachment; filename="report20160619-013623.csv";', 'Content-Transfer-Encoding': 'binary', 'Expires': '0', 'Keep-Alive': 'timeout=5, max=100', 'Server': 'Apache', 'Transfer-Encoding': 'chunked', 'Connection': 'Keep-Alive', 'Pragma': 'public', 'Cache-Control': 'must-revalidate, post-check=0, pre-check=0, private', 'Date': 'Sun, 19 Jun 2016 06:35:18 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Type': 'application/octet-stream'}

My question is how could I download the report20160619-013623.csv using the request module.

I search around the web, there are solutions that download the file given a specific URL to that file, however, in my case, the file is generated on web by clicking something like "Download as excel". How do I know where the file is?

climmunk · Accepted Answer · 2016-06-20 04:30:26Z

4

You will read the response and then write it out as a file to a location you specify.

csv = r.text outfile = open('/path/to/my.csv', 'w') outfile.write(csv) outfile.close()

edited Jun 20, 2016 at 4:30

answered Jun 20, 2016 at 3:02

climmunk

1,2211 gold badge15 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

3kt Over a year ago

OP specified that he would like to use requests module...

climmunk Over a year ago

@GeorgeWang In my original response I used "example.com/downloads.csv" to just represent the path to the file you were downloading, but you already have a response containing that file.

climmunk Over a year ago

@3kt Good point - I've rewritten my response to use the requests module

George Wang Over a year ago

@climmunk, thank you very much for your help. I think I get a feeling how it works. But to confirm, report20160619-013623.csv here is useless, right? it can be replaced by customised name?

climmunk Over a year ago

That's the default file name, and you can use it or change it.

Collectives™ on Stack Overflow

How to download file from web using request module?

1 Answer 1

5 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Related