requests - ChunkedEncodingError with requests.get, but not when using Postman

Question

I have a URL that can only be accessed from within our VPC.

If I enter the URL via Postman, I am able to see the content (which is a Pdf) and I'm able to save the output to a file perfectly fine.

However, when trying to automate this using python requests with

import requests r = requests.get(url, params=params)

I get an exception

ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

The other Stackoverflow questions haven't really helped much with this and this is consistently reproducible with requests

Andrey Semakin · Accepted Answer · 2024-02-08 18:13:35Z

I think the reason for this error is server is not correctly encoding the chunk. some of the chunk sizes are not integer b'' due to which you are getting chunk encoding error.

Try below code

import requests from requests.exceptions import ChunkedEncodingError response = requests.get(url, params=params, stream=True) try: for data in response.iter_content(chunk_size=1024): print(data) except ChunkedEncodingError as ex: print(f"Invalid chunk encoding {str(ex)}")

You may want to adjust the chunk_size. In fact I found chunk_size=64 worked in my case where 1024 or larger was encountering ChunkedEncodingError before transferred all. YMMV.

Collectives™ on Stack Overflow

requests - ChunkedEncodingError with requests.get, but not when using Postman

1 Answer 1

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Linked

Related