430

I'm using the python requests module to send a RESTful GET to a server, for which I get a response in JSON. The JSON response is basically just a list of lists.

What's the best way to coerce the response to a native Python object so I can either iterate or print it out using pprint?

0

5 Answers 5

744

Since you're using requests, you should use the response's json method.

import requests response = requests.get(...) data = response.json() 

It autodetects which decoder to use.

Sign up to request clarification or add additional context in comments.

13 Comments

Just keep in mind that it have appeared somewhere in between v0.12 and v1.0 so that for example Ubuntu 12.04 deb-package for python-requests does not have this function yet (it is v0.8). You can pip install requests though instead of using deb package.
I am a little curious what is now data is it a list of list or a dictionary.
@Krishna_Oza data mirrors the structure of the JSON it's reading. For example, if the response is: [{"a": {...}}], data is a list, with list[0] == {'a': {...}}. If the response is {"a": "b", "c": "d"}, data is a dict. Does that answer your question?
@CGFoX what version are you running? I'm still seeing the API work the same way on the latest version: >>> import requests >>> r = requests.get('http://httpbin.org/get') >>> r.json <bound method Response.json of <Response [200]>> >>> r.json() {'args': {}, ...}
Upvote for this because I was usually using the json.loads(response.text) method until on some large jsons I found that using respons.json() was much faster than the other way.
|
453

You can use json.loads:

import json import requests response = requests.get(...) json_data = json.loads(response.text) 

This converts a given string into a dictionary which allows you to access your JSON data easily within your code.

Or you can use @Martijn's helpful suggestion, and the higher voted answer, response.json().

8 Comments

ok great, however each of the elements would still be unicode.
@felix001: yes, although you can convert any data using str(). On the other hand unicode data isn't bad to have around (in preparation for the future).
Much, much better to use response.json(), as it'll do a better job of figuring out the encoding used. (Disclaimer, I wrote some of that code).
@MartijnPieters: then how can I use requests json parser later on a memcached text of the response? i.e. having the output of response.text()?
@neurino you want the standard library json module. The requests.compat module is there to bridge different Python versions and requests.compat.json is the same thing as json on practically every system you’d care about.
|
41

You can use the json response as dictionary directly:

import requests res = requests.get('https://reqres.in/api/users?page=2') print(f'Total users: {res.json().get("total")}') 

or you can hold the json content as dictionary:

json_res = res.json() 

and from this json_res dictionary variable, you can extract any value of your choice

json_res.get('total') json_res["total"] 

Attentions Because this is a dictionary, you should keep your eye on the key spelling and the case, i.e. 'total' is not the same as 'Total'

3 Comments

json().get() ist the part I was looking for - for half an hour. Thank you!
@ESP32 The incoming response could be a list. It should be noted.
This answer was very helpful since it shows how to access the key directly and returns the value. Thank you
2

What's the best way to parse a JSON response from the requests library?

The top answers show seemingly two different ways to parse a json response into a Python object but they are essentially the same.

response.json() differs in two places:

  • it uses simplejson (which is the externally maintained development version of the json library included with Python) if it's installed in your environment, but uses the built-in json if not (source). I think there used to be a performance difference between json and simplejson in the past (when Python 2 was still widely used) but there's almost no difference between the libraries anymore.
  • if the response doesn't have an encoding (response.encoding is None), then it tries to guess it and try to decode using the guessed encoding (source).

So in 99.9999% of cases, response.json() and json.loads(response.text) will produce the same dictionary. It may differ if the source json has some weird encoding.

Comments

2

in case of not json

# pip install requests import requests r = requests.get(url) try: data = r.json() except requests.JSONDecodeError: content_type = r.headers.get('Content-Type') print(content_type) # r.encoding='utf-8' # data = r.text 

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.