411

In my Unix shell script, when I execute a curl command, the result will be displayed as below which I am redirecting to file:

{"type":"Show","id":"123","title":"name","description":"Funny","channelTitle":"ifood.tv","lastUpdateTimestamp":"2014-04-20T20:34:59","numOfVideos":"15"} 

But, I want this output to put in the readable JSON format like below in the file:

{"type":"Show", "id":"123", "title":"name", "description":"Funny", "channelTitle":"ifood.tv", "lastUpdateTimestamp":"2014-04-20T20:34:59", "numOfVideos":"15"} 

How do I format the output this way?

1
  • Pay attention to the answer below about ijq. This is the only solution that allows you to display json along with headers (key -i). Very useful. Commented May 15, 2024 at 19:18

15 Answers 15

803

A few solutions to choose from:

json json is a fast CLI tool for working with JSON. It is a single-file node.js script with no external deps (other than node.js itself).

$ echo '{"type":"Bar","id":"1","title":"Foo"}' | json { "type": "Bar", "id": "1", "title": "Foo" } 

Require:

# npm install -g json 

json_pp: command utility available in Linux systems for JSON decoding/encoding

echo '{"type":"Bar","id":"1","title":"Foo"}' | json_pp -json_opt pretty,canonical { "id" : "1", "title" : "Foo", "type" : "Bar" } 

You may want to keep the -json_opt pretty,canonical argument for predictable ordering.


: lightweight and flexible command-line JSON processor. It is written in portable C, and it has zero runtime dependencies.

echo '{"type":"Bar","id":"1","title":"Foo"}' | jq '.' { "type": "Bar", "id": "1", "title": "Foo" } 

The simplest jq program is the expression ., which takes the input and produces it unchanged as output.

For additional jq options check the manual


python yq yq: Command-line YAML/XML/TOML processor - jq wrapper for YAML, XML, TOML documents

$ echo '{"type":"Bar","id":"1","title":"Foo"}' | yq { "type": "Bar", "id": "1", "title": "Foo" } 

The version go yq doesn't work here


With xidel Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

$ echo '{"type":"Bar","id":"1","title":"Foo"}' | xidel -e '$json' { "type": "Bar", "id": "1", "title": "Foo" } 

with :

echo '{"type":"Bar","id":"1","title":"Foo"}' | python -m json.tool { "id": "1", "title": "Foo", "type": "Bar" } 

with and :

echo '{"type":"Bar","id":"1","title":"Foo"}' | node -p "JSON.stringify( JSON.parse(require('fs').readFileSync(0) ), 0, 1 )" { "type": "Bar", "id": "1", "title": "Foo" } 
Sign up to request clarification or add additional context in comments.

11 Comments

is this the way ?curl -i -vs -X POST -H "$SESSION_TOKEN" -H "$AUTH_TOKEN" -H "Accept:$ACCEPT_HEADER" -H "Content-Type:text/plain" "http://$BASE_URI/api" | json_pp >> jsonoutput.json
Absolutely. Have you tested ?
json_pp seems to format data in reverse order than it's expected. Is there a way to un-reverse data?
json_pp is a Perl command utility which converts between some input and output formats (one of them JSON). The default input format is json and the default output format is json with pretty option.
Pipe to Python for the win !
|
104

I am guessing that you want to prettify the JSON output. That could be achieved using python:

curl http://localhost:8880/test.json | python -m json.tool > out.json 

6 Comments

Thanks for answering. But i am not using Python, im using shell. can u guide me on shell
@Jams that executes python from the shell giving your a pretty printed output.
Great solution as python comes preinstalled.
That python script gives a "broken pipe" error when I add a " | head" to it.
@CrouchingKitten I don't see that problem myself (maybe different versions), but that may be due to the head command cutting off the output being piped to it before reaching the end, thus "breaking" the pipe. If you allow head to consume the entire data stream from the pipe, I'd expect the error to not show up.
|
66

This is to add to of Gilles' Answer. There are many ways to get this done but personally I prefer something lightweight, easy to remember and universally available (e.g. come with standard LTS installations of your preferred Linux flavor or easy to install) on common *nix systems.

Here are the options in their preferred order:

Python Json.tool module

echo '{"foo": "lorem", "bar": "ipsum"}' | python -mjson.tool 

pros: almost available everywhere; cons: no color coding


jq (may require one time installation)

echo '{"foo": "lorem", "bar": "ipsum"}' | jq 

cons: needs to install jq; pros: color coding and versatile


json_pp (available in Ubuntu 16.04 LTS)

echo '{"foo": "lorem", "bar": "ipsum"}' | json_pp 

For Ruby users

gem install jsonpretty echo '{"foo": "lorem", "bar": "ipsum"}' | jsonpretty 

1 Comment

Thanks @zhenhua! I used cat myFile.json | python -mjson.tool >> myFilePretty.json then opened in my editor (vim, don't hate) and got colors.
48

You can use the json node module:

npm i -g json

then simply append | json after curl. curl http://localhost:8880/test.json | json

3 Comments

I would remove the comment and part about sudo if you don't recommend it haha. I agree it's not necessary. Thanks for the answer
If you don't want to install the json module, you could use it directly with npx: curl http://localhost:8880/test.json | npx json
this should be the accepted response IMO, this is the easiest way to go. no options required, outputs exactly what you'd expect
48

You can install jq and make the query like below:

curl http://localhost:8080/observations/station/221 | jq 

enter image description here

6 Comments

why do I get command not found when I use it like this but not when I just run jq without anything else?
@AhmedHasn. What OS do you use? Anyways, irrespective of that, you will need to install the jq and it should work.
RHEL7 but as I said jq is installed and it works if I don't use it in a pipe
This is the simplest, to-the-point answer. Install jq tool in Linux and jq is used in many cases.
works on windows too
|
29
python -m json.tool Curl http://127.0.0.1:5000/people/api.json | python -m json.tool 

can also help.

Comments

20

I found json_reformat to be very handy. So I just did the following:

curl http://127.0.0.1:5000/people/api.json | json_reformat 

that's it!

2 Comments

If using Ubuntu you can find json_reformat in the yajl-tools package.
In fedora 25 it works fine. Thanks.
17

Motivation: You want to print prettify JSON response after curl command request.

Solution: json_pp - commandline tool that converts between some input and output formats (one of them is JSON). This program was copied from json_xs and modified. The default input format is json and the default output format is json with pretty option.

Synposis: json_pp [-v] [-f from_format] [-t to_format] [-json_opt options_to_json1[,options_to_json2[,...]]]

Formula: <someCommand> | json_pp

Example:

Request

curl -X https://jsonplaceholder.typicode.com/todos/1 | json_pp 

Response

{ "completed" : false, "id" : 1, "title" : "delectus aut autem", "userId" : 1 } 

Comments

7

Check out curljson

$ pip install curljson $ curljson -i <the-json-api-url> 

Comments

6

If you use mac's terminal, jq is built-in and you don't need to install anything. Just append | jq to your CURL command and you will get the formatted json. An example: When you execute this:

curl http://time.jsontest.com/ | jq 

you will get this:

 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 100 100 100 0 0 388 0 --:--:-- --:--:-- --:--:-- 389 { "date": "12-28-2023", "milliseconds_since_epoch": 1703738250474, "time": "04:37:30 AM" } 

1 Comment

Please don’t post duplicate answers.
5

A lot more features (slice, filter and map and transform structured ) apart from formatting.

https://stedolan.github.io/jq/

curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.[0]' 
{ "sha": "d25341478381063d1c76e81b3a52e0592a7c997f", "commit": { "author": { "name": "Stephen Dolan", "email": "[email protected]", "date": "2013-06-22T16:30:59Z" }, "committer": { "name": "Stephen Dolan", "email": "[email protected]", "date": "2013-06-22T16:30:59Z" }, "message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161", "tree": { "sha": "6ab697a8dfb5a96e124666bf6d6213822599fb40", "url": "https://api.github.com/repos/stedolan/jq/git/trees/6ab697a8dfb5a96e124666bf6d6213822599fb40" }, "url": "https://api.github.com/repos/stedolan/jq/git/commits/d25341478381063d1c76e81b3a52e0592a7c997f", "comment_count": 0 }, "url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f", "html_url": "https://github.com/stedolan/jq/commit/d25341478381063d1c76e81b3a52e0592a7c997f", "comments_url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f/comments", "author": { "login": "stedolan", "id": 79765, "avatar_url": "https://avatars.githubusercontent.com/u/79765?v=3", "gravatar_id": "", "url": "https://api.github.com/users/stedolan", "html_url": "https://github.com/stedolan", "followers_url": "https://api.github.com/users/stedolan/followers", "following_url": "https://api.github.com/users/stedolan/following{/other_user}", "gists_url": "https://api.github.com/users/stedolan/gists{/gist_id}", "starred_url": "https://api.github.com/users/stedolan/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/stedolan/subscriptions", "organizations_url": "https://api.github.com/users/stedolan/orgs", "repos_url": "https://api.github.com/users/stedolan/repos", "events_url": "https://api.github.com/users/stedolan/events{/privacy}", "received_events_url": "https://api.github.com/users/stedolan/received_events", "type": "User", "site_admin": false }, "committer": { "login": "stedolan", "id": 79765, "avatar_url": "https://avatars.githubusercontent.com/u/79765?v=3", "gravatar_id": "", "url": "https://api.github.com/users/stedolan", "html_url": "https://github.com/stedolan", "followers_url": "https://api.github.com/users/stedolan/followers", "following_url": "https://api.github.com/users/stedolan/following{/other_user}", "gists_url": "https://api.github.com/users/stedolan/gists{/gist_id}", "starred_url": "https://api.github.com/users/stedolan/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/stedolan/subscriptions", "organizations_url": "https://api.github.com/users/stedolan/orgs", "repos_url": "https://api.github.com/users/stedolan/repos", "events_url": "https://api.github.com/users/stedolan/events{/privacy}", "received_events_url": "https://api.github.com/users/stedolan/received_events", "type": "User", "site_admin": false }, "parents": [ { "sha": "54b9c9bdb225af5d886466d72f47eafc51acb4f7", "url": "https://api.github.com/repos/stedolan/jq/commits/54b9c9bdb225af5d886466d72f47eafc51acb4f7", "html_url": "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7" }, { "sha": "8b1b503609c161fea4b003a7179b3fbb2dd4345a", "url": "https://api.github.com/repos/stedolan/jq/commits/8b1b503609c161fea4b003a7179b3fbb2dd4345a", "html_url": "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a" } ] } 

Comments

3

In real cases sometimes you need use -i option in curl to get headers

$ curl https://petstore.swagger.io/v2/pet/1 -i HTTP/2 200 date: Wed, 26 Jul 2023 16:46:39 GMT content-type: application/json access-control-allow-origin: * access-control-allow-methods: GET, POST, DELETE, PUT access-control-allow-headers: Content-Type, api_key, Authorization server: Jetty(9.2.9.v20150224) {"id":1,"category":{"id":0,"name":"zbyszek"},"name":"pies","photoUrls":["string"],"tags":[{"id":0,"name":"pies"}],"status":"available"} 

in that case, pure jq/json_pp/json_reformat will fail:

curl https://petstore.swagger.io/v2/pet/1 -i | jq % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 138 0 138 0 0 108 0 --:--:-- 0:00:01 --:--:-- 109 parse error: Invalid numeric literal at line 1, column 7 

I offer to use ijq https://github.com/1frag/ijq that ignores non-json lines

curl https://petstore.swagger.io/v2/pet/1 -i | ijq % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 51 0 51 0 0 41 0 --:--:-- 0:00:01 --:--:-- 41 HTTP/2 404 date: Wed, 26 Jul 2023 16:51:58 GMT content-type: application/json access-control-allow-origin: * access-control-allow-methods: GET, POST, DELETE, PUT access-control-allow-headers: Content-Type, api_key, Authorization server: Jetty(9.2.9.v20150224) { "code": 1, "message": "Pet not found", "type": "error" } 

1 Comment

Realy the best answer among other. Thanks! I use your tool in this way: curl -i ... | ijq --no-color | nvim Very useful.
2

With the JSON-parser :

$ curl [...] | xidel -se '$json' 

Or directly with Xidel:

$ xidel -s "<url>" -e '$json' 

Comments

1

Nothing needs to be installed if you have node.js installed. With node.js you can use JSON.stringify function, just add:

| node <<< "console.log(JSON.stringify($(cat), null, 4));"

For example:

curl http://localhost:8080/test.json | node <<< "console.log(JSON.stringify($(cat), null, 4));"

OR

echo '{"foo": "lorem", "bar": "ipsum"}'| node <<< "console.log(JSON.stringify($(cat), null, 4));"

4 = count of spaces for indentation.

Comments

0

Install

$ sudo apt install python3-pip $ sudo pip install curljson 

then using, example:

$ curljson -i localhost:9080/actuator/health ; echo HTTP/1.1 200 Content-Type: application/vnd.spring-boot.actuator.v3+json Transfer-Encoding: chunked Date: Fri, 15 Sep 2023 10:28:55 GMT { "status": "UP" } 

Tested with Ubuntu 22.04 LTS .

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.