2

I have below JSON output written into "out" file and the content is as below.

$ cat out

{ "columns": [ "Tests", "Errors", "Mean Test Time (ms)", "Test Time Standard Deviation (ms)", "TPS", "Peak TPS" ], "status": { "description": "Collection stopped", "state": "Stopped" }, "tests": [ { "description": "Cheetah client test", "statistics": [ 0, 0, "NaN", 0.0, 0.0, 0.0 ], "test": 1 }, { "description": "Reads 95%", "statistics": [ 304000, 0, 8.7931875, 7.948696618436826, 6907.677974959667, 13594.0 ], "test": 101 }, { "description": "Writes 5%", "statistics": [ 16000, 0, 9.963375, 9.92775949594747, 363.5619986820878, 1638.0 ], "test": 102 } ], "totals": [ 320000, 0, 8.851696875, 8.063234652303947, 7271.239973641756, 14259.0 ] }

I need stats pertaining to the block of description "reads 95%" arrange in below format and assign to a string variable using a BASH script. 

var=304000,0,8.7931875,7.948696618436826,6907.677974959667,13594.0

Your help is very much appreciated.

Improve this question

4 Answers 4

Reset to default
1

You need to be using a dedicated command line JSON parser for this e.g. underscore:

With underscore installed you can do:

cat data.json | underscore select '.description, .statistics first-child'| tr -d '[]'

EDIT: Sed only solution (use it with caution):

sed -rn '/"description": *"Reads 95%",/,/],/{/statistics|description/!{1h; 1!H;};/],/{x;s/ *\n *|^ *|],//gp;};}' out # gives 304000,0,8.7931875,7.948696618436826,6907.677974959667,13594.0
Improve this answer
Sign up to request clarification or add additional context in comments.

12 Comments

Thanks for your support. BTW i'm ok to arrange the output as below using bash. 304000,0,8.7931875,7.948696618436826,6907.677974959667,13594.0
Using underscore you can simple do: cat data.json | underscore extract 'statistics' for that
It can be done using pure bash tools also but let me tell you that its going to break it sooner than later.
To be more precise, I got this block after manipulating below two steps. step 1: curl -s -X GET localhost:6373/recording/data | python -mjson.tool > out step 2: sed -nr 'H;/Reads 95%/,/\}/{s/(\})/\1/;T;x;p};/\{/{x;s/.*\n.*//;x;H}' out Note: out consist many blocks like the block I have presented in the quesition Your solution sounds for a json file, to my understanding. appreciate your guidance and support
Thanks for your valuable support. I got the intended result from below. sed -nr "H;/$name/,/\}/{s/(\})/\1/;T;x;p};/\{/{x;s/.*\n.*//;x;H}" out | sed -e '1,/statistics/d' | sed -e '/test/,$d' | sed '$d' | sed '/^$/d' | sed -e 's/^[ \t]*//' | tr "\\n" " " | sed -e 's/[\t ]//g;/^$/d'
|
0

My "pure shell" solution is below

cat data.json | sed 's/^[ \t]*//;s/[ \t]*$//' | awk -F ': *' 'BEGIN { RS=",\n\"|\n}," } { gsub(/[\n\]\[\}]/,"",$2); if ($2) { printf("%s,", $2); } }'
Improve this answer

2 Comments

Thanks a lot for your support. I have tried your solution and it gives all in a single row and comma delimitted. However this required further manipulation to extract the intended portion out of it
Useless use of cat spotted!
0

The out file contains Grinder (an open source load testing tool) test data parsed to JSON

curl -s -X GET http://<localhost>:6373/recording/data' | python -mjson.tool > out ~$ cat out { "columns": [ "Tests", "Errors", "Mean Test Time (ms)", "Test Time Standard Deviation (ms)", "TPS", "Peak TPS" ], "status": { "description": "Collection stopped", "state": "Stopped" }, "tests": [ { "description": "Cheetah client test", "statistics": [ 0, 0, "NaN", 0.0, 0.0, 0.0 ], "test": 1 }, { "description": "Reads 95%", "statistics": [ 304000, 0, 8.7931875, 7.948696618436826, 6907.677974959667, 13594.0 ], "test": 101 }, { "description": "Writes 5%", "statistics": [ 16000, 0, 9.963375, 9.92775949594747, 363.5619986820878, 1638.0 ], "test": 102 } ], "totals": [ 320000, 0, 8.851696875, 8.063234652303947, 7271.239973641756, 14259.0 ] }

Sed command:

sed -nr "H;/$name/,/\}/{s/(\})/\1/;T;x;p};/\{/{x;s/.*\n.*//;x;H}" out | sed -e '1,/statistics/d' | sed -e '/test/,$d' | sed '$d' | sed '/^$/d' | tr "\\n" " " | sed -e 's/[\t ]//g;/^$/d'

gives below output - test statistics for the test "Reads 95%"

304000,0,8.7931875,7.948696618436826,6907.677974959667,13594.0

Next sed command:

sed -nr "H;/"Reads 95%"/,/\}/{s/(\})/\1/;T;x;p};/\{/{x;s/.*\n.*//;x;H}" out gives { "description": "Reads 95%", "statistics": [ 304000, 0, 8.7931875, 7.948696618436826, 6907.677974959667, 13594.0 ], "test": 101 },

  1. pipe result 1 with | sed -e '1,/statistics/d' gives all lines after the line contains "statistics"

     304000, 0, 8.7931875, 7.948696618436826, 6907.677974959667, 13594.0 ], "test": 101 },

  2. pipe result 2 with | sed -e '/test/,$d' gives lines above the line consists "test"

     304000, 0, 8.7931875, 7.948696618436826, 6907.677974959667, 13594.0 ],

  3. pipe result 3 with | sed '$d' removes the last line of output 3 304000, 0, 8.7931875, 7.948696618436826, 6907.677974959667, 13594.0

  4. pipe result 4 with | tr "\n" " " | sed -e 's/[\t ]//g;/^$/d' gives

    304000,0,8.7931875,7.948696618436826,6907.677974959667,13594.0

tr "\n" " " replace CRs (or \n) with a space and sed -e 's/[\t ]//g;/^$/d' removes all spaces and tabs

Note: I'm not an expert in this subject area, thus this may not be the most efficient solution

Improve this answer

Comments

0

Try sql4json (http://github.com/bheni/sql4json)

installation:

sudo pip install sql4json

command

cat out|sql4json --csv 'SELECT statistics FROM tests WHERE description=="Reads 95%"'

output to stdout:

304000,0,8.7931875,7.94869661844,6907.67797496,13594.0

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.