0

I had the following piece of az cli output in plain text:

echo $raw_containers_string [ { "name": "123" }, { "name": "vbm-container" } ]

After some text refinement, I have a string returned containing this (zsh):

echo $raw_containers_string | grep name | cut -d ":" -f2 | tr '\n' " " "123" "vbm-container" %

(It also has a % symbol at the end, but that's expected)

I now need to create an array of these 2 strings (123 and vbm-container) to iterate through it.

  1. declare -a arr=($(echo $raw_containers_string | grep name | cut -d ":" -f2 | tr '\n' " "))
    • returns "123" "vb -co t i r"
  2. arr=($(echo $raw_containers_string | grep name | cut -d ":" -f2 | tr '\n' " "))
    • returns "123" "vb -co t i r"

These are indices of an array (if it matters):

➜ bash-az-list-blobs git:(master) ✗ echo $myvar[0] ➜ bash-az-list-blobs git:(master) ✗ echo $myvar[1] "123" "vb ➜ bash-az-list-blobs git:(master) ✗ echo $myvar[2] -co ➜ bash-az-list-blobs git:(master) ✗ echo $myvar[3] t ➜ bash-az-list-blobs git:(master) ✗ echo $myvar[4] i ➜ bash-az-list-blobs git:(master) ✗ echo $myvar[5] ➜ bash-az-list-blobs git:(master) ✗ echo $myvar[6] r"

Questions

  1. I want to understand why the behaviour between printing it to the terminal and assigning it to a variable is different.
  2. I would also like to know how I assign my refined to an array in zsh, so that echo arr returns a 2-elements iterable array.

UPD (08-16)

Using jq does filter JSON successfully

echo "$raw_containers_string" | jq -r '.[].name' 123 vbm-container

The problem is to pack these values into an array (this was exactly the purpose of this thread):

declare -a arr=$(echo "$raw_containers_string" | jq -r '.[].name') echo $arr 123 vbm-container echo $arr[0] 123 vbm-container[0] echo $arr[1] 123 vbm-container[1]

Why accessing the first element of my array just add index "[0]" to an output?

echo "$raw_containers_string" | jq -r '.[].name' → 123 vbm-container declare -a containers_array=($(echo "$raw_containers_string" | jq -r '.[].name|@sh' | tr '\n' " ")) → "123" eval "containers_array=($(echo "$raw_containers_string" | jq -r '.[].name|@sh'))" → 123 eval "containers_array=($(echo "$raw_containers_string" | jq '.[].name|@sh'))" → '123' echo $IFS →
Improve this question
1
  • 2
    Why is this tagged bash if you're using zsh? Commented Aug 13, 2022 at 5:18

2 Answers 2

Reset to default
2

You're counting on word-splitting of unquoted variable/command substitutions to separate the elements of the array, and have apparently also changed IFS. These are both bad ideas, and together they're a recipe for chaos. You're also using zsh for interactive testing, and it does word splitting very differently from bash, so you'll get different weird results from it.

First, track down what's changing IFS and fix it. You appear to have it set to something really weird, containing at least "m", "n", "a", and "e", so it's probably set by mistake.

If it actually does need to be changed, you need to keep it from staying changed. You can either make the change local to a specific command by making it a prefix of the command (e.g. IFS=',' read field1 field2) or by resetting it after using the changed value (e.g. saveIFS="$IFS"; IFS=","; doSomethingWithIFS; IFS="$saveIFS"), or just find a different way to solve whatever you changed IFS` for in the first place.

Second, get in the habit of double-quoting variable and command substitutions. There are some specific situations where you shouldn't double-quote them, and some situations where it's optional, but it's safest to just assume it should be double-quoted in most cases. shellcheck.net is good at pointing out quoting problems (and many other common mistakes).

Instead of using word-splitting on an unquoted substitution to populate the array, use readarray -t to read each line of input into a separate array element (or read -ra to read words into elements -- but fix IFS first, or it'll also split weirdly). Note that you can't put these in a pipeline, or they execute in a subshell and the value gets lost; instead, use process substitution: readarray -t < <(somecommand)

Third, that grep | cut thing is going to output something with extra spaces and quotes around the actual values. Anyway, when parsing JSON, it's better to use a tool like jq that actually speaks the language.

So, with all that fixed, try this:

readarray -t arr < <(echo "$raw_containers_string" | jq -r '.[].name')

Léa Gris pointed out that you can simplify this even further by passing the string to jq with --argjson instead of echo and a pipe:

readarray -t arr < <(jq -rn --argjson j "$raw_containers_string" '$j[].name')

If you have an older version of bash that doesn't support readarray (i.e. you're on macOS), you can fake it with read and some delimiter switching:

IFS=$'\n' read -d '' -a arr < <(jq -rn --argjson j "$raw_containers_string" '$j[].name')

Here, the -d '' part makes it treat the entire thing as one big line, and IFS=$'\n' tells it to break that "line" into fields on newline delimiters. Since the IFS assignment is a prefix to the read command, it only applies to that one command and won't cause trouble later. One warning, though: technically, the read command will exit with an error status (because it never saw an end-of-line delimiter), so if you use set -e or shopt -s errexit it'll make the script exit (add || true at the end to prevent this).

(And if you need to check the contents of an array -- or even a plain variable -- declare -p arr is more reliable than echo.)

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Save an echo and a sub-shell pipe: readarray -t arr < <(jq -rn --argjson j "$raw_containers_string" '$j[].name')
@LéaGris Nice. I've edited it in...
It is not a full replacement for piping-in the JSON as a stream. because of argument length limitations, the JSON argument might not fit. That said, you usually get the JSON as a result of some command and spending a sub-shell to collect the JSON string, then another sub-shell to stream it to jq is kind of wasteful. Stream pipe the output of the command providing the JSON directly to jq and let the system deal with cashing (what it does better than reinventing the cache within a shell.
readarray doesn't work on macOS. Alternative suggested here is not helpful either, it gives me bash: $raw_containers_string: ambiguous redirect
@feedthemachine I added a way to make it work with older versions of bash.
0

Using jq:

eval "names=($(echo "$raw_containers_string" | jq -r '.[].name|@sh'))"

Now you have a shell array names, where each name is an element. Piping to @sh means each value is quoted as a separate shell word, safe to eval.

Improve this answer

3 Comments

It doesn't work.echo $raw_containers_string - [ { "name": "123" }, { "name": "vbm-container" } ]. I then run your command and my names is this 123. I expect both items to be in it, not only first one
@feedthemachine Read arrays section of man bash. printf '%s\n' "${names[@]}" or echo "${names[0]}"; echo "${names[1]}". Both names are in the shell array.
Thanks, that was the easiest way to solve this problem.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.