How to select unique array values?

Question

I have an array and I want to get the unique 2nd members

bash doesn't really have 2-dimensional array so I've defined it this way, using :: as the separator of the two elements:

ruby_versions=(
'company-contacts::1.7.4'
'activerecord-boolean-converter::1.7.4'
'zipcar-rails-core::1.7.4'
'async-tasks::1.7.13'
'zc-pooling-client::2.1.1'
'reservations-api::1.7.4'
'zipcar-auth-gem::1.7.4'
'members-api::1.7.4'
'authentication-service::1.7.4'
'pooling-api::2.1.1'
)

I can iterate successfully thru the 2nd elements of the array with:

rvm list > $TOP_DIR/local_ruby_versions.txt

for repo in "${ruby_versions[@]}"
do
  if grep -q "${repo##*::}" $TOP_DIR/local_ruby_versions.txt
    then
    echo "ruby version ${repo##*::} confirmed as present on this machine"
  else
    rvm list
    echo "*** EXITING SMOKE TEST *** - not all required ruby versions are present in RVM"
    echo "Please install RVM ruby version: ${repo##*::} and then re-run this program"
    exit 0
  fi
done
echo "A

The only downside is that it repeats the action when the ruby version is the same (which is usually the case), so I get

ruby version 1.7.4 confirmed as present on this machine
ruby version 1.7.4 confirmed as present on this machine
ruby version 1.7.4 confirmed as present on this machine
ruby version 1.7.13 confirmed as present on this machine
ruby version 2.1.1 confirmed as present on this machine
ruby version 1.7.4 confirmed as present on this machine
ruby version 1.7.4 confirmed as present on this machine
ruby version 1.7.4 confirmed as present on this machine
ruby version 1.7.4 confirmed as present on this machine
ruby version 2.1.1 confirmed as present on this machine

when I have

ruby_versions=(
  'company-contacts::1.7.4'
  'activerecord-boolean-converter::1.7.4'
  'zipcar-rails-core::1.7.4'
  'async-tasks::1.7.13'
  'zc-pooling-client::2.1.1'
  'reservations-api::1.7.4'
  'zipcar-auth-gem::1.7.4'
  'members-api::1.7.4'
  'authentication-service::1.7.4'
  'pooling-api::2.1.1'

)

How can I make it so it only does the check for 1.7.4 and 2.1.1 once ?

i.e. How can I turn my array selection into (1.7.4 2.1.1)

The actual repo names can be ignored in this context.

Note that the `re` in `gr̲e̲p` stands for `r̲egular e̲xpression` (so `grep -q 1.7.4` would match on 127-4 for instance). Add `-F` for fixed string search, but note that `grep -Fq 1.7.4` would still match on 11.7.4 or 1.7.40. — Stéphane Chazelas, Nov 10 '14 at 20:26

glenn jackman · Accepted Answer · 2014-11-10T17:48:09.180

2

You could use an associative array:

declare -A versions
for value in "${ruby_versions[@]}"; do
    versions["${value##*::}"]=1
done
printf "%s\n" "${!versions[@]}"

1.7.4
1.7.13
2.1.1

Or with a pipeline:

mapfile -t versions < <(printf "%s\n" "${ruby_versions[@]}" | sed 's/.*:://' | sort -u)
printf "%s\n" "${versions[@]}"

1.7.13
1.7.4
2.1.1

edited Nov 10 '14 at 17:48

answered Nov 10 '14 at 16:35

glenn jackman

84,176
15
116
168

Looks good but I get an error - `./smoke_test_functions.sh: line 6: 1.7.4: syntax error: invalid arithmetic operator (error token is ".7.4")` for `versions["${value##*::}"]=1` – Michael Durrant Nov 10 '14 at 16:49
My mistake, I forgot to copy the variable declaration: that declares the variable as an associative array. Need bash >= 4 – glenn jackman Nov 10 '14 at 17:47
yeah let me check that. I think my brand new mac is still bash v3.x as nuts as that is (I've get a more recent version before for a different issue but not currently I think after rebuild couple of weeks ago). – Michael Durrant Nov 10 '14 at 22:53

Costas · Answer 2 · 2014-11-10T17:19:55.563

1

echo "${ruby_versions[@]}" | sed 's/\S\+:://g;s/\s\+/\n/g'| sort -u

outputs:

1.7.13
1.7.4
2.1.1

Or if you'd like bash builtins

unset u
for i in "${ruby_versions[@]}"
do
  if [[ ! $u =~ ${i##*::} ]]
  then
    u=${u:+$u\\n}${i##*::}
  fi
done
echo -e "$u"

edited Nov 10 '14 at 17:19

answered Nov 10 '14 at 17:01

Costas

14,806
20
36

How to select unique array values?

2 Answers2