Filter YAML file content using sed/awk

Question

I have a text file with the following content in it.

$ cat hosts.yml
[prod_env]
foo.example.com
bar.example.com
[stage_env]
foo_stage.example.com
bar_stage.example.com
[dev_env]
foo_dev1.example.com
dev2_bar.example.com

I would like to filter hosts listed under [prod_env] file in bash. I tried with sed, awk and grep with my limited knowledge, but I am not sure how to filter them.

Example: I would like to print hosts under prod_env and dev_env. so the final output would be:

foo.example.com 
bar.example.com 
foo_dev1.example.com 
dev2_bar.example.com

Answer 1

An sed solution:

sed -nEe '/\[(prod|dev)_env]/!d;N;:loop' -e 's/.*\n//;${p;d;};N;P;/\n\[/D;bloop' hosts.yml

• /\[(prod|dev)_env]/!d drops all lines until [prod_env] or [dev_env] ist found
• N;:loop adds the next line and starts a loop
• inside the loop we remove the first of the two lines with s/.*\n//, because it is either the [...env] line or we already printed it in the last loop cycle
• ${p;d;} prints the remaining lines if we reached the last line while printing
• N;P adds the next line and prints the current one
• /\n\[/D looks if the next line starts with a [. In this case the first line in the buffer (already printed) can be discarded and we start over with that [ line
• bloop otherwise loop

Instead of adding the next line to the buffer, printing and removing the old one, you can go line by line, but this would require another loop, because you can't start over with D

Answer 2

You could do (on a mac):

tr "\n" "\t" < hosts.yml | sed $'s%\t\\[%\\\n\[%g' | grep 'prod\|dev' | sed $'s%.*\]\\\t%%g' | tr "\t" "\n"

1. tr will get everything on one line
2. The first sed will break line before each section
3. grep will cut out the lines (sections) you want
4. Second sed will remove the [] part.
5. Last tr will break line so you get one hostname per line.

Answer 3

awk 'BEGIN{RS="[";FS="]\n"} 
     $1 ~/(prod|dev)_env/ {print $2}' ex1

Answer 4

You can do this using GNU sed as shown:

$ sed -nEe '
   /\[/!{H;$!d;}
   x;1!s/^\[(prod|dev)_env]\n//p
' hosts.yml