2

I'd often like to quickly take some data, and apply it to a template in bash.

e.g. imagine doing the following

  $ seq 1 2 | eztemplate 'this is a {1} test'
  this is a 1 test
  this is a 2 test

  $ eztemplate 'this is a {1} test' hi 'there now'
  this is a hi test
  this is a there now test

  $ eztemplate /tmp/template /tmp/alphabet  | head
  this is a a test
  of this system a
  this is a b test
  of this system b
  ...

I've wrote a very simple bash script that does this, but was considering allowing multiple parameters, like CSV style data, per line of data.

Does something better than my little script already exist, given the following?

  • I want it to require only basic unix posix tools, and commonly installed things like perl or awk, but requires no modules be installed additionally by perl, for example
  • It can accept multiple columns of data, per line of data in the data file.
  • Basically is one bash script that doesn't require installing anything else :D
  • A side intent is to allow others that aren't great at bash scripting to have a simple tool to process templates for repeating data

The data and template will vary greatly, but the first example I wanted to do it with was apply 4 ids to a JSON payload

template

{
  "tenantId": "{1}",
  "key": "some.key",
  "value": "some.value"
}

data

my/super/01cbf4e9779649038f0bd753747c8b26
my/test/01cbf4e9779649038f0bd753747c8b26
ez/test/01cbf4e9779649038f0bd753747c8b26
rad/data/df3a47fed39d453f836f9b2196332029

eztemplate

#!/usr/bin/env bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"; PATH="$DIR:$PATH"

function show_help()
{
  ME=$(basename "$0")
  IT=$(cat <<EOF

  replaces vars in a template, one for each line of stdin

  e.g.

  $ seq 1 2 | $ME 'this is a {1} test'
  this is a 1 test
  this is a 2 test

  $ $ME 'this is a {1} test' hi 'there now'
  this is a hi test
  this is a there now test

  $ $ME /tmp/template /tmp/alphabet
  this is a a test
  of this system a
  this is a b test
  of this system b
  ...

EOF
)
  echo "$IT"
  echo
  exit
}

if [ -z "$1" ]
then
  show_help
fi
if [ "$1" = "help" ] || [ "$1" = '?' ] || [ "$1" = "--help" ] || [ "$1" = "h" ]; then
  show_help
fi

function process_template(){
  DATA=$1
  VAR=$2

  if [ -f "$DATA" ]; then
    DATA=$(cat "$DATA")
  fi

  echo "$DATA" | sed "s#{1}#$VAR#g"
}


TEMPLATE=$1
if [ -t 0 ]
then
  if [ -f "$2" ]; then
    # allow first 2 parameters to be files, TEMPLATE and then DATA
    DATA_FILE=$2
    cat "$DATA_FILE" | while read line
    do
      process_template "$TEMPLATE" "$line"
    done 
  else
    shift;
    for line in "$@" 
    do
      process_template "$TEMPLATE" "$line"
    done
  fi
else
  # loop over lines from stdin
  while IFS= read -r line; do
    process_template "$TEMPLATE" "$line"
  done
fi
Brad Parks
  • 1,619
  • 3
  • 22
  • 38
  • Since you're using bash which is not POSIX I assume you are OK with bash too, right? Or does it need to be a POSIX shell? – terdon Mar 03 '23 at 17:59
  • Please read [why-is-using-a-shell-loop-to-process-text-considered-bad-practice](https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice), https://porkmail.org/era/unix/award, [correct-bash-and-shell-script-variable-capitalization](https://stackoverflow.com/questions/673055/correct-bash-and-shell-script-variable-capitalization) and I expect running your script through http://shellcheck.net would flag some issues. – Ed Morton Mar 03 '23 at 19:18
  • My impression is that the sample input you provided at the top of your question isn't the actual input format you want to be able to handle (e.g. it's not CSV) so please [edit] your question to include concise, testable sample input and expected output such as you want your script to be able to handle. – Ed Morton Mar 03 '23 at 19:26
  • hey... thanks for the feedback - I'm mostly wondering if there's some tool that does what I'm doing "better", and is preinstalled in linux (or mac), with a non bash script maybe? I don't tend to process tons of data, so I'm not too concerned about using a bash loop, but I get your point :D – Brad Parks Mar 03 '23 at 19:41
  • and the data I want to pass in will be totally arbitrary - most of the time a list of ids, but possibly at time a list of comma separated values, e.g. 54,john,doe. Most of the time they'll be in a file I download from splunk or Google Sheets, and I want to quickly tranform the data into json payloads or something. Internally in my company we have some tools that require individual json payloads, so one example would be applying the values to a json payload format. But honestly it could be anything, and I expect most of the time a single value will do, but sometimes I might use multiple :D – Brad Parks Mar 03 '23 at 19:42
  • 1
    I wrote a simple template engine to use csv data in Powershell for windows. You are welcome to any concepts you can use in creating a tool for Unix and bash. Check it out at github.com/dcressey/Expand-Csv – Walter Mitty May 24 '23 at 13:28
  • Thanks for the ideas @WalterMitty ! My goals are to keep things as "little install required" as possible, as I support devs across the board, some that are new to the game, and some total pros, but everyone likes the "zero install" concept. We do use Docker for lots of things, so I'm considering expanding my horizons to auto load many things transparently through the container. Definitely food for thought! – Brad Parks May 24 '23 at 23:01
  • Yeah, that's why I didn't post it as an answer. – Walter Mitty May 25 '23 at 06:27

1 Answers1

3

For the example you cite, the most natural solution appears to be

$ seq 1 2 | xargs -n1 printf 'this is a %s test\n'

Clearly awk is equal to the task, as well, if desired.

terdon
  • 234,489
  • 66
  • 447
  • 667
J_H
  • 636
  • 4
  • 8