Replacing only specific variables with envsubst

Question

I'm trying to perform environment variable replacement through envsubst, but I want to only replace specific variables.

From the docs I should be able to tell envsubst to only replace certain variables but I'm failing to be able to do that.

For example, if I have a file containing:

VAR_1=${VAR_1}
VAR_2=${VAR_2}

how should I execute envsubst so that it only replaces the reference to ${VAR_1}?

don_crissti · Answer 1 · 2022-12-01T22:23:57.923

284

Per the man page:

envsubst [OPTION] [SHELL-FORMAT]

If a SHELL-FORMAT is given, only those environment variables that are referenced in SHELL-FORMAT are substituted; otherwise all environment variables references occurring in standard input are substituted.

Where SHELL-FORMAT strings are "strings with references to shell variables in the form $variable or ${variable}[...] The variable names must consist solely of alphanumeric or underscore ASCII characters, not start with a digit and be nonempty; otherwise such a variable reference is ignored.".

Note that the format ${VAR:-default} is not supported. I mentioned HERE some alternatives that support it along with other features.

Anyway, back to gettext envsubst:
So, one has to pass the respective variables names to envsubst in a shell format string (obviously, they need to be escaped/quoted so as to be passed literally to envsubst). Example:

input file e.g. infile:

VAR1=${VAR1}
VAR2=${VAR2}
VAR3=${VAR3}

and some values like

export  VAR1="one" VAR2="two" VAR3="three"

then running

envsubst '${VAR1} ${VAR3}' <infile

or

envsubst '${VAR1},${VAR3}' <infile

or

envsubst '${VAR1}
${VAR3}' <infile

outputs

VAR1=one
VAR2=${VAR2}
VAR3=three

Or, if you prefer backslash:

envsubst \$VAR1,\$VAR2 <infile

produces

VAR1=one
VAR2=two
VAR3=${VAR3}

edited Dec 01 '22 at 22:23

answered Jul 07 '16 at 10:31

don_crissti

79,330
30
216
245

35

why is this not the accepted answer? – Qsiris Jan 11 '18 at 13:39
10

Yes this is the right answer! – Thomas Decaux Feb 26 '18 at 09:31
11

If you know a variable prefix to use to limit the variables that can be substituted: `envsubst "$(printf '${%s} ' ${!PREFIX*})" < infile` – Sam Liddicott Jan 22 '19 at 15:13
5

Or if you have a list of them in `SUBST_VARS` then: `envsubst "$(printf '${%s} ' $SUBST_VARS)" < infile` – Sam Liddicott Jan 22 '19 at 15:15
1

This answer ALSO substitutes an empty string for variables present that are not set. As opposed to ONLY replacing the specific variables. – Chris Stryczynski Mar 06 '20 at 12:45
Thank you very, very much for explaining the use of "SHELL-FORMAT". I didn't see that in any version of the envsubst manpage or in its `--help` output. – Mr. Lance E Sloan Jul 13 '20 at 18:45
1

If you want to replace all the *exported* items use: `envsubst "$(compgen -e | awk '$0="${"$0"}"')" < infile`. – Nishant Oct 17 '20 at 09:51
You can use the `-v` or `--variables` flag to output the variables given in SHELL-FORMAT (to verify that your syntax is correct): e.g. `envsubst -v '${VAR1} $VAR2 NOVAR'`. – Dario Seidl Nov 23 '20 at 14:10
1

Overall the is a great answer with great comments! And I like to be able to write human kindness into comments!!! – eventhorizon Dec 29 '20 at 12:03
On a Mac, I get `VAR1=one VAR2=two VAR3=three` – Matt Alexander Mar 01 '21 at 09:55
@SamLiddicott Great tips, to complete your suggestion, the SUBST_VARS must be defined as `SUBST_VARS='$MYVAR1 $MYVAR2 $ANOTHER_varX' – Polymerase Apr 03 '21 at 21:51
is it possible to set the variables and invoke envsubst in single command? – weima Aug 17 '21 at 10:37
For anyone having an issue using docker-compose, you will need to use two dollar `envsubst '$$DOCKER_NGINX_LOCAL_IP' < infile` – Snake_py Jul 06 '22 at 13:01

score 3 · Answer 2 · answered Feb 22 '21 at 13:25

3

Although related to docker, the utility envplate should do the job https://github.com/kreuzwerker/envplate

From the readme:

Trivial templating for configuration files using environment keys. References to such keys are declared in arbitrary config files either as:

${key} or

${key:-default value}

gnutext's envsubst only replaces ${key}; if missing is replaced by ''.

answered Feb 22 '21 at 13:25

lrkwz

149
1
7

Another `envsubst` implementation, this time in golang: https://github.com/drone/envsubst – Jarek Dec 22 '21 at 06:07

Rui F Ribeiro · Accepted Answer · 2019-05-30T14:18:40.340

1

Before calling envsubst you should use export using single quotes to get back VAR_1 modified. As in:

export VAR_1='somevalue'

For more details, please see:

How to substitute shell variables in complex text files

edited May 30 '19 at 14:18

answered Jul 07 '16 at 10:10

Rui F Ribeiro

55,929
26
146
227

52

Correct answer is below – Craig Mar 19 '18 at 17:40
1

@JoãoAngelo Would you clarify why this was marked has the correct one over the other answer? For me it seems more appropriate asking the original asker. – Rui F Ribeiro Jan 22 '20 at 11:20
@RuiFRibeiro your answer will replace all variables (not only specific variables). all patterns like `$varname` will be evaluated (not only the variables you're targetting). – MhdSyrwan Feb 26 '20 at 12:31
check this case, https://serverfault.com/questions/577370/how-can-i-use-environment-variables-in-nginx-conf#comment952176_755541 – MhdSyrwan Feb 26 '20 at 12:32
6

@Craig There is no "below" on stack overflow - answers change order depending on various factors. Please link to the actual answer. – rjmunro May 25 '22 at 14:17

score 1 · Answer 4 · answered Jun 15 '22 at 17:50

Unfortunately 'export' is mandatory, otherwise it won't work. Here is an example.

First, the input file. Note that I only want to change NGINX_HOST and NGINX_PORT and don't want to touch other variables in the input file such as '$uri'.

$ egrep 'server_name|listen|try_files' nginx-default.conf.template 
  server_name ${NGINX_HOST};
  listen ${NGINX_PORT};
    try_files $uri $uri/ /index.php?$args;

Now the empty variables:

$ echo ${NGINX_HOST}

$ echo ${NGINX_PORT}

Set default values to these variables:

$ NGINX_HOST=${NGINX_HOST:-localhost}
$ NGINX_PORT=${NGINX_PORT:-80}
$ echo ${NGINX_HOST}
localhost
$ echo ${NGINX_PORT}
80

Try to use envsubst:

$ envsubst \
    '${NGINX_HOST} ${NGINX_PORT}' \
    < nginx-default.conf.template \
    | egrep 'server_name|listen|try_files' 
    
  server_name ;
  listen ;
  try_files $uri $uri/ /index.php?$args;

As you can see, it did not work. The variables must be 'export'-ed for this to work. Here is the second run:

$ export NGINX_HOST=${NGINX_HOST:-localhost}
$ export NGINX_PORT=${NGINX_PORT:-80}

$ envsubst   '${NGINX_HOST} ${NGINX_PORT}' <  nginx-default.conf.template | egrep 'server_name|listen|try_files' 

  server_name localhost;
  listen 80;
    try_files $uri $uri/ /index.php?$args;

As you can see, now it worked. Hope it helps others.

This is like a car user’s manual saying “This car must be used on a solid surface in a gravitational field.   It doesn’t work in water, or when in free-fall or in outer space.”   In other words, everybody knows this. — G-Man Says 'Reinstate Monica', Jun 16 '22 at 04:21
The question is rather: How to avoid replacing `$uri` in the file if there is an `uri` environment variable in the current environment. — Kusalananda, Jun 19 '22 at 06:02

Replacing only specific variables with envsubst

4 Answers4

Linked