Using sed command arguments to be compatible with both GNU & BSD Unix (in-place editing)

Question

I have a shell script that I currently use for some build related stuff for a mobile application.

Due to the the subtle differences between BSD & GNU one of build scripts originally written on a Mac (BSD)

environment=$1

if [[ -z $environment ]]; then 
  environment="beta"
fi
if ! [[ $environment =~ (live|beta) ]]; then
  echo "Invalid environment: $environment"
  exit 1
fi

mobile_app_api_url="https://api"$environment".mysite.com"

cp app/index.html.mob MobileApp/www/index.html

sed -i'' "s#MOBILE_APP_API_URL#\"$mobile_app_api_url\"#g" MobileApp/www/index.html

The sed command has been written on BSD (Mac) but as builds may take place on both Mac or Ubuntu (GNU) I need to modify this to work with on both flavours, what is the best approach for this?

Use [`ed`, it can edit the files in-place](http://unix.stackexchange.com/q/316349) and it's fully portable. — don_crissti, Mar 03 '17 at 14:12
Use `perl -pi -e '...'`. Both BSD and GNU sed derived their `-i` from `perl`'s one. `perl` will be available on most GNU and BSD systems and most other systems. — Stéphane Chazelas, Mar 03 '17 at 15:09
`-i''` is actually `-i` (`-i` concatenated with the empty string) so would only work with GNU `sed`. You'd need `-i ''` for BSDs. — Stéphane Chazelas, Mar 03 '17 at 15:10
@StéphaneChazelas There are Unices out there without Perl in the base system. NetBSD is one (but it may well be alone in that group). — Kusalananda, Mar 03 '17 at 15:21
@Kusalananda, true. In my experience, one is more likely to find `perl` than `bash` though. For instance here, many of the BSDs would not have `bash` or any shell that supports those `[[...]]` or `=~` (which can easily be replaced by a standard and more legible `sh` `case` construct). — Stéphane Chazelas, Mar 03 '17 at 15:43

Kusalananda · Accepted Answer · 2017-03-03T16:00:11.487

2

Do this to circumvent the problematic portability issues with the -i flag of sed:

sed 'sed-editing-commands' thefile >tmpfile && mv tmpfile thefile

I.e., write to a temporary file, and then replace the input file with the temporary file if the sed command didn't fail.

This is portable to all implementations of sed that I know of.

To create a temporary filename safely, use mktemp. Although this isn't a standard utility, it is available on all Unices that I have access to (OpenBSD, NetBSD, Solaris, macOS, Linux):

tmpfile=$(mktemp)
sed 'sed-editing-commands' thefile >"$tmpfile" && mv "$tmpfile" thefile

edited Mar 03 '17 at 16:00

answered Mar 03 '17 at 14:09

Kusalananda

320,670
36
633
936

However, that doesn't make any attempt to preserve the ownership, permissions and other metadata of the original file like GNU or BSD seds would. – Stéphane Chazelas Mar 03 '17 at 15:17
@StéphaneChazelas Good point. To do that, one would have to `cp thefile "$tmpfile"` first, I think. Does that sound about right? – Kusalananda Mar 03 '17 at 15:19
Some good related reading in Scott's answer here: http://unix.stackexchange.com/a/186126/117549 – Jeff Schaller Mar 03 '17 at 15:28

score 0 · Answer 2 · answered Mar 03 '17 at 14:32

0

You need to realize that the $mobile_app_api_url should not contain the chars that are meaningful to sed on the RHS of s///. Specifically, speaking, you need to make sure that this variable has the following properly escaped away: viz., & # \ newline before they are plugged in.

answered Mar 03 '17 at 14:32

It looks as if that is already taken care of. The URL will not cause problems. The issue is with the non-portable `-i` flag for `sed` (it work very differently with different implementations of `sed`). – Kusalananda Mar 03 '17 at 14:44
From the OP's code shown there is nowhere any escaping the characters that I have mentioned above in my post. Just try placing an & in $1 to see what I mean. – Mar 03 '17 at 14:50
1

The URL in `$mobile_app_api_url` will be either `https://apibeta.mysite.com` or `https://apilive.mysite.com`, no other values will be allowed. The escaping of characters in the substitution is furthermore an issue in all implementations of `sed` on all Unices. This question is about portability. – Kusalananda Mar 03 '17 at 14:55
1

@Kusalananda, any value of `$environment` that **contains** either `live` or `beta` would be allowed (though it doesn't look like it was the intention). – Stéphane Chazelas Mar 03 '17 at 15:53
@StéphaneChazelas Right you are. Any URL with `& # \ newline` in the domain part would be a bit broken anyway though. – Kusalananda Mar 03 '17 at 15:56

Stéphane Chazelas · Answer 3 · 2017-03-03T15:52:28.820

Instead of:

cp app/index.html.mob MobileApp/www/index.html
sed -i'' "s#MOBILE_APP_API_URL#\"$mobile_app_api_url\"#g" MobileApp/www/index.html

Just do:

sed "s#MOBILE_APP_API_URL#\"$mobile_app_api_url\"#g" \
  < app/index.html.mob >  MobileApp/www/index.html

You can also remove your dependency on ksh93/bash by changing the top part to:

environment=$1

case $environment in
  "") environment=beta;;
  live|beta) ;;
  *)
    printf >&2 'Invalid environment: "%s"\n' "$environment"
    exit 1;;
esac

Using sed command arguments to be compatible with both GNU & BSD Unix (in-place editing)

3 Answers3