sed - replace string with file contents

Question

I have two files: file1 and file2.

file1 has the following contents:

---
  host: "localhost"
  port: 3000
  reporter_type: "zookeeper"
  zk_hosts: 
    - "localhost:2181"

file2 contains an IP address (1.1.1.1)

What I want to do is replace localhost with 1.1.1.1, so that the end result is:

---
  host: "1.1.1.1"
  port: 3000
  reporter_type: "zookeeper"
  zk_hosts: 
    - "1.1.1.1:2181"

I have tried:

sed -i -e "/localhost/r file2" -e "/localhost/d" file1
sed '/localhost/r file2' file1 |sed '/localhost/d'
sed -e '/localhost/r file2' -e "s///" file1

But I either get the whole line replaced, or the IP going to the line after the one I need to modify.

not sure, but does `cat file1 | sed -e 's/localhost/1.1.1.1/g'` work? — dchirikov, Jul 08 '14 at 18:57
https://stackoverflow.com/questions/6790631/use-the-contents-of-a-file-to-replace-a-string-using-sed Has a proper answer using `sed`. — MayeulC, May 02 '23 at 12:59
After messing around with answers here I discovered there is a `r ` command for sed that does this and is way easier to implement and has no issues with / in the file. https://unix.stackexchange.com/a/32912/201387 — DKebler, May 15 '23 at 15:05

score 29 · Answer 1 · edited Jul 08 '14 at 19:44

29

Here is a sed solution:

% sed -e "s/localhost/$(sed 's:/:\\/:g' file2)/" file1
---
  host: "1.1.1.1"
  port: 3000
  reporter_type: "zookeeper"
  zk_hosts: 
    - "1.1.1.1:2181"

You should use sed -i to make the change inplace.

If you can use awk, here is one way to do:

% awk 'BEGIN{getline l < "file2"}/localhost/{gsub("localhost",l)}1' file1
---
  host: "1.1.1.1"
  port: 3000
  reporter_type: "zookeeper"
  zk_hosts: 
    - "1.1.1.1:2181"

edited Jul 08 '14 at 19:44

Graeme

33,607
8
85
110

answered Jul 08 '14 at 19:13

cuonglm

150,973
38
327
406

5

+1 for `awk`. I imagine `sed` _is_ capable of this, but it will be terribly clumsy. This is where `awk` shines! – HalosGhost Jul 08 '14 at 19:14
1

@HalosGhost: It's seem both me and you had misunderstand the OP question, I updated my answer. – cuonglm Jul 08 '14 at 19:25
The command substitution for the `sed` solution should be double quoted in case the file contains spaces or glob characters. – Graeme Jul 08 '14 at 19:38
@Graeme: Thanks, updated! Feel free to make an editting. – cuonglm Jul 08 '14 at 19:39
Actually, it is a good idea to process the second file too and escape any slashes since these would break the `sed` command. – Graeme Jul 08 '14 at 19:48
2

You need to escape both `/` and `&` in the substitution. That's `"$(sed 's:[/\\&]:\\&:g' file2)"` – Toby Speight Dec 06 '16 at 12:40
I had a multiline pem file with newlines and found this ti work: KEY=$(cat /app/certs/key.pem); awk -v file="$KEY" 'BEGIN{}/\{PRIVATE_KEY\}/{gsub("\{PRIVATE_KEY\}",file)}1' config.json > temp.txt && mv temp.txt config.json . Based on the above, might be improvable but does the job :) – Vincent Gerris Sep 30 '20 at 12:56
`sed` option `-i` is not portable. It is valid for GNU but does not work on BSD systems (macOS included). – reichhart Mar 20 '22 at 17:39

score 18 · Answer 2 · answered Jul 08 '14 at 19:12

18

You can read the file with the replacement string using shell command substitution, before sed is used. So sed will see just a normal substitution:

sed "s/localhost/$(cat file2)/" file1 > changed.txt

answered Jul 08 '14 at 19:12

Volker Siegel

16,983
5
52
79

34

Does this work on multi-lined files and files with special characters? – Trevor Hickey Jun 12 '15 at 18:23
24

@TrevorHickey it doesn't. Sed just fails with an "unterminated `s' command" error. – DarioP Oct 07 '16 at 11:16
Is there any solution for Trevor Hickey's question? – Omid Estaji Feb 27 '23 at 17:03
@TrevorHickey do you mean multiple lines in file2? – Volker Siegel Feb 28 '23 at 21:52
Hey folks check this: https://stackoverflow.com/questions/6790631/use-the-contents-of-a-file-to-replace-a-string-using-sed – Jesse Nickles Aug 07 '23 at 23:22

score 2 · Answer 3 · edited Apr 16 '19 at 18:23

I also had this "problem" today: how to replace a block of text with the contents from another file.

I've solved it by making a bash function (that can be reused in scripts).

[cent@pcmk-1 tmp]$ cat the_function.sh
# This function reads text from stdin, and substitutes a *BLOCK* with the contents from a FILE, and outputs to stdout
# The BLOCK is indicated with BLOCK_StartRegexp and BLOCK_EndRegexp
#
# Usage:
#    seq 100 110 | substitute_BLOCK_with_FILEcontents '^102' '^104' /tmp/FileWithContents > /tmp/result.txt
function substitute_BLOCK_with_FILEcontents {
  local BLOCK_StartRegexp="${1}"
  local BLOCK_EndRegexp="${2}"
  local FILE="${3}"
  sed -e "/${BLOCK_EndRegexp}/a ___tmpMark___" -e "/${BLOCK_StartRegexp}/,/${BLOCK_EndRegexp}/d" | sed -e "/___tmpMark___/r ${FILE}" -e '/___tmpMark___/d'
}

[cent@pcmk-1 tmp]$
[cent@pcmk-1 tmp]$
[cent@pcmk-1 tmp]$ cat /tmp/FileWithContents
We have deleted everyhing between lines 102 and 104 and
replaced with this text, which was read from a file
[cent@pcmk-1 tmp]$
[cent@pcmk-1 tmp]$
[cent@pcmk-1 tmp]$ source the_function.sh
[cent@pcmk-1 tmp]$ seq 100 110 | substitute_BLOCK_with_FILEcontents '^102' '^104' /tmp/FileWithContents > /tmp/result.txt
[cent@pcmk-1 tmp]$
[cent@pcmk-1 tmp]$
[cent@pcmk-1 tmp]$ cat /tmp/result.txt
100
101
We have deleted everyhing between lines 102 and 104 and
replaced with this text, which was read from a file
105
106
107
108
109
110

`sed` is a powerful command; needing to pipe one `sed` into another is rare. I believe that `sed -e "/${BLOCK_EndRegexp}/r ${FILE}" -e "/${BLOCK_StartRegexp}/,/${BLOCK_EndRegexp}/d"` works; do you know of a counterexample? — Scott - Слава Україні, Jul 23 '20 at 14:12

score 0 · Answer 4 · answered Jul 08 '14 at 18:46

0

Try using

join file1 file2

and then, remove any unwanted fields.

answered Jul 08 '14 at 18:46

unxnut

5,908
2
19
27

sed - replace string with file contents

4 Answers4

Linked

Related