41

I'm using sed to find and replace patterns within SAS files I have rather than changing them individually. The problem is I am trying to replace macro variables and when I use the ampersand it is not processing correctly.

Here's my code:
sed -ie 's/user=&uid./user=&sysuserid./g' *_table_*.sas

whenever I run this command it seems to append and do all kinds of funky stuff with the original text.

Question: How do I replace text that contains ampersands with sed command?

Jeff Schaller
  • 66,199
  • 35
  • 114
  • 250
DukeLuke
  • 513
  • 1
  • 4
  • 6
  • 7
    `&` is a special character so it needs to be escaped when used in the RHS - in your case you have to run `'s/user=&uid./user=\&sysuserid./g'` although I suspect you also need to escape the dot in the LHS to match a literal dot so you actually need `'s/user=&uid\./user=\&sysuserid./g'` – don_crissti Jul 18 '16 at 20:02
  • 1
    yes, '&' on the right basically means "everything that matched on the left", so escape it with '\'. You should make that the answer, rather than just a comment. – Edward Falk Jul 19 '16 at 00:06
  • i didnt have to escape the first period...just a heads up in anyone else was curious :) – DukeLuke Jul 20 '16 at 00:50

1 Answers1

45

& is special in the replacement text: it means “the whole part of the input that was matched by the pattern”, so what you're doing here replaces user=&uidX with user=user=&uidXsysuserid.. To insert an actual ampersand in the replacement text, use \&.

Another thing that looks wrong is that . in the search pattern stands for any character (except a newline), but the . at the end of the replacement text is a literal dot. If you want to replace only the literal string user=&uid., protect the . with a backslash.

sed -e 's/user=&uid\./user=\&sysuserid./g'

If you want to replace any one character and preserve it in the result, put the character in a group and use \1 in the replacement to refer to that group.

sed -e 's/user=&uid\(.\)/user=\&sysuserid\1/g'

In fact, given the repetition between the original text and the replacement, you should use groups anyway:

sed -e 's/\(user=&\)u\(id\.\)/\1sysuser\2/g'

i.e. “replace u by sysuser between user=& and id.”.

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
  • could you explain groups? – DukeLuke Jul 20 '16 at 00:53
  • 2
    @lucasdavis500 A group identifies a part of the pattern. It's delimited by backslash-parentheses. For example `\(user=&\)` is a pattern that matches `user&` and stores the match string as a group (group number 1 since it's the first group in the pattern). Then in the replacement `\1` is replaced by the string stored for group number 1. – Gilles 'SO- stop being evil' Jul 20 '16 at 00:57
  • do you mean \(user=&\) stores user=&, not user&? – DukeLuke Aug 17 '16 at 12:40
  • @lucasdavis500 I don't understand your comment. The character `=` in regular expressions and in replacement text stands for itself, so `user=&` matches only `user=&`, and `user=&` in the replacement text yields `user=` followed by the part of the line matched by the regex. – Gilles 'SO- stop being evil' Aug 17 '16 at 13:27
  • are you saying "=" is treated differently? I don't understand what you mean by \(user=&\) in the search will find the pattern user&. My search is for "user=&uid." and replacing it with "user=&sysuserid." how does replacing "user=&" with "user&" work? this code worked with my replacement text and maintained the "=" symbol. The way you described it leads me to believe it would not do so...maybe I'm reading it wrong – DukeLuke Aug 18 '16 at 18:07
  • or do you mean by your second comment that user=& in the replacement text yields user= because of the fact that "&" is treated differently in the replacement text without escaping it with a \. – DukeLuke Aug 18 '16 at 18:10
  • 1
    @lucasdavis500 The `=` character does not have any special meaning. `user=&` in the replacement text produces `user=` followed by the original matched text. `user=\&` in the replacement text produces `user=&` – Gilles 'SO- stop being evil' Aug 18 '16 at 18:15