Use of ^ as a shell metacharacter

Question

I wrote a small script today which contained

grep -q ^local0 /etc/syslog.conf

During review, a coworker suggested that ^local0 be quoted because ^ means "pipe" in the Bourne shell. Surprised by this claim, I tried to track down any reference that mentioned this. Nothing I found on the internet suggested this was a problem.

However, it turns out that the implementation of bsh (which claims to be the Bourne shell) on AIX 7 actually has this behaviour:

> bsh
$ ls ^ wc
      23      23     183
$ ls | wc
      23      23     183

None of the other "Bourne shell" implementations I tried behave this way (that is, ^ is not considered a shell metacharacter at all). I tried sh on CentOS (which is really bash), and sh on FreeBSD (which is not bash). I don't have many other systems to try.

Is this behaviour expected? Which shells consider ^ to be a pipe metacharacter?

I know that `^` is a negation character in zsh and also in regex space. As a separate comment, it is generally advised to use single quotes in grep expression for portability across shells. — Ketan Maheshwari, Dec 09 '13 at 00:48
The bourne shell had a lot of odd behavior that we still see work arounds for in modern shell code eg `[ x"$foo" = x"bar" ]`. — jordanm, Dec 09 '13 at 01:46
`bsh` it not the Bourne Shell. The name is abused for the Bourne Shell on AIX only. `bsh` is rather a shell introduced by me in 1984 at H.Berhold AG on UNOS (the first UNIX clone). Note that AIX was not existent in 1984. — schily, Sep 06 '15 at 13:27

score 21 · Accepted Answer · edited Jun 25 '15 at 10:15

The ^ character as a synonym of | dates back from the Thompson shell. They were introduced at the same time in Unix v4 and are mentioned together in the man page. Sven Mascheck mentions that ^ was “probably [introduced] for reasons of convenience on early upper-case-only terminals” where typing | was “somewhat of a pain”.

The Thompson shell is long gone, but its successor the Bourne shell retained the same syntax (even though its man page only mentions |).

Successor shells such as ash, bash and ksh only understand | as the pipe character. You aren't going to find an actual Bourne shell on open source unix variants since for a long time there was no open source release of the Bourne shell. (I think OpenSolaris included one, but it wasn't adopted elsewhere as by that time it was long obsoleted by newer implementations).

The Single Unix specification does not mention ^ as a special character, which effectively means that POSIX shells should interpret it literally¹. I don't think there's ever been a fully POSIX-compliant variant of the Bourne shell (only independent implementations).

^ is special in zsh when the option extendedglob is enabled, but not in its sh compatibility mode. In its default mode, it deviates from POSIX in many ways.

I recommend quoting ^ in a regular expression anyway for clarity. Quote the regular expression in a script regardless of what characters appear in it.

¹ _{Except as the first character of a bracket expression in a wildcard pattern, where ! is the standard negation character but implementations may also interpret ^ in the same way.}

For completeness, you may want to mention that `^` is special in `fish` where it's a redirection operator, `rc`/`es` where it's a _concatenation operator_, or csh/tcsh/bash/zsh for history expansion when it's the first character of the command line. — Stéphane Chazelas, Apr 18 '16 at 16:21

score 3 · Answer 2 · answered Dec 09 '13 at 01:57

3

It turns out that ^ seems to be an undocumented feature of the SVR 4.2 shell. From GNU Bash Appendix B Major Differences From The Bourne Shell:

The SVR4.2 shell treats ‘^’ as the undocumented equivalent of ‘|’.

answered Dec 09 '13 at 01:57

Greg Hewgill

7,003
2
31
34

score 3 · Answer 3 · edited Jun 25 '15 at 09:53

Yes, OpenSolaris includes the Bourne Shell source but that source is not portable.

A maintained and highly portable version of the Bourne Shell source can be found here in the schily-*.tar.bz2 archives.

Here is the related part of the source in cmd.c:

/* 
* ^ is a relic from the days of UPPER CASE ONLY tty model 33s 
*/ 
if ((t = item(TRUE)) != 0 && (wdval == '^' || wdval == '|'))

You see, this is not related to a specific shell (e.g. the Thompson shell) but to the fact that in the 1970s there still have been upper case only terminals around.

Use of ^ as a shell metacharacter

3 Answers3

Linked