I am looking for a way to add some string to the beginning of every line (same string for every line). Not something customizable but rather something that will be easy to remember and available on every POSIX-compliant platform (and every shell as well).
7 Answers
:|paste -d'foo ' - - - - input > output
(just kidding, though you'll probably find it's the fastest of all the solutions posted here :-b).
The canonical way is:
sed 's/^/foo /' < input > output
However, it's not easily adapted to arbitrary strings. For instance,
sed "s/^/$var /"
Only works if $var doesn't contain, &, \, / nor newline characters.
In that regard,
export var
awk '{print ENVIRON["var"], $0}'
or
perl -pe '$_="$ENV{var} $_"'
would work better.
- 522,931
- 91
- 1,010
- 1,501
-
that's something interesting... :P – Rahul Patil Oct 08 '13 at 20:12
You can use sed:
sed -i 's/^/your_string /' your_file
Thanks to Stephane and Marco's comments, note that the -i option isn't POSIX. A POSIX way to do the above would be
sed 's/^/your_string /' your_file > tmp_copy && mv tmp_copy your_file
or perl:
perl -pi -e 's/^/your_string /' your_file
Explanation
Both commands perform a regex substitution, replacing the beginning of a line (^) with your desired string. The -i switch in both commands makes sure the file is edited in place (i.e. the changes are reflected in the file instead of printed to stdout).
sed should be available on any POSIX-compliant OS and perl should be available on most modern Unices except perhaps for the ones that have gone through the effort of removing it.
-
1Small correction: The `-i` switch does not edit the file in place. It creates a new file and overwrites the original one after it's done (proof: the inode changes). There are not many tools which actually do in-place editing, which is a dangerous operation. Furthermore, since you mention POSIX, the `-i` switch is not mandatory for a POSIX compatible `sed`, it's a feature of some particular implementations. – Marco Oct 08 '13 at 20:00
-
2`sed -i` is not POSIX. It's GNU. FreeBSD `sed` has a similar option, but you need `sed -i ''` there. – Stéphane Chazelas Oct 08 '13 at 20:01
-
@Marco I'm aware that `-i` creates a temporary copy (that's why I said the changes are _reflected_ in the original file); I meant that you get the same semantics as in-place substitution. Please check that the updated answer is POSIX-compliant. – Joseph R. Oct 08 '13 at 20:06
-
Strictly speaking, the changes are reflected in a new file with the same name as the original. – Keith Thompson Oct 08 '13 at 23:17
-
1I often use `$$` (the current shell's process id) as part of a temporary file name; it avoids the effort of thinking of a guaranteed unique name every time: `command filename > filename.$$ && mv filename.$$ filename` – Keith Thompson Oct 08 '13 at 23:18
I present a solution using awk prepending the string “foo”.
awk '{ print "foo", $0; }' input > output
awk is cross-platform and available on any POSIX system. It does not do in-place editing. If you want to edit a file without creating a second one, you will have to use a temporary file. See Joseph's sed answer, it shows the syntax. Another hack is to use the following syntax, which is basically creating a temporary file with the same file name as the original file.
{ rm file; awk '{ print "foo", $0 }' > file; } < file
- 522,931
- 91
- 1,010
- 1,501
- 33,188
- 10
- 112
- 146
-
`reducto# wc -l foo` ` 914 foo` `reducto# { rm foo ; awk '{ print "prepend" , $0 }' > foo } < foo` `>` This doesn't seem to work. I was trying because the redirecting to original made me nervous, but it won't even start. – kurtm Oct 08 '13 at 20:45
-
1This example will work if it uses parentheses for the outside rather than curly braces. – kurtm Oct 08 '13 at 20:49
You can avoid the problems of in-place editing with the stream tools by using a tool that normally does in-place editing - an editor!
ex sample.txt -c "%s/^/foo /" -c wq
There is an additional advantage that the commands are easy and obvious to anyone who is versed in the one true editor.
- 151
- 2
You can use perl to do this:
$ perl -pi -e 's/^/mystring /' afile.txt
Example
Create a sample file.
$ seq 5 > afile.txt
$ cat afile.txt
1
2
3
4
5
Run the above command:
$ perl -pi -e 's/^/mystring /' afile.txt
$ cat afile.txt
mystring 1
mystring 2
mystring 3
mystring 4
mystring 5
- 363,520
- 117
- 767
- 871
-
Perl seems to be universal nowadays, but for a long time it wasn't. And I'm betting there are a couple rare UNIX variants out there that steadfastly refuse to have perl as part of the base. Something to be aware of. – kurtm Oct 08 '13 at 20:41
-
@kurtm see the discussion [here](http://chat.stackexchange.com/transcript/message/11307976#11307976). Apparently, AIX does not have Perl by default and nor do embedded systems. – terdon Oct 08 '13 at 20:53
-
@terdon Yeah. I suspected AIX wouldn't, but I haven't really worked with AIX, so couldn't say for sure. And yeah, perl is a bit bloated for embedded systems. That's why I put the warning on there. I remember the days when perl was only ever an add-on... – kurtm Oct 08 '13 at 20:56
-
Perl is not at all universal. It's popular in Linux distributions to include it in the base, but that's about it. – David Sainty Oct 08 '13 at 20:59
-
@DavidSainty The BSDs also seem to include it and I'm fairly certain Solaris does too (I'm not touching Solaris any more, so I'm not checking). But that's why I said "seems". – kurtm Oct 08 '13 at 21:01
-
2@DavidSainty - it's universal enough. The Q was tagged with Bash so I find it extremely hard to imagine a system that has Bash but not Perl. At any rate, here's one of 5 answers, several of the others show how to use `sed`, I showed how to use Perl....not looking for a holy war on the matter. – slm Oct 08 '13 at 21:01
-
-
1@DavidSainty Apparently only OpenBSD. I guess I made the assumption since the Open folks tend to be conservative about what they allow in base. OpenBSD definitively does have it in base. – kurtm Oct 08 '13 at 21:43
-
1@slm: "*I find it extremely hard to imagine a system that has Bash but not Perl*". I have several such systems sitting on my desk. (They happen to be embedded systems.) – Keith Thompson Oct 08 '13 at 23:20
-
@KeithThompson - What version of Bash, and not some stripped down version that's part of Busybox? – slm Oct 08 '13 at 23:50
-
@sim: It's bash 3.1.17, at least on the one I'm logged into at the moment. – Keith Thompson Oct 08 '13 at 23:56
-
@KeithThompson - Yeah this is a pretty ancient system, it's almost a 10 year old release, http://tldp.org/LDP/abs/html/bashver3.html. Shall I qualify my statement that I can't imagine a system with a release of Bash in the last 5 years 8-). – slm Oct 09 '13 at 00:01
-
@slm - i don't think there were too many *embedded systems* - at least not the kind you'd keep on your desk - capable of running a `bash` - v3 or otherwise - 5 or 6 years ago. However, there are many systems - even built today - which have a `bash` v3 installed by default *(like Mac OSX)*, because later releases are GPL v3 which is *very* unfriendly to to commercialization - especially of the embedded variety - of any kind *(some people say GPL v3 woldn't have happened at all were it not for TiVo)*. And so I doubt Keith's devs were so ancient as all that. – mikeserv Aug 02 '15 at 11:18
Simply using Bash
while IFS= read -r line; do echo "foo" "${line}" ; done < input > Output
Using Python
python -c "import sys; print 'foo '.join([ l for l in sys.stdin.readlines() ])" < input > Output
If you want to edit in-place
import fileinput
import sys
for line in fileinput.input(['inputfile'], inplace=True):
sys.stdout.write('foo {l}'.format(l=line))
- 24,281
- 25
- 80
- 96
-
-
1It's only going to work faster than `awk` on very small files. The behavior varies across echo implementations and it strips leading and trailing blanks and processes backslashes specially. – Stéphane Chazelas Oct 08 '13 at 20:12
-
@StephaneChazelas Yes. you are 100% correct just tested... http://paste.ubuntu.com/6210934/ – Rahul Patil Oct 08 '13 at 20:19
-
`prefix () { while IFS= read -r REPLY; do printf "%s%s\n" "$1" "$REPLY"; done; }` ought to be more correct. – Matt Oct 08 '13 at 20:34
-
1Why the `{}`? I think @Matt was suggesting you make it into a named function, what's the point of `{}` without a function name? – terdon Oct 08 '13 at 20:51
Or you can do the following:
vi filename.txt
Esc (to make sure you're in NORMAL mode), and then enter the following command:
:1,$ s/^/mystring
- 4,732
- 10
- 33
- 41
- 1
-
Note that since the user is asking about every line of the file, you could replace `1,$ ` with simply `%`. This makes the command `:%s/^/string/`. – HalosGhost Sep 12 '14 at 14:26