Is there another simple way to append line to end of file other than `>>`?

Question

Recently I'm echoing short sentences to a tree_hole file.

I was using echo 'something' >> tree_hole to do this job.

But I was always worried of what if I mis-input of > instead of >>, since I did this often.

So I made a global bash func of my own in the bashrc:

function th { echo "$1" >> /Users/zen1/zen/pythonstudy/tree_hole; }
export -f th

But I'm wondering if there is another simple way to append lines to the end of a file. Because I may need to use that often in other occasions.

Is there any?

Will you not forget you are using the work-around each time you enter > ? Once I had an alias `rm="rm -i"` and in an other environment wrote `rm *` waiting for the confirmation questions. You are learning dangerous habits ! — Walter A, Jan 12 '15 at 12:37
@WalterA er, his "workaround" doesn't let him type > instead of >>, he just runs "th some_sentence", which does nothing if the alias is undefined. — Random832, Jan 12 '15 at 14:11
@Ranom832 correct for his th work-around. The warning is for a "solution" like noclobber. When he uses something like noclobber in his normal shell he might use > when he is temporary root and wants to append something. — Walter A, Jan 12 '15 at 14:27
@Random832 & WalterA I normally don't say anything about this when I see it, but i figured maybe a friendly notice once in a while could be useful. Zen's user profile doesn't have much detail, so I'm not sure if you really know that _his "workaround"_ is the correct form. Maybe you should say _their "workaround"_ or _the OP's "workaround"_. Perhaps you know Zen personally and you therefore know that _his_ is correct, in which case, please pardon the noise. Not a big deal, I just mention it because I know I wouldn't have appreciated it very much if you had used that form to talk about me. — Celada, Jan 12 '15 at 14:56
@WalterA I agree about dangerous habits. I usually use `mv -i` for safety, but I've trained myself to type the option, not alias it. — Celada, Jan 12 '15 at 14:58
@WalterA, I'm not quite understand your worrying. Since I adopted Celada's plan, I will only be used to `>|` when I want to write something to a file, and still I'll use `>>` to append something to files. The chance to mis-operate won't improve. — Zen, Jan 13 '15 at 02:48
@Zen, You wrote you are worried to mis-input > instead of >>. Celada's plan will take away the risks in your environment, and is a good solution for you. When you are helping your neighbour (who doesn't has noclobber or is using ksh) and you lost your attention for one or two > characters, you might accidentally overwrite one of his files. So everytime you get the noclobber warning in your own environment, thank God or Celada, say to yourself: Ohh, be carefully please!, shake your head and wait two seconds. — Walter A, Jan 13 '15 at 09:29

score 46 · Accepted Answer · answered Jan 12 '15 at 09:12

46

Set the shell's noclobber option:

bash-3.2$ set -o noclobber
bash-3.2$ echo hello >foo
bash-3.2$ echo hello >foo
bash: foo: cannot overwrite existing file
bash-3.2$

answered Jan 12 '15 at 09:12

Celada

43,173
5
96
105

3

This is the definitive answer. Note that you can still overwrite the file if you force it with `>|`. `zsh` enables this behaviour by default. – orion Jan 12 '15 at 09:17
1

@orion How do you en/disable it in zsh? I don't recall explicitly disabling it, but overwriting works fine on my zsh. – muru Jan 12 '15 at 09:22
2

It's `setopt noclobber` and `setopt clobber`. It appears it isn't exactly "default", it depends on the config files that are shipped with your distro. – orion Jan 12 '15 at 09:36
@muru should work with `set [+-]C` for any modern shell – mikeserv Jan 12 '15 at 10:14
Yes, this is great, to replace `>` by `>|`, it really fixed the potential mis-input. But one more question, how should I cancel the `set -o noclobber`, since I tried `set -o clobber`, which didn't work. – Zen Jan 12 '15 at 10:18
3

@Zen `set +o noclobber` in bash, or `set +C`. – Ruslan Jan 12 '15 at 10:29

score 6 · Answer 2 · edited Nov 03 '16 at 01:47

6

If you are worried your file will be damaged by > operator you can change your file attribute to append only:
In ext2/ext3/ext4 filesystem: chattr +a file.txt
In XFS filesystem: echo chattr +a | xfs_io file.txt

And if you want a function, I made a function for myself already (I used it in service file for logging outputs), You can change it for your purpose:

# This function redirect logs to file or terminal or both!
#@ USAGE: log option data
# To the file     -f file
# To the terminal -t
function log(){
        read -r data       # Read data from pipe line

        [[ -z ${indata} ]] && return 1    # Return 1 if data is null

        # Log to /var/log/messages
        logger -i -t SOFTWARE ${data}

        # While loop for traveling on the arguments
        while [[ ! -z "$*" ]]; do
                case "$1" in
                        -t)
                                # Writting data to the terminal
                                printf "%s\n" "${data}"
                                ;;
                        -f) 
                                # Writting (appending) data to given log file address
                                fileadd=$2
                                printf "%s %s\n" "[$(date +"%D %T")] ${data}" >> ${fileadd}
                                ;;
                        *)
                                ;;
                esac
                shift           # Shifting arguments
        done
}

edited Nov 03 '16 at 01:47

Jeff Schaller

66,199
35
114
250

answered Jan 12 '15 at 09:30

Sepahrad Salour

2,629
3
20
27

2

There are a few problems with this script. 1) You should generally put double quotes around parameter expansions (stuff that starts with a `$`) to prevent globbing, word splitting and related whitespace problems. The online [ShellCheck](http://www.shellcheck.net) app can catch this and various other problems in bash scripts. On a related note `"$@"` is _much_ safer than `$*`. – PM 2Ring Jan 12 '15 at 10:39
3

2) You should normally call the `read` command with the `-r` option to prevent any backslashes in the input from escaping the following character. 3) It's usual in bash scripts to use lower case for script variable names since ALL UPPER CASE is used for system variables. So if you use upper case for your own variables you will confuse people who read your code, and you may accidentally clobber a system variable that you want to use later in the script. And if you [source](http://en.wikipedia.org/wiki/Source_%28command%29) the script, it will clobber the variables for subsequent commands. – PM 2Ring Jan 12 '15 at 10:44
3

4) `echo` is handy for printing fixed strings, but it can do unexpected things with arbitrary data, especially on GNU systems. It's _far_ safer to use `printf`. See http://unix.stackexchange.com/questions/65803/why-is-printf-better-than-echo for more info, especially Stéphane Chazelas's answer. – PM 2Ring Jan 12 '15 at 10:52
3

@PM 2Ring, Today I learned many things from you, Thanks a lot.. I will fix them as soon as possible. – Sepahrad Salour Jan 12 '15 at 12:43
@PM 2Ring, Post edited. Thank you again. – Sepahrad Salour Jan 13 '15 at 15:08
1

Nice work, Sepahrad! – PM 2Ring Jan 14 '15 at 01:30

score 3 · Answer 3 · answered Jan 12 '15 at 08:59

3

Use tee with the append option:

foo | tee -a some-file
# or
tee -a some-file <<EOF
blah blah
EOF
# or 
tee -a some-file <<<"blah blah"

answered Jan 12 '15 at 08:59

muru

69,900
13
192
292

5

Good idea, +1, but I imagine that if the OP is worried about forgetting a `>` character then they might be worried about forgetting the `-a` option! – Celada Jan 12 '15 at 09:10
@Celada I suppose so (but I'd guess a typo from missing one keystroke in a set of repeated keystrokes is more likely to happen than forgetting an option). – muru Jan 12 '15 at 09:11

Jasen · Answer 4 · 2015-01-12T10:23:56.723

0

many programs that can open files for overwriting can alternatively open them for append, eg gnu dd.

dd conv=notrunc oflag=append of=file

it can read stdin or a file named in the if= parameter add 2>/dev/null to suppress the byte count.

edited Jan 12 '15 at 10:23

answered Jan 12 '15 at 09:26

Jasen

3,715
13
14

3

It's a Good Idea™ to test your code before posting it as answer... – PM 2Ring Jan 12 '15 at 10:28
1

Recommending `dd` is a bad idea. [`dd` does not reliably write its input to its output](http://unix.stackexchange.com/questions/17295/when-is-dd-suitable-for-copying-data-or-when-are-read-and-write-partial), not when one of them isn't a regular file or block device – Gilles 'SO- stop being evil' Jan 13 '15 at 22:52
@gilles, you are misrepresenting the information at that link, when count is not specifued dd will copy all the data available, same as cat does. – Jasen Jan 14 '15 at 05:40

score -1 · Answer 5 · answered Jan 12 '15 at 09:27

-1

I'd like use a sed (even with backup-copy - see extention after -i):

sed -i.bak '$ a\something' /Users/zen1/zen/pythonstudy/tree_hole

answered Jan 12 '15 at 09:27

Costas

14,806
20
36

Why am I not surprised... you're a sed maniac, Costas. :) (That's a compliment, BTW) – PM 2Ring Jan 12 '15 at 10:47
@PM2Ring Yes, I'm very dangerous! :P – Costas Jan 12 '15 at 12:07
Of course the way that works is to make a copy of the file, add the new text to the end, then delete the original file and rename the copy to the original name. This (1) doesn't work if you don't have write access to the directory, (2) breaks links, and (3) is resource-expensive if the file is large. Funny that *you* brought up the word "dangerous"! – G-Man Says 'Reinstate Monica' Aug 08 '15 at 21:25

mikeserv · Answer 6 · 2015-01-12T17:48:21.640

You can always seek through the file in other ways...

seq 10000000 >f
{ wc -l >&2; echo new line\!; } <>f >&0; \
{ wc -l >&2; echo new line\!; } <>f >&0; \
{ wc -l >&2; echo new line\!; } <>f >&0; \
{ wc -l >&2; echo new line\!; } <>f >&0; \
{ wc -l >&2; echo new line\!; } <>f >&0; \
wc -l f; tail f

...that weird looking sequence prints:

10000000
10000001
10000002
10000003
10000004
10000005 f
9999996
9999997
9999998
9999999
10000000
new line!
new line!
new line!
new line!
new line!

But that's kinda silly.

A more useful example could look like:

 apnd() if    shift
        then  wc -l  >&2
              printf "$@"
        fi    <>"$1" >&0

You can call it like:

 apnd /path/to/file          \
      "${printf_fmt_string}" \
       arbitrary list of strings

And you'd wind up with a count of file's lines written to stderr just before the append action takes place.

Is there another simple way to append line to end of file other than `>>`?

6 Answers6