5

I have a file name called temp.csv in my script one of the intermediate step is 

echo "some info" > final.csv | cat temp.csv >> final.csv

at times the file final.csv is created without data which is in temp.csv (when it is running through scheduler). And then, when I rerun the job, then the final.csv is created as I expected.

Why it is happening like this (what exactly happening in the command echo "some info" > final.csv | cat temp.csv >> final.csv)?
If I replace the command in the following way:

echo "some info" > final.csv ; cat temp.csv >> final.csv

will this modification be helpful?

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
Aravind
  • 1,559
  • 9
  • 31
  • 44

2 Answers2

4

You are piping the output from echo "some info" > final.csv into cat temp.csv >> final.csv, so these two commands (echo and cat) run in parallel.

Because of that, what ends up in the final.csv depends on what program gets scheduled. What you want to do is replace that | with a ;. And then the echo command will run until finished, and only then cat is started.

Anthon
  • 78,313
  • 42
  • 165
  • 222
  • 1
    Why would the commands run in parallel if the latter one needs the former one to finish in order to receive the former's output piped to its input? – confused00 Oct 30 '14 at 10:55
  • 2
    @confused00 Why would they not, the latter one only needs the former to finish, to be able to finish itself. It can process what it gets in parallel. – Anthon Oct 30 '14 at 10:58
  • @confused00 - read up on how pipes work. http://unix.stackexchange.com/questions/79501/executing-piped-commands-in-parallel – slm Oct 30 '14 at 12:13
4

| mean pipe == output of the previous command piped to next command input. If you use > final.csv that redirect output of echo into file and remains nothing to pipe. So to correct you free use 2 command separatelly as you have offered above or use pipe if your want

echo "some info" | cat - temp.csv > final.csv

Other do the same

cat - temp.csv <<<"some info" > final.csv
Costas
  • 14,806
  • 20
  • 36