0

I have below lines in unix script;

list_table_name=`hive -e 'show tables from schema'| grep -i 'hst_*' |sort -r|head -1`

When executing the script order of the commands is getting changed.

Sometimes the order is coming as

  1. hive -e 'show tables from schema'
  2. grep -i 'hst_*'
  3. sort -r
  4. head -1

but sometimes the order is coming as

  1. hive -e 'show tables from schema'
  2. sort -r
  3. head -1
  4. grep -i 'hst_*'

Why this is happening?

AdminBee
  • 21,637
  • 21
  • 47
  • 71
Gopi
  • 183
  • 5
  • Order of commands getting executed. My intention is to execute hive -e 'show tables from schema' then grep -i 'hst_*' then sort -r then head -1. But sometimes this is getting changed. – Gopi Jul 21 '21 at 08:47
  • 4
    Note that the commands in a pipeline are launched at the same time, or rather there is no deterministic order. See also https://stackoverflow.com/questions/66511243/bash-pipe-execution-order. Why do you need to start `hive` before `grep`? – berndbausch Jul 21 '21 at 08:47
  • 3
    All the processes in a pipeline are (in principle) started *simultaneously*. That is a race condition, which means whatever tool you are seeing their order in (maybe consecutive pids in `ps`) notices them in a random order. The actual data flow is always correct because the pipes are created and fds assigned before the processes are executed. – Paul_Pedant Jul 21 '21 at 08:49
  • @Gopi Is there anything that makes you think the final result (i.e. the value stored in `list_table_name`) is affected by the order the commands in your pipeline are executed? – fra-san Jul 21 '21 at 09:57
  • Yes @fra-san, I see different results than the expected ones. – Gopi Jul 21 '21 at 11:07
  • 1
    How did you determine in which order the commands were started up? What is the difference in the results? Please edit your post to include this information. – AdminBee Jul 21 '21 at 13:29
  • @Gopi "I see different results", but we *dont*. We see your interpretation of some event which you don't actually post. Are you claiming `list_table_name` contains multiple lines, in spite of the `head -1` in the pipeline? Please, show me an actual command, and its verbatim output, that contains the four lines you list. – Paul_Pedant Jul 21 '21 at 16:49

0 Answers0