16

In a larger script to post-process some simulation data I had the following line:

parallel bnzip2 -- *.bz2

Which, if I understand parallel correctly (and I may not), should run n-core threads of the program over all files with the listed extension. You may notice that I misspelled the command bunzip2. I would expect a warning or error message here, but it fails silently. Is this intended? How do I not get bit by this in the future?

Update:

It is possible that I have a different parallel installed than I think I do:

> parallel --version`


parallel: invalid option -- '-'
parallel [OPTIONS] command -- arguments
    for each argument, run command with argument, in parallel
parallel [OPTIONS] -- commands
    run specified commands in parallel

A man page of parallel on my system gives:

 parallel(1)                                                        parallel(1)
NAME
       parallel - run programs in parallel
....
AUTHOR
   Tollef Fog Heen

Which seems this is not the GNU version.

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
Hooked
  • 1,343
  • 3
  • 17
  • 24
  • Prints an error on my system. What does `parallel --version` say? ```$ parallel bnzip2 -- * sh: bnzip2: command not found``` – Mikel Apr 03 '12 at 15:03
  • GNU parallel is "Copyright (C) 2010,2011,2012 Ole Tange, http://ole.tange.dk and Free Software Foundation, Inc." (Same author since inception.) – Mat Apr 03 '12 at 18:02

2 Answers2

22

You have been hit by the confusion with Tollef's parallel from moreutils. See https://www.gnu.org/software/parallel/history.html

You can install GNU Parallel simply by:

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem

Watch the intro videos for GNU Parallel to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Ole Tange
  • 33,591
  • 31
  • 102
  • 198
  • 1
    It all makes sense now! For the record, on Ubuntu 12 right now `apt-get install parallel` pulls Tollef's parallel not your GNU version. Thanks for filling me in. – Hooked Apr 08 '12 at 00:03
  • 1
    For Ubuntu 12, see [Where do I get a package for GNU Parallel?](http://askubuntu.com/q/12764/182660) – Ian Oct 21 '13 at 19:51
12

(I would make this a comment, but I lack the privileges.)

You can also probably just use parallel --gnu. You said that Ubuntu 12(.04?) uses Tollef's version, but it's actually the gnu implementation, it just defaults to Tollef's syntax by default for some reason.

I added alias parallel='parallel --gnu' to my login script so I don't have to fuss with it anymore.

billyjmc
  • 513
  • 6
  • 10
  • 2
    You can also add `--gnu` to $HOME/.parallel/config. – Thor May 08 '13 at 21:03
  • Note: The answer here is addressing a comment from @Hooked. – Brent Bradburn May 30 '13 at 22:04
  • See here for another approach plus a bit more detail regarding versions: http://askubuntu.com/a/298598/11522. – Brent Bradburn May 30 '13 at 22:06
  • 1
    Fedora also frustratingly [puts --tollef in /etc/parallel/config](https://bugzilla.redhat.com/show_bug.cgi?id=920119). As if that is not obnoxious enough, GNU parallel itself [reports that --tollef will be removed on 20140222](http://www.gnu.org/software/parallel/man.html#options). – musicinmybrain Jun 19 '13 at 04:44