1

I was wondering, which of these programming languages are faster at handling big data files of only strings: 

Haskell, C++(Boost), AWK, Bash, Lisp...

So far I have found the paper Haskell vs. Ada vs. C++ vs. Awk vs. .... However, this paper is a little out of date.

Another.Chemist
  • 135
  • 7
  • most time will be spent programming anyway, so the faster is the one that you feel more confident in programming. If you just pick one and start programming right away you'll probably finish your job faster than if you spend some days deciding which language is the fastest, learn the language, write your software, and enjoy the fastest execution. Not to mention, the program you write will be written by a newbie so it will probably not be as fast as the language allow to. – pqnet Sep 05 '14 at 00:37
  • This question appears to be off-topic because it is about programming. See [What kind of topics can I ask about here?](http://unix.stackexchange.com/help/on-topic) in the FAQ. – Michael Mrozek Sep 05 '14 at 00:43
  • Also [crossposted](http://stackoverflow.com/questions/25676625/haskell-c-awk-bash-lisp-which-one-is-faster) (and closed on SO)... – jasonwryan Sep 05 '14 at 00:48
  • Experiment and see for yourself. I haven't tried all these languages, but certainly if you want speed plus low memory usage, C++ would be hard to beat. ocaml is also noted for speed. – Faheem Mitha Sep 05 '14 at 01:37

1 Answers1

3

Since "handle big data files" does not give much more information of what you want to do, i'm guessing some simple data processing.

Bash shell scripts are generally terribly slow by themselves, but for text processing purposes might be just what you need, in combination with the cli tools such as grep, sed, awk, cut, sort, uniq, wc and others. In many cases this is the fastest working solution for your problem, especially if it is a one-time util you need. For more info on this approach, see [1]

If you need to handle big data files in your existing project, I would recommend against rewriting it in a different language [2]

If you need to write a tool for processing these files, and have no constraints, then a C++ program would be able to produce the fastest code. But "speed" also boils down to speed of development and are as I mentioned above, in relation to how much you will run the program.

1: http://www.commandlinefu.com/

2: http://onstartups.com/tabid/3339/bid/2596/Why-You-Should-Almost-Never-Rewrite-Your-Software.aspx

Martin Olika
  • 519
  • 3
  • 5