57

There is a standard command for file splitting - split.

For example, if I want to split a words file in several chunks of 10000 lines, I can use:

split -dl 10000 words wrd

It would generate several files of the form wrd.01, wrd.02 and so on.

But I want to have a specific extension for those files - for example, I want to get wtd.01.txt, wrd.02.txt files.

Is there a way to do it?

buhtz
  • 777
  • 1
  • 10
  • 22
Rogach
  • 6,150
  • 11
  • 38
  • 41

3 Answers3

86

With gnu split's more recent versions (≥ 8.16), one can use the --additional-suffix switch to have control over the resulting extension. From man split:

--additional-suffix=SUFFIX
              append an additional SUFFIX to file names.

so when using that option:

split -dl 10000 --additional-suffix=.txt words wrd

the resulting pieces will automatically end in .txt:

wrd00.txt
wrd01.txt
.........
Intrastellar Explorer
  • 117
  • 1
  • 7
don_crissti
  • 79,330
  • 30
  • 216
  • 245
  • 9
    Not working on mac – ericgu Feb 10 '15 at 04:51
  • 2
    I love your sarcasm. I am a unix n00b from the Apple world. I am using OS X Yosemite and I just didn't want others to crash and burn like I did. I tested and reviewed at the docs and we don't have this parameter. I might have missed something. https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/split.1.html – ericgu Feb 10 '15 at 15:46
  • 8
    @swiftshokunin - my answer pertains to `gnu split`, part of `gnu coreutils`. It is also available on OSX if you install `coreutils` via `homebrew` but note that by default, on OSX, the `gnu` utilities have a `g` prepended to their name (e.g. `gstat` instead of `stat`) so you invoke it as `gsplit` (or alter the PATH as per the guide [here](https://coderwall.com/p/2khjya/install-100-core-cli-utilities-with-1-command) if you want to use it as `split` over the OSX `split`). HTH. – don_crissti Feb 10 '15 at 16:50
  • 2
    Nice answer. on OS X, use `gsplit` to get the numeric suffixes (-d) to work. – Brent Faust Jul 24 '15 at 01:02
  • 2
    wow, i had no idea that there is gsplit - it's probably from coreutils mentioned above and it does have --additional-suffix. Thanks to everyone commenting this solution :) – Łukasz Rysiak May 31 '16 at 10:15
  • 1
    @don_crissti Can we split the file with a custom numeric start. i.e `wrd05.txt`. It should start with the number `5` or `7`. Is this possible? – Hussain7 May 31 '19 at 10:08
  • 2
    @Hussain7 -sure, just use `--numeric-suffixes=START` instead of `-d` e.g with OPs code you'd run `split -l 100 --numeric-suffixes=05 --additional-suffix=.txt words wrd` to have it start at `05`... (assuming `gnu split` here) – don_crissti May 31 '19 at 18:19
22

Such tasks are best managed with the shell. Use split and then write a simple loop to rename the files. E.g.

for file in wrd.*
do
    mv "$file" "$file.txt"
done

would rename your wrd.01, wrd.02, etc. files so they all have a .txt extension.

Kevin
  • 40,087
  • 16
  • 88
  • 112
Kyle Jones
  • 14,845
  • 3
  • 40
  • 51
  • That's quite obvious, but it would break the conciseness of bash script. – Rogach Feb 25 '12 at 05:08
  • 1
    The Unix philosophy is to provide you with a set of simple tools that you then combine to do a job. The "conciseness of the bash script" was not a stated requirement in your question. – Kyle Jones Feb 25 '12 at 05:19
  • I agree, but there are different combinations of simple tools out there, and some do job better. I just hoped that there was a good combination that could do this job better that a 5-line long split with file renaming. – Rogach Feb 25 '12 at 05:20
  • Just a personal obversation: When you actually get both versions to do the same thing with similar var names... `awk` 79 chars, `split+mv` 78 chars - - - - `awk '{f="wrd." sprintf("%03d",int((NR-1)/10000)) ".txt"; print >> f}' inputfile` - - - - `split -dl10000 -a3 inputfile wrd.; for f in wrd.???; do mv "$f" "$f".txt; done` – Peter.O Feb 25 '12 at 10:01
  • 7
    PS: the `split+mv` combo is more that **6 times faster** than `awk` (approx *3s* vs *18s*) for a 10 million line input file (75 MB)... the text in each line was its own line-number... Thanks for re-stating the "obvious" :) – Peter.O Feb 25 '12 at 10:36
  • 3
    PPS: I've just been checking this out a bit further. The speed-difference is related to the number of files created vs the number of formatting and arithmetic calculations awk does for each and every line regardless of the number of output files... Using the same input file as the above example: When there are *100 times fewer* files, `split + mv` is **75** times faster than `awk`: When there are *100 times more* files, `split + mv` is **1.5** times faster than `awk`. So, for me, this `split + mv` method wins, hands down. It is as consice (arguably moreso), and is faster than `awk`. – Peter.O Feb 25 '12 at 15:59
  • 3
    if you're concerned about it being 5 lines long, try this instead: `for file in wrd.*; do mv "$file" "$file.txt"; done` :) – Tony Oct 28 '15 at 04:20
14

Not with split, but you can easily rename them afterwards, or you can do it in awk:

awk '{filename = "wrd." int((NR-1)/10000) ".txt"; print >> filename}' inputfile
Kevin
  • 40,087
  • 16
  • 88
  • 112
  • Looks good - but does not work. In your form, complains about "expression for `>>' redirection has null string value", and if "file" is "changed" to "filename", outputs files of the form wrd.{file number}.{line number}.txt (quite a lot of them :) – Rogach Feb 25 '12 at 05:15
  • @Rogach Sorry, I hadn't tested it, so I forgot awk doesn't do integer division. I've tested this one. – Kevin Feb 25 '12 at 06:07