44

I have this huge folder with thousands of unordered files. Is it feasible to move the first 5000s to a subfolder via the mv command? For now I move files with 

 mv *some_pattern* ./subfolder1/

As for now, I move images quite randomly, it's not really important if there aren't exactly 5000 files in each subfolder. Is there a better way to do it?

Anthon
  • 78,313
  • 42
  • 165
  • 222
Fabinout
  • 543
  • 1
  • 4
  • 5
  • See also [Distributing thousands of files over subfolders](http://unix.stackexchange.com/q/63265/22565) – Stéphane Chazelas Dec 13 '13 at 17:33
  • 2
    Does this answer your question? [How to move 100 files from a folder containing thousands?](https://unix.stackexchange.com/questions/12976/how-to-move-100-files-from-a-folder-containing-thousands) – don_crissti Dec 02 '22 at 10:53

7 Answers7

66
mv `ls | head -500` ./subfolder1/
Stéphane Chazelas
  • 522,931
  • 91
  • 1,010
  • 1,501
schaiba
  • 7,493
  • 1
  • 33
  • 31
  • 22
    (assuming none of the filenames contain space, tab, newline, star, open square bracket, question mark characters or start with `-` or `.` and assuming `subfolder1` itself does not show up in that list.) – Stéphane Chazelas Dec 13 '13 at 17:05
  • @StéphaneChazelas if the file contains those, how do we modify the command? – Peiti Li Nov 10 '17 at 23:19
  • 2
    ```$ sudo mv `ls | head -50000` 01/ sudo: unable to execute /bin/mv: Argument list too long``` – Brian Thomas Oct 07 '19 at 18:38
  • For me in MacOS the solution worked removing the `.` Something like: `mv `ls | head -500` /subfolder1/` – Flavio Dec 09 '20 at 12:41
  • Regarding the comment above in dealing with spaces, tabs, etc. This command worked nicely after **changing ``$IFS``**, as my files contained spaces. – S3DEV Aug 23 '21 at 08:21
  • mkdir ./subfolder2 | mv `ls *.xml | head -500` ./subfolder2/ – Eugene Kaurov Jun 03 '22 at 14:21
19

With zsh and its glob qualifiers:

mv -- *(D.oN[1,5000]) ./subfolder1

To move up to 5000 regular files in the order they are in the directory.

For the first 5000 in the lexicographically sorted list:

mv -- *some_pattern*(D.[1,5000]) ./subfolder1

If you get an error about arg list too long. You can use zsh's buitin mv command by issuing:

zmodload zsh/files

first.

POSIXly:

set --
for f in .*some_pattern* *some_pattern*; do
  [ "$#" -lt 5000 ] || break
  [ -f "$f" ] || continue
  [ -L "$f" ] && continue
  set -- "$@" "$f"
done
[ "$#" -eq 0 ] || mv -- "$@" subfolder1/

On a GNU system, you could also do:

find . ! -name . -prune -name '*some_pattern*' -type f -print0 |
  head -zn 5000 |
  xargs -r0 mv -t ./subfolder1 --
Stéphane Chazelas
  • 522,931
  • 91
  • 1,010
  • 1,501
14

A version that is simple and supports special chars, spaces, etc.

ls -Q dir1 | head -1000 | xargs -i mv dir1/{} dir2/

For this to work as-is dir2 must exist and you have to execute it from the parent directory of dir1 and dir2.

This will move 1000 files from dir1 to dir2.

Luis Antolín Cano
  • 593
  • 4
  • 13
  • nice one! ```ls -Q -S dir1 | head -1000 | xargs -i mv dir1/{} dir2/``` for moving 1000 largest files in dir1 (-S lists file by size) – oneklc May 03 '18 at 23:05
  • 2
    Note that `ls -Q` does not produce an output compatible with `xargs`'s expected input format. It helps for file names containing space characters, but not for double quotes or backslashes and harms for file names containing control characters including TAB. – Stéphane Chazelas Jun 29 '18 at 13:17
8
  1. Go to the directory which you want to move files from.
  2. Run below command: 
    find . -name 'Hello*.gz' | head -n 5000 | xargs -d $'\n' mv -t /data01/path/

In the find command, . (dot) denotes current directory

Finds files which start with Hello and end with .gz, first 5,000 files will be moved to /data01/path/.

Walf
  • 1,254
  • 1
  • 14
  • 19
Pratik Hiremath
  • 81
  • 1
  • 1
3

You might need to do something like this:

x=1
for file in *
do
    if [ "X$x" = "X#####" ]; then
        break
    fi
    mv $file <destination>
    x=`expr $x + 1`
done

This script works in bash, ksh, sh and multiple UNIX variants.

Karlson
  • 5,845
  • 32
  • 51
  • 1
    (provided none of the filenames contain space, tab, newline, star, open square bracket, question mark characters or start with - or . and provided `destination` itself does not show up in that list.) – Stéphane Chazelas Dec 13 '13 at 17:25
  • @StephaneChazelas True. This is not a complete solution just a method of dealing with the problem. – Karlson Dec 13 '13 at 17:39
2

I was able to succesfully do a move of 50,000 without the mv bash error like this 

 ls | head -50000 | xargs -I{} sudo mv {} 01/

Funny enough, this was on a samba share, so the 50k limit is because Windows Explorer GUI does not like more than 60k files in a dir in general.

Brian Thomas
  • 716
  • 6
  • 18
  • Yes solve error `zsh: argument list too long: mv` in my case `ls i | head -500000 | xargs -I {} mv i/{} o/{}` – Ax_ Nov 08 '22 at 11:14
2

Version working even in extremely large directories: moving thousand files from directory containing half million:

ls -U . | head -1000 | xargs mv -t 01/

(Note that the syntax with {} is spawning new mv for every file, while this one doesn't)

Honza
  • 129
  • 1