Randomly select a proportion of files from a given directory

Question

There are about 10000 files under a given directory. Are there any command that can help me randomly pick 1000 files from it and put them into another directory. The picked files should be removed from the original directory.

Answer 1

If you have shuf, it will easily let you do what you want, provided that no filename has a newline character in it, and there are no subdirectories:

mapfile -t sample < <(shuf -n 1000 -e given_directory/*)
mv "${sample[@]}" other_directory

If there are subdirectories, you could get the list of files by using find instead of the glob. Or you could oversample and filter. find will also help you deal with files which could have newlines in their names (which is really a bad idea, but that doesn't necessarily mean that you can ignore the possibility), since you can use the -print0 action combined with the -z flag to shuf. For example,

find given_directory -type f -print0 |
shuf -z -n 1000 |
xargs -0 mv -t other_directory

mv -t is a (very useful) Gnu extension which lets you provide the destination directory at the beginning of the command line, which works nicely with the xargs/find -exec model of putting multiple arguments at the end of the command line.