4

I need to transfer a large number of files from an FTP server to a new server. There could be thousands of files so I would like to limit it to files that were uploaded in the last three months - is that possible and if so how?

Also is it possible to find out how big the download is likely to be before you start the actually download?

Thanks

williamsdb
  • 165
  • 2
  • 8

2 Answers2

4

You can use lftp for that, utilizing its mirror command. Here's a snip from the manpage:

  mirror [OPTS] [source [target]]

   Mirror specified source directory to local target directory. If target
   directory  ends  with  a  slash,  the source base name is appended to 
   target directory name. Source and/or target can be URLs pointing to 
   directories.


[cut...]
        -N, --newer-than=SPEC    download only files newer than specified time
            --on-change=CMD      execute the command if anything has been changed
            --older-than=SPEC    download only files older than specified time
[...]

Definitely have a look at the manual, as there are really many useful options to mirror - like --allow-chown, --allow-suid or --parallel[=N] for example. Lftp also works with other access protocols, like sftp, fish or http(s).

rozcietrzewiacz
  • 38,754
  • 9
  • 94
  • 102
2

You could try mounting it as a filesystem using curlftpfs.
Then copy using find.
e.g:

mkdir /tmp/mountpoint
curlftpfs ftp://example.com/ /tmp/mountpoint/
cd /tmp/mountpoint
find -mtime +90 -exec cp {} /dest/path/ \;

I suspect better solutions exist.

donothingsuccessfully
  • 1,177
  • 7
  • 12