0

I have a folder in HDFS contains 830000 files, and I want to move the last "8797" files enter code here to another folder in HDFS? I tried using xargs but didn't work fine. Any other ideas?

Here is the exact split point between all files. I want to move files after "2021-03-09 15:15"

-rw-rw-r--+ 3 talend_user talend_group 102013 2021-03-09 15:14 /user/file_1

-rw-rw-r--+ 3 talend_user talend_group 9360 2021-03-09 15:15 /user/file_2

Omar AlSaghier
  • 101
  • 2
  • Last, in what sense? – Kusalananda Jun 20 '21 at 12:24
  • What command did you use and what didn't work? Add that and a sample of the files to the question. – Nasir Riley Jun 20 '21 at 12:39
  • Please reread and see the edits in the question – Omar AlSaghier Jun 20 '21 at 12:53
  • I'm assuming that you mean "8797 most recently modified files". With `zsh` that would be easily done with `mv /user/*(.om[1,8797]) /destdir` but I don't know anything about HDFS so I can't give a clear answer. It's unclear _how_ you tried doing this with `xargs`. – Kusalananda Jun 20 '21 at 12:58
  • @Kusalananda, actually it is easy in local file system to do it using "find" command, but in HDFS is it not direct. I think I am downloading all files to local system, do the split, and reupload them to hdfs again in the desired folders. – Omar AlSaghier Jun 20 '21 at 13:03
  • What hadoop version are you running? – mazaneicha Jan 04 '22 at 11:53

0 Answers0