3

I want to delete all log files in some directory, but not the latest 3.

I've done:

DATA_PATH=$(gadmin config get System.DataRoot)
ZK_PATH=${DATA_PATH}/zk/version-2

log_count=$(ls -ltrh ${ZK_PATH} | grep log | wc -l)
limit_files=`expr $log_count - 3`

echo There is ${log_count} files found in ${ZK_PATH}, ${limit_files} will be deleted, here the list:
ls -ltrh ${ZK_PATH} | grep log | head -${limit_files}

while true; do
    read -p "Are you sure to delete these files? " yn
    case $yn in
        [Yy1]* ) echo execute to delete the files; break;;
        [Nn0]* ) exit;;
        * ) echo "Please answer y or n.";;
    esac
done

How can I delete 9 of 12 files listed?

I thought about printing the list to the files first and then delete it one by one using looping, but I am sure there is code to do it only with one line.

I try to use find ... -delete, -exec, and xargs rm, but I cannot use it properly.

How can I do it?

Peter Mortensen
  • 1,029
  • 1
  • 8
  • 10
Yudha Restu
  • 33
  • 6
  • Re: `log_count=…`: [Why not to parse `ls` output (and what to do instead)](https://unix.stackexchange.com/questions/128985/why-not-parse-ls-and-what-to-do-instead); Grepping for `log` is also a bit strange; you could just have done something like `${ZK_PATH}/*log*` and gotten a list of the files, which is properly countable. – Marcus Müller Mar 20 '23 at 08:17
  • i try to use find and delete all the log files found, but i want to delete the files but no with the latest 3. like i found 12 log files, so i want to delete 9 files of it – Yudha Restu Mar 20 '23 at 08:25
  • yes, you wrote that. See my comment above. – Marcus Müller Mar 20 '23 at 08:26
  • is the find command already sort the files found by time modified? – Yudha Restu Mar 20 '23 at 08:36
  • You haven't written a `find` command, so we can't tell you whether you told it to do that. But also, find is not a useful tool here. As Stéphane's answer shows, this can be done through automated filename generation alone (see my first comment!). On its own, `find` can't, but you can decorate with time, zero-terminate, sort, de-decorate the things. Again, complicated and not necessary, as Stéphane's answer shows. – Marcus Müller Mar 20 '23 at 08:46

3 Answers3

6

Use zsh instead of bash where this kind of thing can be done much more easily and reliably:

#! /bin/zsh -
data_path=$(gadmin config get System.DataRoot) || exit
zk_path=$data_path/zk/version-2
keep=3

files=( $zk_path/*log*(N.om) )

if (( $#files > keep )); then
  printf >&2 '%s\n' "$#files files found in $zk_path, $(( $#files - keep )) will be deleted, here is the list from newest to oldest:"
  shift keep files
  printf >&2 ' - "%s"\n' $files:t

  read -q '?OK to delete? ' && rm -f -- $files
fi

Of special interest is the $zk_path/*log*(N.om) glob; it expands to all file names matching the pattern *log* in $zk_path, and the parentheses (…) specify that if no files are found, that's no error (Nullglob), that we're only looking for plain files and not directories, symbolic links, devices… (.), and that we want to have the files sorted in ascending order of age (based on modification time, like ls -t does).

Stéphane Chazelas
  • 522,931
  • 91
  • 1,010
  • 1,501
  • That's a very elegant approach there with the N-shift. Like it a lot! – Marcus Müller Mar 20 '23 at 08:45
  • Thanks, and for the edit @Marcus – Stéphane Chazelas Mar 20 '23 at 08:47
  • I felt a bit bad making that edit, it's your answer after all :) – Marcus Müller Mar 20 '23 at 08:49
  • Thank for the answer, i've tryed ```find . -name "log.*" | head -5 | xargs rm -r``` as a test and its looks work as far, but i still doubt. I just now about zsh, i'll learn about it, thanks for the answer and the knowladge – Yudha Restu Mar 20 '23 at 08:54
  • @YudhaRestu that is not sorted by date at all! Instead, it's just ordered in the order that `find` is presented with the files by the file system – which isn't guaranteed to follow any specific ordering. So, your `find` solution (in which the `find` is frankly a bit superfluous – `echo log.*` would have done the same) is complicated but wrong. Also very fragile! I sketched how you could do this correctly with find [in my previous comment](https://unix.stackexchange.com/questions/740331/find-files-and-delete-x-from-total-of-the-list#comment1406016_740331). Your solution really **isn't**. – Marcus Müller Mar 20 '23 at 09:16
  • @MarcusMüller. Don't. I've answered hundreds of questions with zsh glob qualifiers. Describing them every time is quite tedious. Any help is appreciated. – Stéphane Chazelas Mar 20 '23 at 09:19
  • @YudhaRestu see also [Why is looping over find's output bad practice?](https://unix.stackexchange.com/q/321697) for the good and bad ways to process `find`'s results. – Stéphane Chazelas Mar 20 '23 at 09:20
  • 1
    @MarcusMüller, strictly speaking, some `find` implementations do or can do some ordering of the files based on inode number as an optimisation so the inode table can be read sequentially when find needs to get information from there like the file's type. In any case, the order will still appear random. – Stéphane Chazelas Mar 20 '23 at 09:26
6

In a similar situation I used the following approach (probably, this is not the most robust solution, but it still may be useful in common situation...)

LOGDIR=my/logs    ## 

ls -dpt -- "$LOGDIR"/log* | # get log files sorted by date
  grep -v '/$'            | # but remove directories/ from the list
  tail -n +4              | # remove also the newest 3
  vidir -

In vidir, if you agree, delete all the lines (example: use dG vim command), leave, and they get deleted.

vidir from moreutils is a projection-editor for directories (the best tool to see, rename, delete, directory contents)

Stéphane Chazelas
  • 522,931
  • 91
  • 1,010
  • 1,501
JJoao
  • 11,887
  • 1
  • 22
  • 44
0

This gets you a list of all but the youngest limit_files files, which you can then remove.

FileList=`ls -1rt ${ZK_PATH}/*log* | head -${limit_files}`
rm $FileList

It won't work with files that contain spaces or other special characters, but the log files in my log directories only ever have alphanumeric characters, dash and period, so works perfectly.

RonJohn
  • 1,015
  • 1
  • 12
  • 23
  • Whether it fails of files with spaces depends on what you have in `$IFS`. `$IFS` also needs to contain the newline character as you rely on split+glob to split `$FileList`. None of the files must be directories, nor *start* with a dash. – Stéphane Chazelas Mar 21 '23 at 06:36