How to awk and printf columns using formatting and including all columns after 9th

Question

I have the below script to print all specified columns from the 1st to the 9th using printf to format, however I would like to print any columns thereafter the 9th column and I am unsure of how to do so

pwd=`pwd`
#Megabytes
max_size_1=`ls -ltr "$pwd" | grep -v "total" | awk '{print $1}' | sort -n | tail -1 | wc -m`
max_size_2=`ls -ltr "$pwd" | grep -v "total" | awk '{print $2}' | sort -n | tail -1 | wc -m`
max_size_3=`ls -ltr "$pwd" | grep -v "total" | awk '{print $3}' | sort -n | tail -1 | wc -m`
max_size_4=`ls -ltr "$pwd" | grep -v "total" | awk '{print $4}' | sort -n | tail -1 | wc -m`
max_size_5=`ls -ltr "$pwd" | grep -v "total" | awk '{$5=sprintf("%.0f M", $5/1024^2)} 1' | awk '{print $5}' | sort -n | tail -1 | wc -m`


ls -ltr "$pwd" | grep -v "total" | awk '{$5=sprintf("%.0f M", $5/1024^2)} 1' | eval "awk '{printf \"%$max_size_1-s %$max_size_2-s %$max_size_3-s %$max_size_4-s %$max_size_5-s %-3s %-3s %-3s %-6s\\n\", \$1, \$2, \$3, \$4, \$5, \$6, \$7, \$8, \$9}'"

Output is:

-rwxr-xr-x  1  informix  informix  0  M   Mar 1   13:45 
-rwxr-xr-x  1  informix  informix  0  M   Mar 1   13:45 
-rwxr-xr-x  1  informix  informix  0  M   Mar 1   13:46 
-rwxr-xr-x  1  informix  informix  0  M   Mar 9   10:51 
-rw-r-----  1  informix  informix  0  M   Mar 9   12:36 
-rwxrwxrwx  1  informix  informix  0  M   Mar 9   13:01

What I would like:

-rwxr-xr-x  1  informix  informix  0  M   Mar 1   13:45 ls-ltrg
-rwxr-xr-x  1  informix  informix  0  M   Mar 1   13:45 ls-ltrk
-rwxr-xr-x  1  informix  informix  0  M   Mar 1   13:46 ls-ltrb
-rwxr-xr-x  1  informix  informix  0  M   Mar 9   10:51 ls-ltrm
-rw-r-----  1  informix  informix  0  M   Mar 9   12:36 split word
-rwxrwxrwx  1  informix  informix  0  M   Mar 9   13:01 test.sh

It should print all columns after nine even files or names that are split thereafter.

if you could better describe your problem instead of just code, perhaps there might be simple solution other than having to parse `ls`... using `find` can filter files based on size, and then the filtered files can be passed on to another command/script.... — Sundeep, Mar 09 '17 at 12:05

score 0 · Accepted Answer · edited Apr 13 '17 at 12:37

I found this to be an interesting problem to tackle, mainly as a learning opportunity in awk.

Parsing ls robustly is tricky and not recommended, so I won't go over those details here. If your filenames never contain any newlines, then you can use the following awk script to do all of the work that you had been doing in separate pieces:

grepping out the "total" line
getting the length of each field
dynamically formatting the columns

... all without calling ls, grep, awk, sort, tail, and wc multiple times, and without needing to use eval.

Since ls lists the current directory (pwd) by default, I also removed that.

A couple random notes:

my awk didn't initialize i at first use to zero, so I force it to zero in the BEGIN block
the grep -v total part happens with the one line: /^total / { next; }
the actual solution to your actual question is in the for loop where we consolidate the filename information in columns 9 .. NF.
the formatting comes in two pieces: the first for loop in the END block determines the width that we need; the second for loop uses the * format specifier to do the real work -- by passing in the desired width ahead of the actual value.

Here's the resulting script/code:

ls -ltr | awk '
        BEGIN {
                i=0;
        }

        /^total / { next; }
        {
                perms[i]=$1
                links[i]=$2
                owner[i]=$3
                group[i]=$4
                $5=sprintf("%.0f M", $5/1024^2)
                size[i]=$5
                date[i]=$6 " " $7 " " $8
                for(x=9; x <= NF; x++) name[i] = name[i] " " $x
                i++
        }

        END {
                for(j=0;j<i;j++) {
                        if (length(perms[j]) > maxperms) maxperms = length(perms[j])
                        if (length(links[j]) > maxlinks) maxlinks = length(links[j])
                        if (length(owner[j]) > maxowner) maxowner = length(owner[j])
                        if (length(group[j]) > maxgroup) maxgroup = length(group[j])
                        if (length(size[j]) > maxsize) maxsize = length(size[j])
                }
                for(j=0;j<i;j++) {
                        printf "%-*s %-*s %-*s %-*s %-*s %s %s\n",
                                maxperms, perms[j],
                                maxlinks, links[j],
                                maxowner, owner[j],
                                maxgroup, group[j],
                                maxsize, size[j],
                                date[j],
                                name[j]
                }
        }
        '

A huge thanks to G-Man, who provided a ready-made cheat sheet for me to use for awk's dynamic column formatting.

If you increment `i` _before_ storing data you don't need init to zero, and in fact you could just use `NR-1` but that's kind of cheating. Also real date-part fields vary in width, although sample in Q didn't show it and OP didn't deal with it; and FWIW this incorrectly handles a filename containing whitespace other than single SP, which people rarely do but is possible. — dave_thompson_085, Mar 10 '17 at 22:47
I had to make a few alterations to the above, but this is perfect for what I need to do. — Christopher Karsten, Mar 13 '17 at 06:28
Happy to hear it helped, @Christopher! If you think the alterations matter, feel free to edit my answer! — Jeff Schaller, Mar 13 '17 at 10:29

How to awk and printf columns using formatting and including all columns after 9th

1 Answers1