How to collapse consecutive numbers into ranges?

Question

Given a sorted input file (or command output) that contains unique numbers, one per line, I would like to collapse all runs of consecutive numbers into ranges such that

n
n+1
...
n+m

becomes

n,n+m

---

input sample:

2
3
9
10
11
12
24
28
29
33

expected output:

2,3
9,12
24
28,29
33

Answer 1

With dc for the mental exercise:

dc -f "$1" -e '
[ q ]sB
z d 0 =B sc sa z sb
[ Sa lb 1 - d sb 0 <Z ]sZ
lZx
[ 1 sk lf 1 =O lk 1 =M ]sS
[ li p c 0 d sk sf ]sO
[ 2 sf lh d sj li 1 + !=O ]sQ
[ li n [,] n lj p c 0 sf ]sM
[ 0 sk lh sj ]sN
[ 1 sk lj lh 1 - =N lk 1 =M ]sR
[ 1 sf lh si ]sP
[ La sh lc 1 - sc lf 2 =R lf 1 =Q lf 0 =P lc 0 !=A ]sA
lAx
lSx
'

Answer 2

awk '
    function output() { print start (prev == start ? "" : ","prev) }
    NR == 1 {start = prev = $1; next}
    $1 > prev+1 {output(); start = $1}
    {prev = $1}
    END {output()}
'

Answer 3

awk, with a different (more C-like) approach:

awk '{ do{ for(s=e=$1; (r=getline)>0 && $1<=e+1; e=$1); print s==e ? s : s","e }while(r>0) }' file

the same thing, even less awk-ward:

awk 'BEGIN{
    for(r=getline; r>0;){
        for(s=e=$1; (r=getline)>0 && $1<=e+1; e=$1);
        print s==e ? s : s","e
    }
    exit -r
}' file

Answer 4

Using Perl substitute with eval (Sorry for the obfuscation...):

perl -0pe 's/(\d+)\n(?=(\d+))/ $1+1==$2 ? "$1," : $& /ge; 
           s/,.*,/,/g' ex

• first substitution creates lines with "," separated consecutive int sequences;
• second substitution, removes middle numbers.

Answer 5

Another awk approach (a variation of glenn's answer):

awk '
    function output() { print start (start != end? ","end : "") }
    end==$0-1 || end==$0 { end=$0; next }
    end!=""{ output() }
    { start=end=$0 }
END{ output() }' infile

Answer 6

Yet another awk solution similar to the other:

#!/usr/bin/awk -f

function output() {
    # This function is called when a completed range needs to be
    # outputted. It will use the global variables rstart and rend.

    if (rend != "")
        print rstart, rend
    else
        print rstart
}

# Output field separator is a comma.
BEGIN { OFS = "," }

# At the start, just set rstart and prev (the previous line's number) to
# the first number, then continue with the next line.
NR == 1 { rstart = prev = $0; next }

# Calculate the difference between this line and the previous. If it's
# 1, move the end of the current range here.
(diff = $0 - prev) == 1 { rend = $0 }

# If the difference is more than one, then we're onto a new range.
# Output the range that we were processing and reset rstart and rend.
diff > 1 {
    output()

    rstart = $0
    rend = ""
   }

# Remember this line's number as prev before moving on to the next line.
{ prev = $0 }

# At the end, output the last range.
END { output() }

The rend variable is not actually needed, but I wanted to keep as much range logic as possible away from the output() function.

Answer 7

An alternative in awk:

<infile sort -nu | awk '
     { l=p=$1 }
     { while ( (r=getline) >= 0 ){
           if ( $1 == p+1 ) { p=$1;  continue };
           print ( l==p ? l : l","p );
           l=p=$1
           if(r==0){ break };
           }
       if (r == -1 ) { print "Unexpected error in reading file"; quit }
     }
    ' 

On one line (no error check):

<infile awk '{l=p=$1}{while((r=getline)>=0){if($1==p+1){p=$1;continue};print(l==p?l:l","p);l=p=$1;if(r==0){ break };}}'

With comments (and pre-processing the file to ensure a sorted, unique list):

<infile sort -nu | awk '

     { l=p=$1 }    ## Only on the first line. The loop will read all lines.

     ## read all lines while there is no error.
     { while ( (r=getline) >= 0 ){

           ## If present line ($1) follows previous line (p), continue.
           if ( $1 == p+1 ) { p=$1;  continue };

           ### Starting a new range ($1>p+1): print the previous range.
           print ( l==p ? l : l","p );

           ## Save values in the variables left (l) and previous (p).
           l=p=$1

           ## At the end of the file, break the loop.
           if(r==0){ break };

           }

       ## All lines have been processed or got an error.
          if (r == -1 ) { print "Unexpected error in reading file"; quit }
     }
    ' 

Answer 8

A nice discussion from 2001 on perlmonks.org, and adapted to read from STDIN or files named on the command line (as Perl is wont to do):

#!/usr/bin/env perl
use strict;
use warnings;
use 5.6.0;  # for (??{ ... })
sub num2range {
  local $_ = join ',' => @_;
  s/(?<!\d)(\d+)(?:,((??{$++1}))(?!\d))+/$1-$+/g;
  tr/-,/,\n/;
  return $_;
}
my @list;
chomp(@list = <>);
my $range = num2range(@list);
print "$range\n";

Answer 9

There's a perl module called Set::IntSpan which already does this (it was originally written in 1996 to collapse lists of article numbers for .newsrc files, which could be enormous).

There is also a similar module for python called intspan, but I haven't used it.

Anyway, with perl and Set::IntSpan (and tr to get the input data into comma-separated format, and tr again to munge the output), this is trivial.

$ tr $'\n' ',' < input.txt  | 
    perl -MSet::IntSpan -lne 'print Set::IntSpan->new($_)' |
    tr ',-' $'\n,'
2,3
9,12
24
28,29
33

Set::IntSpan is packaged for debian and ubuntu as libset-intspan-perl, for fedora as perl-Set-IntSpan, and probably for other distros too. Also available on CPAN, of course.

Answer 10

How about

awk '
$0 > LAST+1     {if (NR > 1)  print (PR != LAST)?"," LAST:""
                 printf "%s", $0
                 PR = $0
                }
                {LAST  = $0
                }
END             {print (PR != LAST)?"," LAST:""
                }
' file
2,3
9,12
24
28,29
33

Answer 11

Perl approach!

#!/bin/perl
    print ranges(2,3,9,10,11,12,24,28,29,33), "\n";

sub ranges {
    my @vals = @_;
    my $first = $vals[0];
    my $last;
    my @list;
    for my $i (0 .. (scalar(@vals)-2)) {
        if (($vals[$i+1] - $vals[$i]) != 1) {
            $last = $vals[$i];
            push @list, ($first == $last) ? $first : "$first,$last";
            $first = $vals[$i+1];
        }
    }
    $last = $vals[-1];
    push @list, ($first == $last) ? $first : "$first,$last";
    return join ("\n", @list);
}

Answer 12

Ugly software tools bash shell code, where file is the input file:

diff -y file <(seq $(head -1 file) $(tail -1 file))  |  cut -f1  | 
sed -En 'H;${x;s/([0-9]+)\n([0-9]+\n)*([0-9]+)/\1,\3/g;s/\n\n+/\n/g;s/^\n//p}'

Or with wdiff:

wdiff -12 file <(seq $(head -1 file) $(tail -1 file) ) | 
sed -En 'H;${x;s/([0-9]+)\n([0-9]+\n)*([0-9]+)/\1,\3/g;s/=+\n\n//g;s/^\n//p}'

---

How these work: Make a gapless sequential list with seq using the first and last numbers in the input file, (because file is already sorted), and diff does most of the work. The sed code is mainly just formatting, and replacing in-between numbers with a comma.

For a related problem, which is the inverse of this one, see: Finding gaps in sequential numbers

Answer 13

On a "Unix & Linux" site, a simple, readable, pure (bash) shell script feels most appropriate to me:

#!/bin/bash

inputfile=./input.txt

unset prev begin
while read num ; do
    if [ "$prev" = "$((num-1))" ] ; then
        prev=$num
    else
        if [ "$begin" ] ; then
            [ "$begin" = "$prev" ] && echo "$prev" || echo "$begin,$prev"
        fi
        begin=$num
        prev=$num
    fi
done < $inputfile