How to split a file based on context?

Question

I have some files that contain the results of the lldpneighbors command from all our servers. I would like to split these files into individual files for each server in order to make it easier to import this data into our inventory system.

Sample Input

=== Output from 00000000-0000-0000-0000-000000000000 (SERVERNAME1):
Interface 'ixgbe0' has 1 LLDP Neighbors: 
Neighbor 1:
    Chassis ID:         MAC Address - 00 01 02 03 04 05 
    Port ID:        Interface Name - TenGigabitEthernet 0/6
    Time To Live:       120 seconds
    System Name:        name-of-switch-01
    End Of LLDPDU:  
Interface 'igb0' has 1 LLDP Neighbors: 
Neighbor 1:
    Chassis ID:         MAC Address - 00 01 02 03 04 05 
    Port ID:        Interface Name - TenGigabitEthernet 0/23
    Time To Live:       120 seconds
    System Name:        name-of-switch-02
    End Of LLDPDU:  
=== Output from 00000000-0000-0000-0000-000000000000 (SERVERNAME2):
Interface 'ixgbe0' has 1 LLDP Neighbors: 
Neighbor 1:
    Chassis ID:         MAC Address - 00 01 02 03 04 05 
    Port ID:        Interface Name - TenGigabitEthernet 0/2
    Time To Live:       120 seconds
    System Name:        name-of-switch-01
    End Of LLDPDU:  
Interface 'igb0' has 1 LLDP Neighbors: 
Neighbor 1:
    Chassis ID:         MAC Address - 00 01 02 03 04 05 
    Port ID:        Interface Name - TenGigabitEthernet 0/19
    Time To Live:       120 seconds
    System Name:        name-of-switch-02
    End Of LLDPDU:

This is roughly what all the results look like with some variation(They are not all the same length, some are several lines longer because of more interfaces). The delimiting string I would like to match on is:

=== Output from [UUID] ([HOSTNAME]):

Ideally I would like each file to be named the hostname(this would just be convenience and is not necessary), so above results would be split into files like:

SERVERNAME1

=== Output from 00000000-0000-0000-0000-000000000000 (SERVERNAME1):
Interface 'ixgbe0' has 1 LLDP Neighbors: 
Neighbor 1:
    Chassis ID:         MAC Address - 00 01 02 03 04 05 
    Port ID:        Interface Name - TenGigabitEthernet 0/6
    Time To Live:       120 seconds
    System Name:        name-of-switch-01
    End Of LLDPDU:  
Interface 'igb0' has 1 LLDP Neighbors: 
Neighbor 1:
    Chassis ID:         MAC Address - 00 01 02 03 04 05 
    Port ID:        Interface Name - TenGigabitEthernet 0/23
    Time To Live:       120 seconds
    System Name:        name-of-switch-02
    End Of LLDPDU:

SERVERNAME2

=== Output from 00000000-0000-0000-0000-000000000000 (SERVERNAME2):
Interface 'ixgbe0' has 1 LLDP Neighbors: 
Neighbor 1:
    Chassis ID:         MAC Address - 00 01 02 03 04 05 
    Port ID:        Interface Name - TenGigabitEthernet 0/2
    Time To Live:       120 seconds
    System Name:        name-of-switch-01
    End Of LLDPDU:  
Interface 'igb0' has 1 LLDP Neighbors: 
Neighbor 1:
    Chassis ID:         MAC Address - 00 01 02 03 04 05 
    Port ID:        Interface Name - TenGigabitEthernet 0/19
    Time To Live:       120 seconds
    System Name:        name-of-switch-02
    End Of LLDPDU:

I'm trying to use csplit to accomplish this but I'm not able to match the regex for some reason. The commands I've tried:

$ csplit jbutryn_us-west-a_neighbors %===.*:% '{20}'
csplit: ===.*:: no match

$ csplit jbutryn_us-west-a_neighbors /===.*:/ '{20}'
552
552
552
csplit: ===.*:: no match

$ csplit jbutryn_us-west-a_neighbors '/===.*:/' '{20}'
552
552
552
csplit: ===.*:: no match

$ csplit -ks -f test jbutryn_us-west-a_neighbors '/===.*:/' '{20}'
csplit: ===.*:: no match

Any suggestions?

RomanPerekhrest · Accepted Answer · 2017-08-24T16:41:26.780

3

awk solution:

awk '/^===/{ fn=substr($NF,2,length($NF)-3) }{ print > fn }' file

Each file will be named according to hostname (SERVERNAME<number>)

/^===/ - on encountering line starting with ===
fn=substr($NF,2,length($NF)-3) - constructing filename fn, substr($NF,2,length($NF)-3) - will extract hostname ignoring parentheses around it ($NF - last field)
print > fn - print underlying line into file

edited Aug 24 '17 at 16:41

answered Aug 24 '17 at 16:28

RomanPerekhrest

29,703
3
43
67

TY Sir. Works perfectly. – jesse_b Aug 24 '17 at 16:39

score 1 · Answer 2 · answered Aug 24 '17 at 16:23

1

Don't over-specify the regular expression:

$ csplit logfile '/^===/'

This creates xx00 for the first section and xx01 for the second.

Alternatively, if you have other lines starting with === that you don't want to split on:

$ csplit logfile '/^=== Output from/'

To replace xx with another fixed string, use -p (the string can't be taken from the input data unfortunately).

answered Aug 24 '17 at 16:23

Kusalananda

320,670
36
633
936

So this is only splitting the first occurrence of "=== Output from...". I believe it's not liking my repetition count. – jesse_b Aug 24 '17 at 16:34
@Jesse_b I didn't need a repetition count for the data that you posted. – Kusalananda Aug 24 '17 at 16:35
Yea but that is just a short sample as my input files are thousands of lines long I didn't want to make my question too long. – jesse_b Aug 24 '17 at 16:36

How to split a file based on context?

2 Answers2