bash remove common lines from two files

Question

I have two files, (no blank lines/Spaces/Tabs)

/tmp/all

aa  
bb  
cc  
hello  
SearchText.json  
xyz.txt

/tmp/required

SearchText.json

and the end output I want is : (all uncommon lines from /tmp/all)

aa  
bb  
cc  
hello  
xyz.txt

I have tried below commands :-

# comm -23 /tmp/required /tmp/all

SearchText.json

# comm -23 /tmp/all /tmp/required

aa  
bb  
cc
hello  
SearchText.json  
xyz.txt

# comm -13 /tmp/all /tmp/required

SearchText.json

# comm -13 /tmp/required /tmp/all

aa  
bb  
cc  
hello  
SearchText.json  
xyz.txt

# grep -vf /tmp/all /tmp/required

# grep -vf /tmp/required /tmp/all

aa  
bb  
cc  
hello  
SearchText.json  
xyz.txt

# comm -23 <(sort /tmp/all) <(sort /tmp/required)

aa  
bb  
cc  
hello  
SearchText.json  
xyz.txt

while updating the formatting on your post, I noticed some trailing spaces on some of the lines; are there trailing spaces on any lines of your actual files? — Jeff Schaller, Mar 18 '19 at 13:10
@JeffSchaller in actual file there are no trailing spaces or tabs — Girish, Mar 19 '19 at 03:46

score 6 · Accepted Answer · edited Apr 12 '21 at 05:35

As an alternative to comm, consider grep:

grep -vxFf /tmp/required /tmp/all

This asks for the lines in /tmp/all that do not (-v) exist in the file (-f) /tmp/required. To avoid interpreting any line in /tmp/all as a regular expression, I added the "fixed strings" -F flag. In addition, we want to force the entire line in /tmp/all to match the one(s) from /tmp/required, so we use the -x option.

This method does not require sorted input.

I suspect that your comm -23 <(sort ...) <(sort ...) command would work, if the "SearchText.json" line matched exactly in both files (same amount of trailing spaces, if any).

bash remove common lines from two files

/tmp/all

/tmp/required

1 Answers1