Piped greps for looking inside files

Question

(self-migrated from ask-ubuntu because it's linux-related, not ubuntu, and my os isn't ubuntu)

I'm trying to make a grep that looks like this:

grep -r 2019 | grep -riv FAILED | grep -rl DSL

I want to get filenames (-l) of files containing 2019 in them, AND NOT (-v) containing FAILED AND containing DSL.

Here, only the last grep is executed. I understand it's because of the -r, so each grep greps on all files instead of the previous result. But I can't figure out how to make it work without -r.

Maybe there's another way to use multiple patterns on a grep but with "positive" and "negative" match I haven't found anything.

Answer 1

The last grep in the pipeline would be reading from the previous grep (if it hadn't used the -r option, see later), so it would have no idea from what file the data came from, which in turn means it can't report the pathname of the file.

Instead, consider using find like so:

find . -type f \
    -exec grep -q 2019 {} \; \
    -exec grep -q DSL {} \; \
    ! -exec grep -qi FAILED {} \; \
    -print

This would take each regular file from the current directory and any subdirectory (recursively) and test whether it contains the strings 2019, DSL, and FAILED (case insensitively). It would print the pathnames of file that contain the first two string but that does not contain the third.

If a file does not contain 2019 the other two tests will not be carried out, and if it does not contain DSL, the last test will ont be carried out, etc.

Note that instead of grep -v -qi FAILED I'm using a negation of grep -qi FAILED as the third test. I'm not interested in whether the file contains lines not containing FAILED, I'm interested in whether the file contains FAILED, and in that case I'd like to skip this file.

Related:

• Understanding the -exec option of `find`

---

The issue with your pipeline,

grep -r 2019 | grep -riv FAILED | grep -rl DSL

is that the last grep will look recursively in all the files in the current directory and below and will ignore the input from the previous stages of the pipeline. The two initial grep invocations may produce some data, but they would fail to forward this through the pipeline and will eventually be killed when the last grep is done.

Also, as I already noted above, the middle grep would not find files that does not contain FAILED, it would find files that contain lines with things other than FAILED. Incidentally, it would also ignore the input from the preceding grep.

Answer 2

With GNU grep (-r is already a GNU extension) and GNU xargs or compatible:

grep -rlZ 2019 . |
  xargs -r0 grep -LiZ FAILED |
  xargs -r0 grep -l DSL

You need xargs to be able to pass the list of files output by one grep as arguments to the next grep. And -Z for that list of files to be NUL-delimited. To report the list of files that don't contain FAILED, it's -L (a GNU extension as well), not -vl which report the files that contain at least one line that doesn't match.

That should limit the number of grep invocations to a minimum, and for a large number of files could leverage up to three processors concurrently.