0

In this question, a solution was found to automatically send an email each time a specific URL is visited (such as ?src=foo), as seen in the Apache logs:

tail -F -n0 /var/log/apache2/other_vhosts_access.log | grep --line-buffered "?src=foo" | \
        { while IFS= read -r line; do echo "$line" | mail [email protected]; done } &

But the problem is that any other ressource being loaded on this page (images, favicon, etc.) have this pattern ?src=foo in the referrer column too:

www.example.fr:80 111.111.111.111 - - [12/Sep/2017:17:30:32 +0200] "GET /images/P26372.jpg 
  HTTP/1.1" 200 86276 "http://example.fr/?src=foo" "Mozilla/5.0"

Then I'm receiving 15 emails in a row with this method.

How to limit to do a "digest" of this?

I thought about limiting the search for the pattern inside "GET ... HTTP" so it would exclude a result in the referred column, but how to do this?

Or, alternatively, would it be possible to send only an email twice per day with all the results of grep? (no email sent is nothing is found)

Basj
  • 2,351
  • 9
  • 37
  • 70
  • Sending an email when an URL is visited is a really bad idea (what would happen if I `curl`ed your URL in a tight loop?). Just use a log analyzer for Apache. – Satō Katsura Sep 15 '17 at 14:08
  • @SatōKatsura I use it *only in a very narrow specific case* for being informed quickly that someone visited the link I sent via email. Example: you send `example.com/superimportantfile.pdf?src=JH2KS79x0v9t8` (with an impossible-to-guess pattern) to someone and you want to be informed by email. – Basj Sep 15 '17 at 14:11
  • Again: would you mind if I `curl`ed that page in a tight loop? – Satō Katsura Sep 15 '17 at 14:14
  • @SatōKatsura for this you need to have access to the pattern `JH2KS79x0v9t8` to trigger an email. And this pattern is only communicated to one person that I know. – Basj Sep 15 '17 at 14:23
  • Security by obscurity. What could possibly go wrong with that. :) – Satō Katsura Sep 15 '17 at 14:24
  • I learned something cool thanks to your comment @SatōKatsura, thanks! https://en.wikipedia.org/wiki/Security_through_obscurity – Basj Sep 15 '17 at 14:46
  • Can you provide some example lines which should and should not trigger the email sending? Do not forget, you can always add new layers of `grep` pattern searches through pipes :-) You said in the question, alternatively, would it be possible to send an email twice a day but in the comments that this is for to be quickly informed. If you get emails in predetermined intervals, it will not be as quick as I think you want it to be. But I would redirect the output of the command/script to a file and check it periodically if it's empty. If you want it I'll write my idea more briefly. – Wax Sep 15 '17 at 15:08

1 Answers1

0

There are log scanners that offer rate limits on the noise they generate; I'd use sec.pl and then configuration along the lines of

type=SingleWithThreshold
ptype=SubStr
pattern=?src=foo
desc=specific-url-thing
action=pipe 'that url thing happened' /usr/bin/mail -s url-thing [email protected]
window=86400
thresh=1

For a daemon instance watching the log in question. This should trigger right away, and then shut up for a day; there are other options to aggregate the hits and so forth, see the manual for details.

thrig
  • 34,333
  • 3
  • 63
  • 84