How to remove lines containing IP address?

Question

I have a file like:

http://example.mx
https://test.com
http://4.3.4.4
http://dev.somedomain.com
http://1.3.4.2

I want to get rid of the ones with IPs so that the result should be:

http://example.mx
https://test.com
http://dev.somedomain.com

What I did is:

cat  in.log |  sed  '/^http:\/\/[0-9]/d'  > out.log

But it does not work.

Any specific reason to use `cat in.log | ... ` and not e.g. ` ... | cat > out.log` or `.... | cat | cat | cat | cat > out.log`. `cat` is not necessary here, you can use `< in.log` and with `sed` you can also specify the input file as a commandline argument ( `sed 's//^http:\/\/[0-9]/d' in.log > out.log` ) — Anthon, Dec 25 '16 at 16:24

score 7 · Accepted Answer · answered Dec 25 '16 at 10:55

7

But it does work. The correct answer appears to be already in your question.

$ cat in.log
http://example.mx
https://test.com
http://4.3.4.4
http://dev.somedomain.com
http://1.3.4.2
$ cat  in.log |  sed  '/^http:\/\/[0-9]/d'  > out.log
$ cat out.log
http://example.mx
https://test.com
http://dev.somedomain.com
$

answered Dec 25 '16 at 10:55

steve

21,582
5
48
75

1

Oh my bad. I had an unseen space at the begining of each line on original file! – supermario Dec 25 '16 at 11:06
1

It is bad advice to create an extra process with `cat` and pip the output thereof to `sed`. You can specify the input file to `sed` on the commandline: `sed '/^http:\/\/[0-9]/d' in.log > out.log` and even if that was not the case you should not use `cat`, but use `< in.log sed '/^http:\/\/[0-9]/d' in.log > out.log` – Anthon Dec 25 '16 at 14:54
sure, agree. was keeping with the approach suggested by the OP, which was perfectly functional, albeit used a 'useless cat' etc. – steve Dec 25 '16 at 14:57

Konrad Rudolph · Answer 2 · 2016-12-25T13:59:33.127

3

It seems like you want to remove lines in your input. In addition to sed, there’s a very simple solution using grep -v:

grep -v 'http://\([0-9]\+\.\)\{3\}[0-9]\+' in.log

Furthermore, there’s no need for cat and the pipe here either way: cat foo | bar can be replaced by bar < foo in general, and in this particular instance you can just pass the filename as an argument to either grep or sed without the need for the pipe.

Shell purists will generally frown on this use of cat because it’s unnecessarily starting a separate process (and because cat’s purpose is to concatenate files …).

edited Dec 25 '16 at 13:59

answered Dec 25 '16 at 13:06

Konrad Rudolph

3,689
3
23
29

1

removing line based on simple pattern can be made by either sed, grep, perl or awk. – Archemar Dec 25 '16 at 13:43
@Archemar Yeah, I’m going to change the wording. In fact I wanted to show an alternative, not The One True Solution. – Konrad Rudolph Dec 25 '16 at 13:58
I actually love using `grep -v` with regex – MaXi32 Oct 08 '21 at 07:52

score 2 · Answer 3 · answered Dec 25 '16 at 10:56

2

If you want to remove the lines from in.log, you can use the -i flag of sed:

sed -i '/^http:\/\/[0-9]/d' in.log

answered Dec 25 '16 at 10:56

janos

11,171
3
35
53

Archemar · Answer 4 · 2016-12-25T11:12:17.600

2

this already work, however /^http:\/\/[0-9]/ will match

http://1.hello.word

to match "full IP" use /^http:\/\/[0-9\.]*$/

where

[0-9\.] match digit and dot ...
* ... 0 or more time ..
$ ... till end of line.

edited Dec 25 '16 at 11:12

answered Dec 25 '16 at 11:01

Archemar

31,183
18
69
104

How to remove lines containing IP address?

4 Answers4