Get public IPs of accessed webpages?

Question

I would like to be able to get the public IPs of the websites I am accessing with my PC in a way such as:

www.google.es - public IP1 
www.cdn.facebook.com - public IP2

and so on. I think this should be done by logging DNS traffic, so I tried using wireshark as part of a solution I found in another answer:

tshark -f "udp port 53" -Y "dns.qry.type == A and dns.flags.response == 0"

However this seems to only show connections between my router and my machine, the list is full of pairs such as:

192.168.200.250 -> 192.168.200.1
192.168.200.1 -> 192.168.200.250`

do you want get public IPs for specific sites or all your traffic ? — Wissam Roujoulah, Nov 28 '16 at 09:54
all my traffic, as my goal is to create somewhat of a database — Ruben, Nov 28 '16 at 09:55
If you have a consumer/home ISP provided router it is probably set as your DNS server to optimize network performance, so DNS requests and responses travel at the IP level between your machine and the router. Even if you used another DNS server that server still would never be the destination system(s). You want to look at the name and address(es) _in the body of the DNS response_, but you are excluding the responses with `flags.response==0`. Instead select `dns.flags.response==1` and add `dns.flags.rcode==0` to ignore responses that don't actually contain a result. — dave_thompson_085, Nov 28 '16 at 13:38

Wissam Roujoulah · Accepted Answer · 2016-11-28T10:15:20.277

2

You can install DNSmasq locally and add this option to the conf file log-facility=/var/log/dnsmasq.log log-queries then set your system to use 127.0.0.1 or ::1 as the DNS resolver its work for me.

Then extract data as any format you want and do what ever you want with it

or install Bind locally. Most distros default install of Bind will be non-autoritative caching-only and add a logging {} config block (as described in the Bind 9 Configuration Reference).

edited Nov 28 '16 at 10:15

answered Nov 28 '16 at 10:00

Wissam Roujoulah

3,204
1
12
21

1

Thanks for your answer, finally I installed dnsmasq and as you said, everything was in the log file, just needed to filter for replies! Many thanks! – Ruben Nov 28 '16 at 13:09

roaima · Answer 2 · 2016-11-28T10:16:15.030

This little script may provide the results you're looking for. I've avoided DNS lookups, instead preferring to use actual HTTP requests (ports 80/http and 443/https).

tshark -nlp -f '(port 80 or port 443) and (tcp[tcpflags] & (tcp-syn|tcp-ack)) == (tcp-syn|tcp-ack)' 2>/dev/null |
    stdbuf -oL awk '{print $3}' |
    while IFS= read -r ip
    do
        name=$(dig +short -x "$ip")
        printf "%-16s%s\n" "$ip" "${name:-$ip}"
    done |
    uniq

Example output

212.58.244.27   bbc-vip146.telhc.bbc.co.uk.
78.129.164.123  free.hands.com.
195.20.242.89   195.20.242.89

This code will generate results only for HTTP requests, whereas searching on DNS queries will find anything and everything. However, be aware that it generates the names from a rDNS lookup on the IP address, so there is not always a direct correspondence between the HTTP hostname you accessed and the name returned in the results.

thanks a lot, this seems to be working. I am going to try to install dnsmasq as suggested in another answer to try and complement what the script cannot identify. Thanks again for your time! — Ruben, Nov 28 '16 at 10:22
I did upvote both answers, however as in this stackexchange my reputation is too low the votes are not publicly shown. — Ruben, Nov 28 '16 at 11:43

Get public IPs of accessed webpages?

2 Answers2