4

I have a freebsd box, which serves as a router from LAN to the outside world. It has several Internet providers, OpenVPN servers and clients, NAT and so on.

I would like to have complete statistics on when, who, to whom, via which provider, via which protocol, and how many bytes have been sent.

I agree that the minimal time scale will be an hour, i.e. no need for millisecond precision.

I tried vnstat2, ntopng and some other programs, but they don't seemed to have what I need or it is not obvious.

I don't understand, why this feature is not easily available. But I am not good in freebsd, so please, give me some clues.

Rui F Ribeiro
  • 55,929
  • 26
  • 146
  • 227
Dims
  • 3,181
  • 9
  • 49
  • 107

1 Answers1

8

For getting details of network transactions, you have got a implementation of a Netflow generator for FreeBSD or Linux:

ng_netflow

NAME ng_netflow - Cisco's NetFlow implementation

DESCRIPTION The ng_netflow node implements Cisco's NetFlow export protocol on a router running FreeBSD. The ng_netflow node listens for incoming traffic and identifies unique flows in it. Flows are distinguished by endpoint IP addresses, TCP/UDP port numbers, ToS and input interface. Expired flows are exported out of the node in NetFlow version 5/9 UDP datagrams.

As for NetFlow itself:

NetFlow is a network protocol developed by Cisco for collecting IP traffic information and monitoring network traffic. By analyzing flow data, a picture of network traffic flow and volume can be built.

also rfc 3954 - NetFlow Services Export Version 9

For storing the Netflow data you also need what is know as a server collector. It can be either a Linux or a FreeBSD box. It should not be installed on the actual router. One such known implementation is nfsen

NfSen is a graphical web based front end for the nfdump netflow tools.

NfSen allows you to:
- Display your netflow data: Flows, Packets and Bytes using RRD (Round Robin Database).
- Easily navigate through the netflow data.
- Process the netflow data within the specified time span.
- Create history as well as continuous profiles.
- Set alerts, based on various conditions.
- Write your own plugins to process netflow data on a regular interval.

nfsen2

Be aware that, depending on your available bandwidth, generating NetFlows can be taxing on the CPU. A known strategy in some cases is doing a mirror of the switch port of the router, and using another machine for those operations.

After a certain threshold of bandwidth it probably makes more sense going for a professional router if generating NetFlows is a requirement.

As a final alert, having NAT, the NetFlows have to be captured in the inside/LAN interface, as otherwise you will lose the sense of whom is doing what.

I use up around 100GB of data for 5-6 months of traffic, using NfSen collecting NetFlow data from Cisco equipment, your mileage may vary.

Glorfindel
  • 805
  • 2
  • 10
  • 19
Rui F Ribeiro
  • 55,929
  • 26
  • 146
  • 227
  • What is the purpose of requiring to gather information of one machine and then send it to another via network? This loads network additionally. Why don's gather info on router itself? It has big harddrive and CPU. Isn't this CISCO solution better fit for embedded systems? – Dims Dec 04 '16 at 09:14
  • And one more thing. Listening for LAN I will keep information about destination IP, but loose information about used gateway. I would like to keep both. – Dims Dec 04 '16 at 09:16
  • @Dims Netflow is not intended to save information about the gateway, I am afraid, unless you capture it in several interfaces. – Rui F Ribeiro Dec 04 '16 at 10:10
  • 1
    @Dims offloading load basically. You are not sending the full packets, you are just sending summaries of the transactions, the network load per se is negligible. I have a 1Gbps and even when having peak utilisation at home at 100Mbps, I sure would not want that additional load in my router. At the end of the day, actually a proper configured system and low bandwidth it wont add much to the load, but I am not adding a web browser service to a router/firewall configuration of mine anytime soon. From the security point of view is not a good idea. – Rui F Ribeiro Dec 04 '16 at 10:36