Generating and capturing Netflow on a Linux router

We currently have a dual-NIC Ubuntu server at our data centre acting as the gateway router between our public networks and our ISP. We have a /30 cross connect network on the ISP-facing NIC, and one IP from each of the three networks attached to our interior-facing NIC.

I would like to configure network traffic statistic generation and collection on this server, using Cisco's Netflow protocols. This will allow me to confirm our ISP's billing, as well as break-down data flow within our network.

What tools or packages would you recommend to passively capture traffic statistics and record them for later processing? Extra points if the Netflow collector has a MySQL data-store connector.

From serverfault mlambie

• nprobe netflow generator

  And I personally use flow-tools to store flows on disk, generate reports.

  Regards K

  Edit: here are many more tools for logging to mysql, charting, etc.

  From Khb

• I know argus can read and process netflow data and it is quite good at collecting and processing network flow data by itself.

  I've never used it to create netflow data as I usually just use it to collect and process the data natively, or use it to take a variety of flow types (tcpdump captures, netflows, etc) and use argus for the aggregation and summarization and analysis.

  monomyth : I used argus to generate netflows for tapped/mirrored ports, flows were send to flow-tools for processing. It worked great, I vaguely remember some data missing from flows, but nothing I needed.
  From chris

• I suggest you looking at argus, as chris says. From my experience it's the best behaving flow collector. But there are good alternatives like flowd and pfflowd that might work for you too. If you have any decent load (terrabytes per day) forget about storing your flows in any SQL database :) oh, and yes, flow-tools are great once you learn all the filtering magic and such.

  From monomyth

• For generating a tool like nprobe or fprobe will work fine as others have mentioned.

  For collecting I like nfdump/nfsen. It doesn't use mysql, but it is really easy to work with and get data out of it in a machine readable form.

  You probably don't want the full netflow data in mysql, instead it usually makes more sense to run an aggregation query and load the summary into mysql. Having 10,000,000 records in mysql is not going to be fun to work with, but inserting a daily or hourly summary of (ip,total flows,total bytes,total packets) works a lot better.

  David Mackintosh : +1 for SQL summaries. I store (ip, flows, in bytes, outbytes) every five minutes and the database is reasonably usable. I tried dumping netflow data into a database, and found it storage-intensive and incredibly slow to access. For specific flow information, linear searches through netflow files turned out to be faster in all cases (of the scenarios we tried anyways).
  From Justin