There has been discussion about this before, but it now came to my mind (again).
If you want to do some accounting on Linux based routers, you don't have any
reasonable way of doing so. All you can do is
- capture all packets, do any kind of evaluation later
This is what you can do with nacctd, ULOGD/ulogd, and various other approaches. The problem is, that you collect an incredible amount of data which needs to be processed.
- insert iptables rules, account only what you're really interested in
This requires prior knowledge of exactly what you want to account. You immediately get the results, and it's not possible to do any arbitrary calculation at some later point.
So there is a need for something else: conntrack based accounting. The
idea is: Let connection tracking count how many bytes+packets a connection has.
When the connection terminates, the total amount is sent to some userspace
process. This means you will have one record of accounting data per
connection. In the worst case of extremely short-lived connections, you would
end up with almost as much DMA as in the nacctd approach - but even then,
significantly less processing for the actual accounting itself.
I haven't looked into the details yet, but even generating netflow data should be possible quite easy this way.
As for the implementation, a single set of counters should be sufficient.
Adding per-CPU counters doesn't make sense, since the cache lines of the
conntrack entry have to be valid on the current CPU anyway. We're also already
under ip_conntrack_lock, so writing two more counters per packet shouldn't be
that expensive. Per-CPU counters also don't make sense if they are within the
same cache line...
One set of counters would have to be: bytes for each direction, packets for
each direction. They could be u_int32_t, since almost all connections have
less than 4GB traffic these days.