There has been discussion about this before, but it now came to my mind (again).
If you want to do some accounting on Linux based routers, you don't have any reasonable way of doing so. All you can do is
- capture all packets, do any kind of evaluation later This is what you can do with nacctd, ULOGD/ulogd, and various other approaches. The problem is, that you collect an incredible amount of data which needs to be processed.
- insert iptables rules, account only what you're really interested in This requires prior knowledge of exactly what you want to account. You immediately get the results, and it's not possible to do any arbitrary calculation at some later point.
So there is a need for something else: conntrack based accounting. The idea is: Let connection tracking count how many bytes+packets a connection has. When the connection terminates, the total amount is sent to some userspace process. This means you will have one record of accounting data per connection. In the worst case of extremely short-lived connections, you would end up with almost as much DMA as in the nacctd approach - but even then, significantly less processing for the actual accounting itself.
I haven't looked into the details yet, but even generating netflow data should be possible quite easy this way.
As for the implementation, a single set of counters should be sufficient. Adding per-CPU counters doesn't make sense, since the cache lines of the conntrack entry have to be valid on the current CPU anyway. We're also already under ip_conntrack_lock, so writing two more counters per packet shouldn't be that expensive. Per-CPU counters also don't make sense if they are within the same cache line...
One set of counters would have to be: bytes for each direction, packets for each direction. They could be u_int32_t, since almost all connections have less than 4GB traffic these days.