Doing lots of benchmarks / tuning / profiling lately

During the last weeks I've been working on tuning/benchmarking/profiling the Sun V20z dual Opteron boxes for high-speed packet filtering purpose.

Some of my findings:

• i386 kernels give you higher pps than x86_64 (because sk_buff is smaller)
• e1000 are way faster than tg3 boards (could be hardware or driver issue)
• Intel PRO/1000MT Quad e1000 boards suck (apparently problems with the onboard PCI-X bridge)
• Connection Tracking performance is not that bad...
• ip_tables performance sucks, even if the ruleset is empty ?!?
• 2.4.x has slightly worse results than 2.6.x if you use IRQ affinity, but really sucks if you don't, since the kernel doesn't balance IRQ's by itself (and irqbalance daemon only balances every 10 seconds)
• You can route up to 1Mpps at 64bytes packet size
• ip_conntrack and iptable_filter at suck at least 300kpps, giving 700kpps as a result

Expect a more detailed report within the next weeks.