Setup:
Linux-A <-> OPN-A <-> OPN-B <-> Linux-B
Every system is conntected via a Twinax 10G DAC cable. For testing, also Linux-A and Linux-B (both Debian with older Xeons (6-8 years old). OPN-X are newer SuperMicro boards, about 1 or 2 years old. NICs, dmesg and ifconfigs are here:
All systems have a 1G uplink in a mgmt network 10.24.80.0/24 so I dont loose connction. LAN-A is 10.1.0.0/24, LAN-B is 10.2.0.0/24, WAN for OPN is 10.0.0.0/24 where A has .1 and B has .2.
All standard results for my previous tests are here:
When I disable firewall rules with ‚pfctl -d‘ on both firewalls, the iperf gets full 9,4G.
Now I look into the ruleset. Now I create an emty file and do:
‚pfctl -d && pfctl -e empty.txt‘
Now the iperf get’s 8,6Gbit. And then I put this line in empty.txt and load again:
pass in quick from {any} to {any}
The thoughput drops from 8,6Gbit to 8,1Gbit.
To test if everything is correct I also did the iperf from Linux-A to Linux-B with the direct link and there is also 9,4Gbit with single stream. Also when adding iptables rules (INPUT/OUTPUT) the same performance.
With pf disabled and just using ipfw I get 8,1Gibt with this ruleset:
root@Zeus1:~ # ipfw list
00100 allow pfsync from any to any
00110 allow carp from any to any
00120 allow ip from any to any layer2 mac-type 0x0806,0x8035
00130 allow ip from any to any layer2 mac-type 0x888e,0x88c7
00140 allow ip from any to any layer2 mac-type 0x8863,0x8864
00150 deny ip from any to any layer2 not mac-type 0x0800,0x86dd
00200 skipto 60000 ip6 from ::1 to any
00201 skipto 60000 ip4 from 127.0.0.0/8 to any
00202 skipto 60000 ip6 from any to ::1
00203 skipto 60000 ip4 from any to 127.0.0.0/8
06000 skipto 60000 tcp from any to any out
06199 skipto 60000 ip from any to any
60000 return ip from any to any
65533 allow ip from any to any
65534 deny ip from any to any
65535 allow ip from any to any
top -CHIPS with one pf rule
12 root -92 – 0K 672K CPU2 2 9:43 49.35% intr{irq266: ix1:q0}
0 root -92 – 0K 912K – 5 4:17 48.71% kernel{ix1:q0}
top -CHIPS with pf unloaded
12 root -92 – 0K 672K CPU2 2 10:24 81.45% intr{irq266: ix1:q0}
0 root -92 – 0K 912K – 1 1:36 10.48% kernel{ix0:q0}