FortiGate VPN Speedtests

Triggered by a customer who had problems getting enough speed through an IPsec site-to-site VPN tunnel between FortiGate firewalls I decided to test different encryption/hashing algorithms to verify the network throughput. I used two FortiWiFi 90D firewalls that have an official IPsec VPN throughput of 1 Gbps. Using Iperf I measured the transfer rates with no VPN tunnel as well as with different IPsec proposals.

I first ran into really slow performances which were related to the default “Software Switch” on the FortiGate. After deleting this type of logical switch, the VPN throughput was almost as expected.

Lab

My lab consists of the following components:

FortiGate VPN Speedtests Labor

Both FortiWiFi 90D firewalls had the firmware version v5.2.5, build701. The two notebooks were booted with Knoppix 7.6.1 and used Iperf version 2.0.5. The “left” machine ran as the server with either:

while the “right” machine started Iperf with the following commands for different TCP and UDP tests:

 

I tested the throughput without a VPN at all (only routing) and with a few different proposals (see table below). The Diffie-Hellman group for PFS was always set to 14. This is not related to the test results because it is only used for the key establishment and not for the actual symmetric encryption of the traffic.

I also switched the offloading of encryption to “enable” (refer to the Hardware Acceleration Guide), which did not change anything, either.

 

Furthermore, I tested the differences between a normal TCP test and the manual set of the TCP window size and buffer length with “-w 512k -l 512k”, such as shown here or here. But this made no differences, too, since Knoppix Linux seems to auto set the window size pretty optimal.

Results

These are the results. The first four tests are without a VPN. While the first two are without routing (simply plugged in both clients into the same software switch on the FortiGate), tests 3 & 4 are routed through the FortiGates. This was the first time at which I was really shocked about the bad performance of only 180 Mbit/s routing speed. Furthermore, almost all IPsec proposals ran at a speed of 86 MBit/s, which is only 9 % of the IPsec throughput listed in the data sheet.

ProposalsTCP
Tx/Rx
[MBit/s]
TCP
Tx/Rx
[MBit/s]
UDP
Tx/Rx
[MBit/s]
IPerf Options-r-r -P 8-u -r -b 1000M
Same Software Switch
H - FGSW - H
942/937941/936807/805
Same Software Switch
+ Hardware Switch
H - FGSW - SW - H
942/936941/936807/804
No VPN, only Routing
FortiGate directly
H - FG - FG - H
155/177151/168211/206
No VPN, only Routing
H - FG - SW - FG - H
155/177152/168211/210
DES-MD586/8683/8293/94
3DES-MD586/8683/8393/94
3DES-SHA186/8683/8395/94
AES128-SHA186/8683/8388/87
AES256-SHA25686/86122/13393/93
AES256-SHA51285/8580/8084/92

The software switch was the problem!

After hours of investigating the slow VPN speed results, I tested the VPN without the software switch on the network ports side, which led to the following results (first column with a “Hardware Switch”, second column with a single interface):

ProposalsHardware Switch
TCP
Tx/Rx
[MBit/s]
Single Interface
TCP
Tx/Rx
[MBit/s]
Iperf Options-r-r
No VPN, only Routing
H - FG - SW - FG - H
937/937933/932
DES-MD5852/840845/839
3DES-SHA1707/642701/634
AES128-SHA1825/835826/830
AES256-SHA1820/830816/825
AES256-SHA256723/819814/825
AES256-SHA512637/808812/810

Now the speed was quite acceptable, for the mere routing as well as for the VPN throughput. 940 MBit/s for routing through both FortiGate is almost realistic for TCP, and about 830 MBit/s for VPN encryption/decryption is realistic, too.

Here are the “single interface” results in a graph. Only the 3DES tests are a bit slower than all the other ones:

Conclusion

Well, it was my fault that I left the default software switch in place. I should have know better. However, it was the default setting on this FortiWiFi devices.

At the end, the VPN throughput between those FortiGates was really acceptable.

6 thoughts on “FortiGate VPN Speedtests

  1. Well I am using Fortigate 60D,I require a switch. In my case How to improve the IPsec vpn speed.

  2. I’ve a Fortigate 100E in the main site, with a 1000/1000 Mbit/s connection.
    I’ve configured 2 IPSEC VPN site to site, with 2 remote offices, both with a 50/50 Mbit/s connection and 2 Fortigate 61E.

    The maximum transfer speed I got is about 1Mbit/s

    Could you give me any advice on how to improve this unacceptable speed?

    Thanks

    Maurizio

    1. HI Maurizio,

      uh, I am sorry, but I have no idea why you are getting this bad results. You must troubleshoot several things:
      – Have you tried simple routing via the Internet without VPN? What is the throughput then? (Have a look at the interface stats, do you have Layer 2 errors?)
      – Have you tried other IPsec VPNs to other firewalls/offices?
      – Try to debug the VPN. Rekeying? Any errors there?
      – Are there any (new) CPU offloading features on either FortiGate you are using?
      – Use a packet capture on both sides (trust and untrust) to see the packets. Analyze it with Wireshark to see if there are any strange behaviours. Latency? Jitter?
      – Open a ticket at https://support.fortinet.com/
      – And if all does not help: Buy a Palo Alto Networks firewall 😉

      Ciao, Johannes

  3. Hi Maurizio,

    I faced the same problem (low performance with FGT 61E) today but I think I solved the problem.
    First, lower down the TCP MSS to 1400 (set tcp-mss on the interface interface).
    Second, for the IPsec Phase 2, use 3DES-SHA1. Don’t use SHA256-SHA256 because the NPU of the FGT61E is able to handle it.

    Hope it can help you.

    Regards,

    HA

  4. Dear all, thanks for the replies.

    I’ve set tcp-mss on both wan intefaces (100E and 61E) to 1400, but I still see mtu = 1438 on my diag vpn tunnel list:

    name=VPNXXX ver=1 serial=2 aaa.aaa.aaa.aaa:0->bbb.bbb.bbb.bbb:0
    bound_if=5 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/8 options[0008]=npu
    proxyid_num=1 child_num=0 refcnt=158 ilast=3 olast=3 auto-discovery=0
    stat: rxp=1015915 txp=595452 rxb=747165290 txb=371688797
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=966
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=VPN_XXX proto=0 sa=1 ref=137 serial=1
    src: 0:192.168.0.0/255.255.255.0:0
    dst: 0:192.168.43.0/255.255.255.0:0
    SA: ref=6 options=2e type=00 soft=0 mtu=1438 expire=11939/0B replaywin=1024 seqno=69a69 esn=0 replaywin_lastseq=000d4fc0
    life: type=01 bytes=0/0 timeout=43172/43200
    dec: spi=1c1c078e esp=aes key=16 xxxxxxxxxxxxxxxxxxxxxxxxxxxx
    ah=sha1 key=20 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    enc: spi=339e134c esp=aes key=16 xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    ah=sha1 key=20 xxxxxxxxxxxxxxxxxxxxxxxxxxx
    dec:pkts/bytes=872385/698105906, enc:pkts/bytes=449037/241204129
    npu_flag=03 npu_rgwy=80.80.250.160 npu_lgwy=aaa.aaa.aaa.aaa npu_selid=0 dec_npuid=1 enc_npuid=1

  5. I’ve not understand if I’ve to set the tcp-mss on the vpn interface, or directly on the wan interface used by the vpn.

Leave a Reply

Your email address will not be published. Required fields are marked *