Speed up Ipsec : AES-NI Processors?

[onenerdyguy]

I have a site to site VPN that has been working fine through Ipsec for the past 2 years. However, it is slow. Each site is on a dedicated 100/100 connection, yet I can't usually get over 20Mb/s on the ipsec vpn. I know Ipsec has considerable overhead for security, and this should be expected.

My question is, if I was to upgrade both hosts (which are currently E5500 era xeons with 12g of ram) to a newer processor that supported the AES-NI instruction set, would I see a substantial increase in my speed? Mind you, this is of course talking about pure IPSEC speed, I understand how other activity will lower my rates. I'm getting at max 20Mb/s when there is no other activity happening on the devices i.e. I've disconnected the lan and am connecting back and forth directly from the boxes.

[Jim.Alles]

I have not used that configuration, but I would look at NIC manufacturers, drivers, buffers; and maybe a packet capture...

[onenerdyguy]

I have not used that configuration, but I would look at NIC manufacturers, drivers, buffers; and maybe a packet capture...

That was my first thought as well, but no dice so far. They're all on Intel nics, using whatever driver Untangle has detected for them for generations.

I guess I'll need to schedule some downtime, and play with the settings a bit to see if I can increase speed. In our region, a fiber link between our sites is cost prohibitive due to a tariff line that is older than the hills. So site to site is all I have offered from ISPs

[sky-knight]

We've got some hardware that has encryption offloading capabilities. The independent stuff doesn't work, but the server level Intel NICs that have it built in seem to work fine.

That being said, if you aren't showing a server with high CPU or RAM loads, those speeds seem a little broken. Perhaps there's an MTU problem?

I'd open a support ticket and see if support can see anything.

[onenerdyguy]

We've got some hardware that has encryption offloading capabilities. The independent stuff doesn't work, but the server level Intel NICs that have it built in seem to work fine.

That being said, if you aren't showing a server with high CPU or RAM loads, those speeds seem a little broken. Perhaps there's an MTU problem?

I'd open a support ticket and see if support can see anything.

That is my next step, as soon as I find time I can drop it if needed. Thanks for the help guys.

[Jim.Alles]

are they gigabit NICs?

also, this: https://calomel.org/network_performance.html

[onenerdyguy]

are they gigabit NICs?

also, this: https://calomel.org/network_performance.html

They sure are Gigabit. Looking at that now.

[dmor]

I also have sites with 100/100Mbps links with Untangle NG IPSec tunnels interconnecting them. We have seen this same low (but consistently reliable) performance.

Because of this, we also ran some lab tests in my office in the past 2 weeks. The results were quite revealing and dissapointing. The max throughput we were able to get was nearly 30Mbps.

Meanwhile watching top on both UTMs, CPU usage was very low throughout and hardly noticably changed from the time that we would test file transfers, and when the UTMs were totally idle.

This was all done on my workbench with absolutely no outside/ISP potential issues. Simply using direct Ethernet links between the UTM WANs and one computer with an SSD plugged into the LAN side of each UTM.

I have been planning to do a proper write-up with a blog post, etc.

But based on the results, we guessed that there is some code programmed into Untangle NG that intentionally limits the throughput of the IPSec tunnels (since no hardware resources appeared to be a bottle-neck).

As a side note, we also ran a test where the same Untangle NGs were doing direct routing (no NAT enabled anywhere, and static routes added to each UTM for the neighboring UTM's LAN subnet). In those tests our file transfer throughput was consistently wire speed (100Mbps).

So it is not an issue with the NICs not being able to run at full speed. It is an issue with the IPSec implementation (at least that's what it looks like).

We have been planning to repeat these tests with firewalls from another manufacturer so that we can compare the performance. Our thought is that Untangle NG may stop being our tool of choice for site-to-site tunnels. We would still use it where appropriate, but probably switch to a better performing solution for these tunnels.

[onenerdyguy]

I also have sites with 100/100Mbps links with Untangle NG IPSec tunnels interconnecting them. We have seen this same low (but consistently reliable) performance.

Because of this, we also ran some lab tests in my office in the past 2 weeks. The results were quite revealing and dissapointing. The max throughput we were able to get was nearly 30Mbps.

Meanwhile watching top on both UTMs, CPU usage was very low throughout and hardly noticably changed from the time that we would test file transfers, and when the UTMs were totally idle.

This was all done on my workbench with absolutely no outside/ISP potential issues. Simply using direct Ethernet links between the UTM WANs and one computer with an SSD plugged into the LAN side of each UTM.

I have been planning to do a proper write-up with a blog post, etc.

But based on the results, we guessed that there is some code programmed into Untangle NG that intentionally limits the throughput of the IPSec tunnels (since no hardware resources appeared to be a bottle-neck).

As a side note, we also ran a test where the same Untangle NGs were doing direct routing (no NAT enabled anywhere, and static routes added to each UTM for the neighboring UTM's LAN subnet). In those tests our file transfer throughput was consistently wire speed (100Mbps).

So it is not an issue with the NICs not being able to run at full speed. It is an issue with the IPSec implementation (at least that's what it looks like).

We have been planning to repeat these tests with firewalls from another manufacturer so that we can compare the performance. Our thought is that Untangle NG may stop being our tool of choice for site-to-site tunnels. We would still use it where appropriate, but probably switch to a better performing solution for these tunnels.

I will be very interested in your results. I still love Untangle for it's content filtering and bandwidth management capabilities, but if I need to do something else for a site to site, I will. This is critical to my current network setup, and will continue to be going in the future.