19

I am trying to compare the performance of several network security protocols between two hosts connected in Gigabit Ethernet.

My goal here is to see if I can saturate my bandwidth, and if not, what is the limiting factor.

  • with SSL, I can reach 981 MBit/s, so the Ethernet link is obviously the limiting factor ;
  • with SSH, I can reach only 750 MBit/s, but one of my cores is at 100% usage. As SSH is single-threaded, the CPU is the limiting factor ;
  • with IPsec, I read around 500 MBit/s but none of my cores are at 100% (they are under 50%).

So my question is : why can't IPsec reach a higher bandwidth ?

The two hosts are running Debian Wheezy and Strongswan for IPsec.

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
Tim
  • 191
  • 1
  • 3
  • 3
    You will need to ensure that your CPU's have the `aes` instruction to better decript packets on both sites http://www.intel.co.jp/content/dam/www/public/us/en/documents/white-papers/aes-ipsec-performance-linux-paper.pdf, and be sure that you are making ipsec parallel and using Tunnel Mode + ESP - https://www.strongswan.org/docs/Steffen_Klassert_Parallelizing_IPsec.pdf . That is the best way to achieve performance with this protocol. –  Mar 25 '15 at 13:42
  • Unfortunately I have i3 processors without AES-NI support, and I work in tunnel mode between the two hosts. I understand how both of your advices would increase the bandwidth in the case where one CPU was at 100% usage. Using AES-NI would allow more packets to be processed and would increase the BW. But here the CPU doesn't seem to be the limiting factor. – Tim Mar 26 '15 at 08:47
  • 3
    Hmm, that is interesting. I'd suspect that something in the kernel is slowing things up. Would you mind sharing your ipsec configuration? I'd like to instrument a VM and see what results I'll get with different settings. – Lmwangi Apr 17 '15 at 10:06

3 Answers3

1

I am wondering if it is because artificial waits or sleeps have been inserted into the crypto. It's not very likely imo, but...

http://www.tau.ac.il/~tromer/acoustic/

Couldn't post this as a comment, would have been a better fit that way.

Raymond Bannan
  • 61
  • 5
1

There are soooo many factors that go into this. The NIC hits more parts of the board than you can imagine. Any set if instructions can come across the wire and hit a part of the driver system and bottle neck you down. You can take a singleboard 1200mhz single core and tune out the hardware to blow the doors off of and quad core 3600mhz. This truly is a hardware specific question.

How do they do this? with something like this http://www.ixiacom.com/products/ixn2x That's the $165k "I will break you" device. i.e. 2 gigs erroneous traffic into a 1gig pipe. Once you start beating the system up and breaking stuff, your bottle neck 'will reveal its self'. Brush up on your GDB skills!

Every system has a different method to resolving the issue. Some boards have technology limitations that can cap you.

The answer is ambiguous because the solution is ambiguous. I can think of 20 different possibilities off the top of my head including paging which can vary from OS version to version.

brad
  • 11
  • 2
0

It may be that at any specific time only one of the two cores is saturated, but on average it looks like they are both at about 50% (because kernel randomly assigns a single-threaded IPsec process to both cores; however as far as I have noticed Linux (unlike Windows) usually tries to keep a thread on the same core).

At some short periods IPSec may wait also for the network which, in conjunction with slow transfer rate, would be a sign of poor buffering.

Also compression (if any) and protocol overhead may affect your tests.

mik
  • 1,332
  • 11
  • 15