I’ve been testing a new DMVPN with IPSec encryption utilising brand new Cisco 3945 ISR G2 routers. I performed some basic performance tests using “iperf” with just a GRE tunnel (no encryption) between 2 sites and I was consistently getting 91Mbps throughput (not bad). Upon adding the encryption (AES-128) and re-running the tests the result were erratic with the throughput ranging from 16Mbps – 52.7Mbps with an average around 30Mbps.
Due to the additional overhead of the encryption I was expecting the throughput to be less than un-encrypted, but I was not expecting it to be that bad. I checked the routers for high CPU and memory with nothing standing out. I did confirm a huge portion of the packets were dropped but nothing initially stood out as to why. I confirmed that the MTU size on the tunnel interface was 1400 and the TCP Maximum Size (MSS) was 1360 as per Cisco’s recommendations, if not set this can cause an issue with fragmented packets. I then went to site and consoled onto one of the routers and re-ran the tests it was at this point I noticed the following error appear on the console ” %CERM-4-RX_BW_LIMIT: Maximum Rx Bandwidth limit of 85000 Kbps reached for Crypto functionality with securityk9 technology package license.”.
To begin with I assumed this could not be the cause of the issue otherwise I’d expect the bandwidth to be nearer the 85Mbps limit not 30Mbps. I gave the issue some further thought and wondered whether this licensing limit was actually dropping the majority of the traffic. I then thought of an idea to use QOS to limit the trafffic to be less than 85Mbps and observe the results. Never having used QOS before I googled and created the policy below. This defines a policy map called “QOS-WAN” and shapes the traffic to ensure no more than 75000000Bps is transmitted, I applied this policy to both routers.
shape average 75000000
descripton “WAN INTERFACE”
service-policy output QOS-WAN
Once I applied the QOS policies on both ends I re-ran the tests and low and behold I now haev a consistant throughput average of 71Mbps. The QOS policy can be modified to increase throughput until the license kicks in again (this can be observed in the logs). Obviously I will now need to go and buy the HSEC license for our routers but at least I have a work around to provide consistant throughput for our users. This document on Cisco’s website has more information on the HSEC License for the ISR G2 routers.