Page 1 of 2 12 LastLast
Results 1 to 10 of 12
  1. #1
    Untangle Ninja sky-knight's Avatar
    Join Date
    Apr 2008
    Location
    Phoenix, AZ
    Posts
    26,542

    Default Untangle Benchmarking

    As a continuation of the following thread.

    http://forums.untangle.com/hardware/...gle-boxes.html

    I have been toying with different techniques to test an Untangle in near real world conditions. Far was correct in using iperf as a starting point. However, I think a combination of iperf, http, and smtp load tests are really required. I'm working on a server upgrade internally that will allow me to take a single client and pump the Untangle server full of HTTP and SMTP traffic. This will force all of the relevant filters to fire, and load the box right up until it breaks.

    At this point using only iPerf, my bare metal Untangle I'm testing against is an HP Pavilion a810n I had laying around. I've upgraded it to 1gb of ram, and removed the modem to install an Intel Pro100 network interface, and a 3com 905ctx interface. The onboard interface is a sis900 soft nic and active, but unused. For the purposes of this test the Intel is external, the 3com is internal.

    The iperf server is running via linux live cd from my recovery station, iperf client is running via same linux live cd running on my laptop. The three stations are connected with crossover cables, so no switch in the mix to gum up the works. Untangle is configured as a router.

    So far, using iperf -c x.x.x.x -d -t 120 -p 80 -P 25. This command generates 25 threads each burning 4 sessions, testing both directions of communications simultaneously for 120 seconds over TCP on port 80. This does trigger the web filter's checks.

    Also for the purposes of this test the open source rack is installed, the attack blocker is disabled.

    The results, are 11 connections terminating after transmitting and receiving approximately 115MBytes of data at an average of 8Mbit/Second. The remaining connections are failing with a "broken pipe" timeout. That yields us only 44% success rate. Not unsurprising considering it's an Athlon 64 3300+ single core. CPU load climbs to a threshold of 8, after 6 it starts having issues establishing TCP sessions.

    So according to these results this unit appears to lose stability if more than 11 people are wanting to download a large file all at the same time. Assuming a 40% resource distribution across users, gives us a theoretical limit of 27.5 users. I would say this hardware is consistent with a box that can handle 25 users given my experience with Untangle. So far, this seems accurate.

    Also interesting, and many thanks to the dev team for there efforts here. Untangle is no longer maxing out on ram with heavy traffic flows. The bottleneck on this server appears to be CPU related completely. After all, 11 connections each consuming 8mbit is 88mbit. Getting 88% of a given network pipe is darn good, most of the time I only see ~80%.

    So now onto rigging something to test HTTP more fully, and SMTP. And somewhere down the road I'm going to need to pickup some gigabit interfaces. All of the gigabit capable equipment I have is in service.

    The live DVD I'm using is Scientific Linux 5.
    Rob Sandling, BS:SWE, MCP
    NexgenAppliances.com
    Phone: 866-794-8879 x201
    Email: support@nexgenappliances.com

  2. #2
    Untangle Ninja dwasserman's Avatar
    Join Date
    Jun 2008
    Location
    Argentina
    Posts
    4,375

    Default

    Good post!!
    May be interesting to establish the limits of the Atom based equipment

    And register the utm sky´s benchmark quickly
    The world is divided into 10 kinds of people, who know binary and those not

  3. #3
    Untangle Ninja sky-knight's Avatar
    Join Date
    Apr 2008
    Location
    Phoenix, AZ
    Posts
    26,542

    Default

    Further testing shows my implementation of iperf to be only capable of running 16 threads consistently. Anything more gives me a thread error on the server, and this is breaking connections.

    Moving the testing off port 80 to iperfs's default port of 5001 and using the following test.

    iperf -c x.x.x.x -d -t 120 -P 16

    This generates 16 threads, each using 4 sessions. All sessions complete successfully. CPU load on Untangle doesn't climb over 2, and the average bandwidth is 5.87mbit per connection, each transmitting about 84mb in the 120 second window. Total aggregate bandwidth for the test is 1.31gb, 93.9mbit/sec.

    Adjusting this test back to port 80 by adding the -p 80 flag, the connections cannot be immediately established. The iperf client has to negotiate the window size from the 85.5kb default down to 16kb before it can connect. The connections are taking an additional 20-30seconds to establish due to the handshaking. I don't have a stop watch so... I'm guessing there. CPU load is peaking at the 8 marker it was before, hovering between 7.5 and 7.85. All connections completed successfully, at an median transfer rate of 5.65, and a median transfer amount of 81mb. I say median here because on this test the ranges are 5.6mbit-5.8mbit depending on the connection, and the amount transferred is changed slightly across the range of connections. Totals are 1.27GB transferred, at 90.5mbit.

    I don't think getting 16 people an 80mb file, through UT's web filters, in 120seconds is a bad thing. It though it does indicate a 3mbit performance loss to use the filters.

    Next step I think is to try and get multiple iperf servers and clients running at the same time... if 16 is all I can push with one client... let's see what we can do with 10.
    Rob Sandling, BS:SWE, MCP
    NexgenAppliances.com
    Phone: 866-794-8879 x201
    Email: support@nexgenappliances.com

  4. #4
    Untangle Ninja dwasserman's Avatar
    Join Date
    Jun 2008
    Location
    Argentina
    Posts
    4,375

    Default

    Take the phone of the firefighters near you
    The world is divided into 10 kinds of people, who know binary and those not

  5. #5
    Untangle Ninja Mathiau's Avatar
    Join Date
    Feb 2008
    Location
    Costa Frickn' Rica
    Posts
    1,630

    Default

    awesome information, i am tempted to test this on my def2 box with the quad core in it see how it performs
    kv-2 | UT 11.0.1 | Dell R610 Server | Intel Xeon 2.8Ghz Quad Cores | 24Gb DDR3 ECC | 1 Intel QPort NIC | Integrated Broadcom QP | Dell Perc 4i | 6 x 73G 2.5 15k SAS raid 10 | 100mb/100mb | 30mb/30Mb

  6. #6
    Untangle Ninja sky-knight's Avatar
    Join Date
    Apr 2008
    Location
    Phoenix, AZ
    Posts
    26,542

    Default

    Quote Originally Posted by dwasserman View Post
    Take the phone of the firefighters near you
    There's a station so close to the back yard I can hit the side of the thing with a rock if I angle it right. I'm sure they'd see smoke.

    That said... so far no good. The alternate IP bindings don't appear to work the way I think they do. And the 16 connection limit appears to be a server side limitation... so more servers are needed.

    I'm getting pretty good with ifconfig at least. :P
    Rob Sandling, BS:SWE, MCP
    NexgenAppliances.com
    Phone: 866-794-8879 x201
    Email: support@nexgenappliances.com

  7. #7
    Untangle Ninja sky-knight's Avatar
    Join Date
    Apr 2008
    Location
    Phoenix, AZ
    Posts
    26,542

    Default

    Ok... after working with ifconfig, route and a few other things I managed to get each testing station working on 3 IP addresses. I bound 3 iperf servers to the port 80 of all three addresses on the server side... then I ran 3 16 thread tests from the client for a total of 48 concurrent connections to a server on port 80.

    I then had the Untangle server dumping top every second into a text file for the duration of the test. Something seems odd because the tests aren't completing in the 120 seconds they are supposed to be. Yet the log files generated indicate everything ran as expected... open source software... joy...

    Anyway I still have my 8 load peak, max logged CPU load is 7.58. and each connection is reporting 1.6-1.7mbit/sec and somewhere between 20-25mb of data moved.

    test one reports 404MB of data at 28mbit/sec
    test two reports 420MB of data at 29mbit/sec
    test three reports 413MB of data at 28.6mbit/sec

    28+29+85.6=85.6mbit Getting closer to the 80mbit I'm used to. So now that we've established that on a pure traffic transiting level this box can handle 50 users... I'm going to have to change testing methods. All I have proven here is that Untangle, like any other linux based router, can handle more traffic than you could ever throw at it. It's just going to slow down based on how much connectivity you have.

    The real trial... what does this thing do when the AV module and SPAM modules are pushing the limits with the connections.
    Rob Sandling, BS:SWE, MCP
    NexgenAppliances.com
    Phone: 866-794-8879 x201
    Email: support@nexgenappliances.com

  8. #8
    Untangle Ninja sky-knight's Avatar
    Join Date
    Apr 2008
    Location
    Phoenix, AZ
    Posts
    26,542

    Default

    Ok, remembering http://forums.untangle.com/networkin...s-command.html and using mrunkel's instructions for session counting.

    While the test was running I ran ucli sessions 4 > sessions.txt. (4 is the isntance ID of http-casing) After the test was complete I ran cat sessions.txt | wc -l to get the number of lines. The result was 98. After reviewing sessions.txt I noticed that the output of ucli sessions 4 had 2 extra lines at the beginning that aren't session related, as they are table headers. So, we have 96 sessions active. 96 Sessions, but 16 x 3 connection threads. Indicating each thread is burning 2 sessions. I'm not sure why there are 2... unless the port response is handled as a separate session...

    I also discovered that top has a -b flag which removes the special formatting required to allow it to live update the screen. The command:
    top -b > top.txt
    This will create top.txt in the local directory and update the file with new data every second until you ctrl-c it. Once the test was terminated I inspected the log with the following command.

    cat top.txt | grep "top -" | less

    That dug out just the first line of the top output containing the three CPU load numbers and spewed them into less so I could navigate.
    Peak numbers are 8.37, 3.34, 1.67

    So, I'm not sold that this test is accurate. But here's the scripts I used to configure the server.

    Code:
    /sbin/ifconfig eth0 172.16.1.2 up
    /sbin/ifconfig eth0:0 172.16.1.3 up
    /sbin/ifconfig eth0:1 172.16.1.4 up
    And the script to start the three iperf servers

    Code:
    iperf -s -B 172.16.1.2 -p 80 > 2.txt & iperf -s -B 172.16.1.3 -p 80 > 3.txt & iperf -s -B 172.16.1.4 -p 80 > 4.txt
    Be careful, the server doesn't terminate with a single test. If you want to rerun it you have to use PS and kill the iperf processes that are still running. Not to mention remove the .txt files for a fresh start.

    The client side is a bit more

    Code:
    /sbin/ifconfig eth0 192.168.1.10 up
    /sbin/ifconfig eth0:0 192.168.1.11 up
    /sbin/ifconfig eth0:1 192.168.1.12 up
    /sbin/route add default gw 192.168.1.1
    Then the code to run the test

    Code:
    iperf -c 172.16.1.2 -t 120 -P 16 -B 192.168.1.10 -p 80 > 10.txt & iperf -c 172.16.1.3 -t 120 -P 16 -B 192.168.1.11 -p 80 > 11.txt & iperf -c 172.16.1.4 -t 120 -P 16 -B 192.168.1.12 -p 80 > 12.txt
    Before you do anything, on Untangle's console run

    Code:
    ucli instances
    Make a note of the instance ID associated with http-casing. You'll need it when you run

    Code:
    ucli sessions #instanceid | wc -l
    To get a session count, remember to subtract two to get an accurate number... and I'm thinking it may need to be divided by two as well. As that's what it took to get down to the connection threads I had.

    Also note that this reveals the documentation I had that said iperf would use 4 sessions per connection thread to be false. I only see 1 connection, and the response to that connection. That's 2 sessions per thread.

    And I'm going to have to remember top -b > file.txt, it's handy and allows you to easily log CPU load every second as long as you need. There is a -t flag that allows you to specify a number of seconds.
    Rob Sandling, BS:SWE, MCP
    NexgenAppliances.com
    Phone: 866-794-8879 x201
    Email: support@nexgenappliances.com

  9. #9
    Untangle Ninja YeOldeStonecat's Avatar
    Join Date
    Aug 2007
    Posts
    1,565

    Default

    Quote Originally Posted by dwasserman View Post
    Good post!!
    May be interesting to establish the limits of the Atom based equipment

    And register the utm sky´s benchmark quickly
    It would be good to see, there are so many implementations where clients of mine that have multiple sites...and I'd like to build a WAN using small Untangle units at the satellties instead of the Linky/Cisco RV0 series of routers...naturally with a decent box at mothership. Although as I think about it, it's not so much the web traffic at satellite offices...it's their ability to maintain throughput across the VPN tunnels.

  10. #10
    Untangle Ninja sky-knight's Avatar
    Join Date
    Apr 2008
    Location
    Phoenix, AZ
    Posts
    26,542

    Default

    That's part of where I'm going with this Stonecat, these stress tests are part of the process I'm going through to try and get a stable IPSec tunnel working with a Sonicwall TZ170 I have sitting here.

    I need OpenVPN and IPSec connection stats.
    Rob Sandling, BS:SWE, MCP
    NexgenAppliances.com
    Phone: 866-794-8879 x201
    Email: support@nexgenappliances.com

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

SEO by vBSEO 3.6.0 PL2