How to messure throughput?

[sky-knight]

The best way I've found to "benchmark" untangle is a unit test for a web app. There are tests out there that probe for all sorts of things. I just had my web dev buddy configure some unit tests against his web server I host, and put an Untangle between the test unit and the server.

Let that run for an hour and you get a solid idea of what an Untangle server will do when barraged by a TON of HTTP sessions.

Because at the end of the day it isn't good enough to generate network sessions, you need to generate sessions that will engage the UVM in a meaningful way.

To make a real benchmark (something I've been poking at for several years) we need a standardized test that will barrage HTTP, HTTPs, and SMTP simultaneously. There are limits to how much you can do with a single machine too. So in the end the numbers end up anecdotal.

The model numbers on my server weren't arbitrarily chosen. They do match what I expect to be maximum productive user count on the device. However, I will admit that there is far too much fairy magic in those numbers. They aren't nearly as defined as I wanted them to be. I know they work because of the installation base, not completely because I knew they would work up front.

It's a best guess game, and I hate it.

[keith_h]

To make a real benchmark (something I've been poking at for several years) we need a standardized test that will barrage HTTP, HTTPs, and SMTP simultaneously. There are limits to how much you can do with a single machine too. So in the end the numbers end up anecdotal.

Agree entirely. How do you simulate "real traffic". It's like putting a car on a dyno, you can see how much power is there but really, how does it drive?

I have pondered this such a testing scenario for stress testing gateways before commissioning. Such a tool would be quite complex to thoroughly test all the application components in a meaningful way. Thats why I think the application performance monitoring method although crude is not a bad way to go, but again, its situational.

I suspect what we are saying is there is no silver bullet here.

[sky-knight]

No, but developing a benchmark we can easily use against any Untangle installation just to get some statistical measures, grants benefits when compared to the same statistical measure used on similar equipment.

So assuming the test is done properly, we in theory could "UntangleMark" a given server. The number means jack over all, but it would give us a unit of measure to compare different servers. Once those numbers are linked to enough installations with user counts a more realistic basis of expectation for performance can be built.

There's just a TON of variables to take into account. And customers will misunderstand it anyway. So while it's a curiosity of mine to do this, even I have to admit in the end the information may actually be more destructive than anything else.

[MStauning]

I've could just have a bunch of kids to play wildly on the net, that should get the traffic up fast ;-)

I've have a look at all the good sucestions, and see what I've come up with - and report back.

[sky-knight]

I've could just have a bunch of kids to play wildly on the net, that should get the traffic up fast ;-)

I've have a look at all the good sucestions, and see what I've come up with - and report back.

That's basically all we do now! BUT, that isn't scientific, and you can't duplicate the "test" even yourself!

[keith_h]

For testing various things over the years including hardware and software, a "test harness" is typically developed. In the old days this was something we physically plugged into equipment that would continuously stress test in a known way which could then be measured and analysed.

I figure a "test harness" for untangle is achievable. But as noted, it probably only serves interests sake.

[MStauning]

I've know, so we are going to do some test on the running system next week (holidays in DK, so I've can kill the system if I've like ;-)).