Weird Load Spikes

[psmith]

I currently have Untangle Premium in production running on it's own server. I've built a new Untangle Premium box in an ESXi 4.1 environment in the hopes of moving this one into production once it proves to be stable.

I'm seeing some load spikes that occur throughout the day like clockwork. This would normally not be unusual of course as the load would change and be very hard to guage, however I have a very consistent load being placed on this test machine.

2 computers are running through the Untangle test box with most modules turned on. Both computers have around 10 different tabs open in Chrome with an extension that automatically refreshes every tab at staggered 5 second intervals. Based on this, I should see very even performance throughout the day with the exception of the reports generating. Attached is what I'm seeing. The web sessions remain very constant with not much else going on but every 3-4 hours the load goes up dramatically.

Is there anyway to figure out what is causing this spike? Any help would be very appreciated.
-Paul

[WebFooL]

Hi psmith,

Is untangle the only VM running on the ESXi server?

CPU spikes can be untangle doing signature updates etc..

What VMware Tool Version do you have installed?

[psmith]

Hi WebFool,

Yes, Untangle is the only VM running on this server which has dual Xeon E5-2665s and 96GB of RAM. This is only a test box as I'm hoping to run other VMs along with Untangle on a less powerful server. I've assigned 8 vCPUs to this VM and I didn't install VMware tools on the server yet.

The signature update theory is interesting but I'm hoping stuff like that could be scheduled to run over night.

Thanks for the quick reply!

[WebFooL]

Untangle "OS" updates can be scheduled but the Virus/etc definitions might be updated now and then.

I would install the VMWare tools and then run the test.
(If you wan't a pre installed one you can find then on the sf.net)


Also read this VMware post
http://pubs.vmware.com/vsphere-4-esx...shoot_cpu.html

[dmorris]

whats your data retention setting in reports?

http://www.untangle.com/inside-untan...ance-guidance/

[psmith]

Thanks WebFooL, I tried to install VMware tools and couldn't get them to take so I'll try again next week. I typically use the prebuilt VM on sf.net but had some lockup issues so I built from ISO to eliminate that as a possibility (which I'm pretty sure was an issue with ESXi 5 or VM machine Version 8).

Hi dmorris, my data retention setting is at 7 days and report retention is 60 days. Also, a big thanks for linking to that performance guidance page. I had never run across that and it was very enlightening.

I'm still stumped on this one but I have a couple things to check now. I don't suppose there's anyway to view linux log files or anything of that sort that would show me what module or process is eating my cpu?

[dmorris]

My guess is that its nothing to do with CPU but its a shortage of disk I/O. Its a common issue on big vmware servers as the often have tons of CPU and memory to split between machines but just normal disk I/O to split between lots of machines. You can use tools like atop or iostat or vmstat to verify.

[psmith]

Thanks dmorris, I'll check that stuff and report back. Just to be clear, would disk I/O show up under load in the reports?