pfSense and UT on same VMWare ESX 5 Host

[photonman]

I saw some bug reports you are referring to so I think I will stick with the tried and true E1000 drivers for now.

My pfsense 2.0 box uses the VMXNET 2 drivers as the VMXNET 3 ones are not ready for BSD either.

[sky-knight]

I always used the e1000 emulation on anything *nix. I only used VMXNics on Windows VMs.

Citrix emulates realtek... ouch.

[photonman]

I wonder how VMWare DirectPath I/O would work with UT, unfortunately my PowerEdge 2950 host machine cannot support those VT features.

[photonman]

I am thinking of taking my bridge mode UT server off of my old PowerEdge 2950 server (which is also my pfSense host) and putting it on my newer Dell R710 ESX server as it is much more modern hardware and I am going to attempt to do VMWare Direct Path I/O for the UT nics with a couple of Intel nics available on this server.

This basically eliminates vNic's and vSwitch's by directly attaching the nic hardware to the UT virtual machine.

But here is my question and/or problem: this Dell R710 server also houses my entire network such as 2 Citrix servers, SQL server, Timberline pervasive SQL server, Ubuntu web server, Windows 2008 general purpose server.

good or bad idea?

[photonman]

well, I thought everything was great but right at the 2 week mark, funny things started happening. imap email connections were dropping all day. so finally, I thought, well, its a server, lets reboot...but after the reboot, the I could not do anything, the ut was frozen, could not get to command line or gui.

then I looked at the vm properties, and my network adapters were really messed up. Like both ut interfaces were on the same vSwitch which I know was not the case. I really did not put a lot of effort into troubleshooting, but it was wierd.

so I am rebuilding a new appliance and instead of putting the UT external interface on the same vSwitch as the pfSense Internal, I am going to use a crossover cable instead.

If this does not prove to be solid, I will have to put UT on dedicated hardware as I just do not want to add unnecessary complexity into the network.

EDIT: It is the VMWare Tools upgrade that kills the UT vm...not liking such a fragile setup.

Maybe it is the fact I am trying to virtualize a UT server that is already virtualized?

[sky-knight]

Don't touch it, you'll break it!

The VMTools will get a TON smoother when we're on Squeeze. Lenny + being in a VM is just strange. VMWare has to hit it with some rather aggressive hacks to get the NICs to work, Citrix needs a custom kernel!

Still I've never heard of a VM magically reassigning NICs. I submit you have something up with your vSphere host, not Untangle.

[photonman]

Don't touch it, you'll break it!


...Still I've never heard of a VM magically reassigning NICs. I submit you have something up with your vSphere host, not Untangle.

I broke it alright. I think part of the problem now that I do my post mortem, is that I tried to revert to a snapped image of the server...something did not go right and that is when I noticed the vmnic mess.

so I am back up with a new UT and I am using my crossover cable to connect the vmnics for pfs and ut and I actually get less performance hits. previously with the vSwitch setup, if I turned on virus blocker lite the speed tests results were horrible. now there is no difference in speed tests with vb lite on or off.

I am thinking I need to be on dedicated hardware but I have another 2 weeks to figure a plan out...see ya in 14!

[sky-knight]

Well we can't completely compensate for good old operator error!

I find Untangle, in any operation, is best left in a set it and forget it thought process. It always manages to get goofy when I start tinkering. I can't tell you how many times I've put in a rule to test something, forgot to turn it off, and find out three days later this intermittent nightmare is happening because of it.

That's a side effect of production and lab being merged realities. We just deal with it, because most of us can't afford a proper lab.

[photonman]

got myself a dual proc Dell 1750 with intel Pro/1000 GT for pfsense

got myself a dual proc Dell 2950 with 6x15K SAS 146 GB drives in Raid 10, 16 GB RAM, Intel Pro/1000 MT for untangle.

bye bye vmware for my perimeter defense devices

[sky-knight]

PFSense is on an overkill box to such a degree as it to never have a measurable load.
Untangle will likely not be able to see the RAID 10, unless you're putting VMWare on it first.

RAID + Untangle = Silly anyway.