New install stops traffic dead

MechanicalThinker · 12-29-2011, 12:31 AM

I'm trying to replace an old server with new untangle box. Did a new install on new box, installed available upgrades and copied all the config and rules from old machine. Old machine is an old 2x Xeon 2,4 2 gb RAM server running Untangle 9.0, the new machine is a Phenom 975 box with 4 gb ram running Untangle 9.1.

The thing is that when I'm connecting one machine to new box, everything seems to run fine and smooth. I can access all the services that the network uses. When I put the new machine into full production, everything crawls to halt in no time. CPU and RAM utilization remain low, so problem can not be there. Machine stays responsive and does not lock.

So I tried to switch modules on and off (running all the Lite Package modules at the moment). Only thing that seems to make any difference is the Firewall. When I switch it off, traffic seems to return. When I switch it on, everything pretty soon returns to halt. I have checked and double checked all the rules and they mirror exactly the rules on the old box. Except I have added two rules to end that block all incoming and outgoing traffic that is not explicitly allowed.

So I'm a bit at loss here.

Mathiau · 12-29-2011, 06:31 AM

did you load a backup or import individual module rules?

i beleive full backups from older versions dont work properly.

can you install the firewall module, reboot, reinstall it and manually re-add all your rules?

are you in router or bridge mode, i presume router...

MechanicalThinker · 12-29-2011, 08:51 AM

I'm in router mode.

Well, actually I did recreate the rules from scratch by hand. As we need to keep in check with the compliance requirements I had to document all the rules. So I thought that I do the config by hand and at the same time write everything down according to compliance rules.

The other thing I noticed is that when firewall is on, web traffic is first to come to halt. It will grind to a very slow until it just stops. At the same time one of our database applications that works over MAN is still going nice and smooth. But eventually this also will get dreadfully slow.

Edit: the usual average session count this site has is about 1000. On new box I have not seen it rise above 400, but everything is almost dead by this time.

dmorris · 12-29-2011, 08:57 AM

Quote:

Originally Posted by MechanicalThinker

So I tried to switch modules on and off (running all the Lite Package modules at the moment). Only thing that seems to make any difference is the Firewall. When I switch it off, traffic seems to return. When I switch it on, everything pretty soon returns to halt. I have checked and double checked all the rules and they mirror exactly the rules on the old box. Except I have added two rules to end that block all incoming and outgoing traffic that is not explicitly allowed.

I'm betting your issue is in your rules somewhere.

Post a screenshots of or the export of the rules. Or just look at the eventlog and see whats being blocked that shouldn't be. Sounds like you're just blocking something crucial for normal operation. (like DNS or something)

MechanicalThinker · 01-18-2012, 11:35 PM

Ok, I finally had some time to put this machine back to production environment and do some more troubleshooting. I think dmorris is right, it comes down to last firewall rule that is set to block all traffic that is not allowed before it. The thing is that with one machine connected to untangle all works as it should. With many machines no single machine can not go to net. As I disable this rule, everything goes back to normal. Moreover, this somewhat affects web more than other traffic. For example our database application clients will continue to work, but very slowly.

What is the best way to create this rule that would block all traffic that is not explicitly allowed. I have tried several ways, but the effect seems to be the same - network grinds to halt.

On a more general note, I really can not understand why the default block action was removed from firewall. On the other machine that has been upgraded to untangle 9 this feature has remained intact and everything works as it should. What good is a firewall that is not to set to block traffic that is not allowed? I can't be the only person who needs keep all the ports closed that are not used. Due to compliance reasons we just can't set firewall to open all ports; everything that is opened needs to be documented etc.

Ps. As I messed around with this machine, I installed latest patches also to this machine. This had no effect whatsoever to the problem at hand.

dmorris · 01-18-2012, 11:51 PM

Quote:

Originally Posted by MechanicalThinker

On a more general note, I really can not understand why the default block action was removed from firewall.

Because its redundant. You just create a rule at the bottom that blocks everything and then you have default block.

Post a screenshots of or the export of the rules. Or just look at the eventlog and see whats being blocked that shouldn't be. Sounds like you're just blocking something crucial for normal operation. (like DNS or something)

bhackbarth · 01-20-2012, 03:00 PM

Quote:

Originally Posted by dmorris

Because its redundant. You just create a rule at the bottom that blocks everything and then you have default block.

Then the underlying iptables --policy option is redundant too. Can just put the drop rule at the bottom. I think it comes down to how some people are more visual than others and will miss the default block radio buttons. To me it makes sense to add the block rule at the bottom and not need the radio buttons. It would stand to reason that all those radio buttons really did was change the policy chain I just mentioned.

sky-knight · 01-20-2012, 03:39 PM

The Firewall module is NOT IPTables, so the argument you make doesn't make sense.

More importantly the firewall module's default block didn't log anything so anyone that was serious about default block ended up setting it to default pass and putting in their own default block rule with the ability to turn logging on and off anyway.

dmorris · 01-20-2012, 04:19 PM

and with no associated rule #, logging blocks blocked by the magical invisible default rule with no rule number was problematic. It made reports a total mess.

Either you include the special rule with no rule number and you just have no real metrics over which are your top rules and how much is being blocked for a reason. Or you exclude the magical rule in which case your blocked/passed percentages and statistics are just plain wrong.

If you are reluctant to post your rules so we can help, email support@untangle.com and they can help you privately.

MechanicalThinker · 01-30-2012, 01:24 AM

To follow up on this. In short: dmorris was right. I had some time at the weekend to again deal with this case, and after almost going mad, I finally saw that the outgoing DNS rule had both source and destination interfaces set to internal. A big DOH! Fixed this and put the machine in production environment, seems to work for now.

So thanks to everyone for suggestions.

12-29-2011, 12:31 AM	#1 (permalink)
MechanicalThinker Untangler Join Date: Apr 2010 Posts: 34	New install stops traffic dead I'm trying to replace an old server with new untangle box. Did a new install on new box, installed available upgrades and copied all the config and rules from old machine. Old machine is an old 2x Xeon 2,4 2 gb RAM server running Untangle 9.0, the new machine is a Phenom 975 box with 4 gb ram running Untangle 9.1. The thing is that when I'm connecting one machine to new box, everything seems to run fine and smooth. I can access all the services that the network uses. When I put the new machine into full production, everything crawls to halt in no time. CPU and RAM utilization remain low, so problem can not be there. Machine stays responsive and does not lock. So I tried to switch modules on and off (running all the Lite Package modules at the moment). Only thing that seems to make any difference is the Firewall. When I switch it off, traffic seems to return. When I switch it on, everything pretty soon returns to halt. I have checked and double checked all the rules and they mirror exactly the rules on the old box. Except I have added two rules to end that block all incoming and outgoing traffic that is not explicitly allowed. So I'm a bit at loss here.

12-29-2011, 06:31 AM	#2 (permalink)
Mathiau Join Date: Feb 2008 Location: Costa Frickn' Rica Posts: 1,467	did you load a backup or import individual module rules? i beleive full backups from older versions dont work properly. can you install the firewall module, reboot, reinstall it and manually re-add all your rules? are you in router or bridge mode, i presume router... __________________ Def1:Started:UT 7.1 x64 -- Current :UT 9.1 x64\| Gigabyte GM-G31 mATX \| Intel Q8200 \| 8G DDR2 800 \| 80G WD \| 4x Intel Pro 1000 GT NIC's \| Corsair 550W PSU \| Norco RPC-250 2U Case \| 50mb/50mb \| 10 users

01-20-2012, 03:39 PM	#8 (permalink)
sky-knight Join Date: Apr 2008 Location: Phoenix, AZ URLs submitted: 8 Posts: 15,457	The Firewall module is NOT IPTables, so the argument you make doesn't make sense. More importantly the firewall module's default block didn't log anything so anyone that was serious about default block ended up setting it to default pass and putting in their own default block rule with the ability to turn logging on and off anyway. __________________ Rob Sandling, BS:SWE, MCP Intouch Technology Phone: 480-272-9889 rob@intouchtechllc.com UntangleAppliances.com Phone: 866-794-8879

01-20-2012, 04:19 PM	#9 (permalink)
dmorris Untangle Junkie Join Date: Nov 2006 Location: San Mateo, CA URLs submitted: 10 Posts: 10,612	and with no associated rule #, logging blocks blocked by the magical invisible default rule with no rule number was problematic. It made reports a total mess. Either you include the special rule with no rule number and you just have no real metrics over which are your top rules and how much is being blocked for a reason. Or you exclude the magical rule in which case your blocked/passed percentages and statistics are just plain wrong. If you are reluctant to post your rules so we can help, email support@untangle.com and they can help you privately. __________________ Attention: Support and help on the Untangle Forums is provided by volunteers and community members like yourself. If you need Untangle support please call or email support@untangle.com Last edited by dmorris; 01-20-2012 at 04:25 PM..

Protect

Filter

Perform

Connect

Manage

Add-Ons

12-29-2011, 08:51 AM	#3 (permalink)
MechanicalThinker Untangler Join Date: Apr 2010 Posts: 34	I'm in router mode. Well, actually I did recreate the rules from scratch by hand. As we need to keep in check with the compliance requirements I had to document all the rules. So I thought that I do the config by hand and at the same time write everything down according to compliance rules. The other thing I noticed is that when firewall is on, web traffic is first to come to halt. It will grind to a very slow until it just stops. At the same time one of our database applications that works over MAN is still going nice and smooth. But eventually this also will get dreadfully slow. Edit: the usual average session count this site has is about 1000. On new box I have not seen it rise above 400, but everything is almost dead by this time.

01-18-2012, 11:35 PM	#5 (permalink)
MechanicalThinker Untangler Join Date: Apr 2010 Posts: 34	Ok, I finally had some time to put this machine back to production environment and do some more troubleshooting. I think dmorris is right, it comes down to last firewall rule that is set to block all traffic that is not allowed before it. The thing is that with one machine connected to untangle all works as it should. With many machines no single machine can not go to net. As I disable this rule, everything goes back to normal. Moreover, this somewhat affects web more than other traffic. For example our database application clients will continue to work, but very slowly. What is the best way to create this rule that would block all traffic that is not explicitly allowed. I have tried several ways, but the effect seems to be the same - network grinds to halt. On a more general note, I really can not understand why the default block action was removed from firewall. On the other machine that has been upgraded to untangle 9 this feature has remained intact and everything works as it should. What good is a firewall that is not to set to block traffic that is not allowed? I can't be the only person who needs keep all the ports closed that are not used. Due to compliance reasons we just can't set firewall to open all ports; everything that is opened needs to be documented etc. Ps. As I messed around with this machine, I installed latest patches also to this machine. This had no effect whatsoever to the problem at hand.

01-30-2012, 01:24 AM	#10 (permalink)
MechanicalThinker Untangler Join Date: Apr 2010 Posts: 34	To follow up on this. In short: dmorris was right. I had some time at the weekend to again deal with this case, and after almost going mad, I finally saw that the outgoing DNS rule had both source and destination interfaces set to internal. A big DOH! Fixed this and put the machine in production environment, seems to work for now. So thanks to everyone for suggestions.