Load Balance Issues

[AdamB]

255.255.255.255 is only one IP address.

255.255.255.252 or /30 is 4 (2 usable)

http://unixwiz.net/techtips/netmask-ref.html

Yep I understand this. When adding a single IP as a route and using 255.255.255.255, the route doesn't work. When changing to a /24 or 255.255.255.0, it starts to work. I didn't try a /30 on it. Let me try that real quick.

[AdamB]

routes.jpg

Here is SS of my routes.

Snipit of my traceroute.

[root@adamb ~]# traceroute 66.133.115.131
traceroute to 66.133.115.131 (66.133.115.131), 30 hops max, 60 byte packets
1 10.80.1.5 (10.80.1.5) 1.004 ms 1.016 ms 1.095 ms
2 L301.SYRCNY-VFTTP-12.verizon-gni.net (71.176.110.1) 7.661 ms 9.325 ms 7.398 ms

As you can see from the route. Traffic should be going out 96.238.145.164, not 71.176.110.1.

Added the below SS on accident, won't let me remove for some reason.

[dmorris]

Sometimes. It depends on the route cache in the kernel, and the cache in WAN Balancer.

btw 66.133.115.129/29 will handle all those.
http://www.subnet-calculator.com/cidr.php

I would just do that.

[AdamB]

Sometimes. It depends on the route cache in the kernel, and the cache in WAN Balancer.

btw 66.133.115.129/29 will handle all those.
http://www.subnet-calculator.com/cidr.php

I would just do that.

Just made the change to the /29, that was my plan, I just wanted to test it out first with the single routes.

I stopped and started wan balancer with no luck. Looks like a reboot is my only option, allthough I have never had to do that after adding or changing a route.

[dmorris]

No idea.
If you want to know whats going on then shoot us an email.

If you want to debug it yourself just use
ip rule list
ip route show table main
route -Ceen
ip route flush cache

and it should pretty much tell you why the kernel does what it does

[AdamB]

No idea.
If you want to know whats going on then shoot us an email.

If you want to debug it yourself just use
ip rule list
ip route show table main
route -Ceen
ip route flush cache

and it should pretty much tell you why the kernel does what it does

Appreciate the tip.

Very odd. I even tried changing the the gateway of the route to eth2, which is the same. This is after flushing the routes. Traffic seems to be hitting almost all of my circuits.

route -Ceen | grep 66.133.115
96.238.145.164 66.133.115.130 96.238.145.161 0 0 2 eth2 1500 0 0 0 -1 0 96.238.145.164
66.133.115.132 10.80.8.112 10.80.8.112 0 0 2 eth0 1500 0 0 0 -1 0 96.238.145.164
10.80.8.112 66.133.115.132 71.176.110.1 i 0 0 89 eth1 1500 0 0 0 725 1 10.80.1.5
66.133.115.130 10.80.8.150 10.80.8.150 0 0 2 eth0 1500 0 0 0 -1 0 96.238.145.164
96.238.145.164 66.133.115.132 96.238.145.161 0 0 2 eth2 1500 0 0 0 -1 0 96.238.145.164
66.133.115.132 108.12.29.3 108.12.29.3 l 0 0 3 lo 0 0 0 0 -1 0 108.12.29.3
66.133.115.132 108.12.29.3 108.12.29.3 l 0 0 265 lo 0 0 0 0 -1 0 108.12.29.3
10.80.8.150 66.133.115.130 71.176.110.1 i 0 0 2 eth1 1500 0 0 0 -1 0 10.80.1.5
10.80.8.150 66.133.115.130 108.12.28.1 i 0 0 44 eth3 1500 0 0 0 776 1 10.80.1.5
10.80.8.150 66.133.115.132 108.12.29.1 i 0 0 34 eth4 1500 0 0 0 -1 0 10.80.1.5
108.12.29.3 66.133.115.132 108.12.29.1 0 3 69 eth4 1500 0 213 0 743 1 108.12.29.3

I will more than likely open a ticket in the AM once I get to work.

[dmorris]

setting the route to eth2 is definitely NOT the same.
That means local to eth2 (it will ARP for these addresses).
I'm kinda surprised you have connectivity at all in this case.
Makes me suspect your routes aren't getting applied at all or something.

important edit:
you are SURE that 96.238.145.164 isn't the IP of the WAN interface?
You should be routing to the *gateway* of the WAN interface, not to the *address* of the WAN interface.
A route that routes traffic to yourself will probably be ignored. I've never tried this but I can't imagine it would do anything good. just thought I'd make sure Unfortunately you blacked out exactly the needed info. sigh.

[AdamB]

setting the route to eth2 is definitely NOT the same.
That means local to eth2 (it will ARP for these addresses).
I'm kinda surprised you have connectivity at all in this case.
Makes me suspect your routes aren't getting applied at all or something.

important edit:
you are SURE that 96.238.145.164 isn't the IP of the WAN interface?
You should be routing to the *gateway* of the WAN interface, not to the *address* of the WAN interface.
A route that routes traffic to yourself will probably be ignored. I've never tried this but I can't imagine it would do anything good. just thought I'd make sure Unfortunately you blacked out exactly the needed info. sigh.

Yep I was using the WAN interface. Its now changed to the gateway of the WAN. Looking at Untangle and all seems well. But then doing traceroutes and thats were it gets odd. Untangle shows all the traffic going out over the 96.238.145.x network, but my traceroutes on two seperate machines show it going out the 71.176.110.x network. Very odd.

route -Ceen | grep 66.133.115
96.238.145.164 66.133.115.132 96.238.145.1 0 0 8 eth2 1500 0 0 0 -1 0 96.238.145.164
66.133.115.132 10.80.8.112 10.80.8.112 0 0 8 eth0 1500 0 0 0 -1 0 96.238.145.164
66.133.115.129 96.238.145.164 96.238.145.164 l 0 0 3 lo 0 0 0 0 -1 0 96.238.145.164
66.133.115.129 96.238.145.164 96.238.145.164 l 0 0 116 lo 0 0 0 0 -1 0 96.238.145.164
66.133.115.129 10.80.8.150 10.80.8.150 0 0 2 eth0 1500 0 0 0 -1 0 96.238.145.164
10.80.8.150 66.133.115.129 96.238.145.1 i 0 0 32 eth2 1500 0 0 0 801 1 10.80.1.5
10.80.8.150 66.133.115.129 96.238.145.1 i 0 0 2 eth2 1500 0 0 0 -1 0 10.80.1.5
10.80.8.150 66.133.115.129 96.238.145.1 i 0 0 14 eth2 1500 0 0 0 -1 0 10.80.1.5
96.238.145.164 66.133.115.129 96.238.145.1 0 0 29 eth2 1500 0 246 0 801 1 96.238.145.164
10.80.8.112 66.133.115.132 96.238.145.1 i 0 0 121 eth2 1500 0 0 0 801 1 10.80.1.5
96.238.145.164 66.133.115.129 96.238.145.1 0 0 2 eth2 1500 0 0 0 -1 0 96.238.145.164

[root@adamb ~]# traceroute 66.133.115.132
traceroute to 66.133.115.132 (66.133.115.132), 30 hops max, 60 byte packets
1 10.80.1.5 (10.80.1.5) 0.808 ms 0.872 ms 0.945 ms
2 L301.SYRCNY-VFTTP-12.verizon-gni.net (71.176.110.1) 5.380 ms 5.803 ms 6.481 ms
3 L301.SYRCNY-VFTTP-12.verizon-gni.net (71.176.110.1) 7.358 ms 8.094 ms 8.672 ms
4 G5-0-2-216.SYRCNY-LCR-02.verizon-gni.net (130.81.179.122) 10.508 ms 11.181 ms 12.174 ms

C:\Users\netx>tracert 66.133.115.129

Tracing route to 66.133.115.129 over a maximum of 30 hops

1 1 ms <1 ms <1 ms 10.80.1.5
2 5 ms 4 ms 1 ms L301.SYRCNY-VFTTP-12.verizon-gni.net [71.176.11
.1]
3 4 ms 3 ms 3 ms G5-0-2-116.SYRCNY-LCR-01.verizon-gni.net [130.8
.177.122]
4 19 ms 18 ms 19 ms so-2-0-3-0.NY5030-BB-RTR1.verizon-gni.net [130.
1.28.184]
5 16 ms 16 ms 16 ms 0.xe-10-1-0.BR1.NYC1.ALTER.NET [152.63.18.225]

[dmorris]

weird, just PM me your UID and enable remote support and I'll look in a few.

i would test with wget and tcpdump on the appropriate wan interface.

[AdamB]

weird, just PM me your UID and enable remote support and I'll look in a few.

i would test with wget and tcpdump on the appropriate wan interface.

PM sent, very odd though as it didn't show up in my sent folder. Let me know if you didn't get it.