A problem and suggestion

[GeneralEclectic]

Hi All, I've been a very happy Untangle user (free version, home use) for about a year now. I've had practically no issues with it (and I'm in no position to offer much useful help to others) so you haven't seen me around here very much. However, I have run into one nagging problem that I'm pretty sure others may run into occasionally...

This installation is the default free version, set for automatic updating, so it's very likely the latest configuration. There are a number of filters configured, but the one of interest is the Ad Blocker. There are some sites (CNN and MSNBC to name two popular ones) where the Ad Blocker causes serious loss of functionality.

On CNN, there are several "sections" (click on the "Justice" heading on the main page) that come up blank. There is some HTML for the "Justice" page, but nothing is displayed. Turning off ad blocking allows the page to be displayed. I've searched the page source for a clue to what's not loading, but have not turned up anything useful.

On MSNBC, some/most videos won't play properly. Strangely enough, the video AD plays properly, but the content itself never loads. Again, turning off ad blocking allows things to work.

Now, I'd love to troubleshoot this myself. I encountered many situations like this pre-Untangle when I was using Winproxy. Winproxy had a very useful tool for resolving such problems -- it could log every connection attempt with a time stamp. So finding the culprit was easy -- once you know when the problem occurred, you just looked at the log and could see in detail what it was trying to do. Then you could adjust the ad blocker and, presto, problem solved.

Well, you can't do that with Untangle, far as I know, because there's no such thing as a connection logger.

So that's my suggestion/request: 1) Could someone who knows how to do such things add a function like this to, say, the Report module? I can't be the only person who'd appreciate detailed connection logging! And 2) Could whoever maintains the Ad Blocker PLEASE PLEASE PLEASE put in a string search function? Looking for "xyz" pattern is a real chore as it is now.

Thanks for reading, and thanks for any insights or suggestions you may have.

[sky-knight]

The connection logger you're after is the view dependencies plugin for firefox.

It would be nice if something was built into Untangle itself... but that stuff is really starting to get into a realm not unlike black magic.

The Ad Blocker causes problems all over the place, and as more and more web sites rely on AD based revenue, and they start actually testing if the AD content makes it to the browser, you're going to have more and more sites break down in the face of that module.

There is a reason why the Ad Blocker was made, and a reason why AD based systems have been removed from the spyware module. I do NOT deploy the ad blocker on ANY network. Why? Reasons you've just pointed out. A very large swath of the Internet simply doesn't function anymore. And troubleshooting it is a management nightmare.

Don't take this as me discounting your suggestion... better tools are never a bad thing. But we need this kind of functionality all over the place. And I'd like to see a string search system that would look through ALL the modules, for a given event at a given time, so we can query any likely blocking module all and once and see where problems are more quickly.

[GeneralEclectic]

Thanks for the tip on the Dependencies plugin. It's helpful, no question. However (at the risk of hijacking my own thread) even after going through that "Justice" page's dependencies, line by line (you get a longer list with ad-blocking turned off) I still haven't been able to pinpoint where the problem lies. I'm starting to think that the ad blocker either doesn't implement the Pass List, and/or ignores the enable/disable flag on individual filters. This is becoming a bit of a frustrating exercise, to say the least.

In comparison to Untangle, Winproxy would probably be considered primitive, but when it came to thoughtful implementation of user-friendly features, it acquitted itself well. I don't think I ever spent more than a minute or two resolving an ad-blocking issue even with the relatively primitive tools it offered. Its ad-blocking process was far superior to that of Untangle's too, in my opinion. Instead of simply suppressing the connection to the ad content, it would return a one-pixel transparent GIF so that the browser wouldn't display "The web site declined to show....". The result was a very clean ad-free page every time. I'd really like to see Untangle's ad-blocker implement this transparent-GIF approach rather than the 404 it does now. This also might address the page-breaks-without-ads dilemma in many cases.

I wonder if a new post with a poll about connection logging would draw more attention to this and, depending on the results, might get someone who's capable of implementing such a thing sufficiently motivated to put it on a list of things to do.

[sky-knight]

The adblocker is based on the adblocker plugin for firefox. That plugin returns a 1px white graphic set to size itself to match the size of the ad being replaced. Untangle does the same thing, or else pages would break because layout flow is often determined by the ad.

As for the giant PITA to troubleshoot, it isn't just adblocker here. The spyware blocker, web filters, and everything else that does URL filtration on this system takes the same effort. It's not so much about the tools, though you're right they are a component, it's the way the content is delivered.

Major sites use content networks that will change URLs on the fly to certain content and redirect requests to local data centers. Those URLs are long, unfriendly, and random. So how are you going to statically list them?

[GeneralEclectic]

Dang. That's interesting. There are some pages where it does 404 on ads. Unfortunately, I can't come up with an example at the moment, but I have seen a few recently. Now this shows up only with IE, which accounts for the infrequency I've seen them since I'm a FF user 99% of the time.

Point taken on redirection and random huge URLs. Even so, I'd put up with a few ads just to get these problem sites fully functional.

[sky-knight]

That's what I've done... I just don't use that module. The spyware module grabs the dangerous stuff. I honestly don't care about seeing an ad or two on a web site, it's that company's site to do with as they please. But, not being able to use a massive site or two just generates complaints I don't have time to deal with.

P.S. That module used to present a block page! That was some funny stuff when you go to MSN or something similar and get 5-6 blocks of tiny little block pages.

[Casey]

I'm having this exact problem with the pages you mentioned on CNN not displaying. I really don't want to tackle the cdn strings and such, but I also really don't want to turn adblock off. It looks like I'll have to, tho.

[hescominsoon]

not to bring up a competitor but I use Astaro at the house and i have ads blocked there. I don't know what they do differently but cnn works fine even with the ad blocking on. The ad blocking for FF is draconian at best.. useless at worst. I ran it for a while but i spent so much time whitelisting things it wasn't worth it. I think the ad blocker needs to be base don something else...All of my clients who run firewalls run UT.

[dmorris]

If you block the CNN ads they won't show the content.
You need to add those ads to the passlist.

[hescominsoon]

wrong..i'm watching cnn videos right now with astaro ad blocking enabled.