Penthouse.com categorized as news

[Sam Graf]

its not realistic (performance wise) to do a categorization lookup on every web request

Understood. There is a need to flush the cache manually if there is a significant mis-categorization and the cache is not updated automatically.

There are two disturbing events here: The categorization of penthouse.com became corrupted (and who knows if corruption was limited to that site), and when it was corrected, the cached version persisted in use for some time on the order of hours. Admittedly I have no way of saying exactly how much time elapsed between the category update and the failure of the cache to be refreshed, but it was at least a couple of hours (the time between WebFool's post and my recheck, which ended in a manual refresh in an attempt to understand why there were differing results). Is this the correct operation?

[theoak]

There are a few "news" sites that I have run across like that.

I have had to add the sites specifically in the block list.

I suggest "pornography" and other categories as I stumble across them ... or my kids ...

My suggestions however never seem to get approved

At least I can add them to the block list.

[BillyBob]

Nothing against you theoak but I don't have time to add porn sites to the block list and neither do my customers. Part of what we pay Untangle should cover that. If the categorization service that Untangle uses thinks that penthouse.com belongs in the "news" category then I've lost all confidence in their categorization abilities. Not intending to be rude at all. If you're running across porn sites that aren't categorized correctly and you're having to manually add them to a block list then somethings wrong. I know some new sites will slip through but I don't have time for this!

[BillyBob]

Sam Graf,

Are you using the home version of Untangle? I only noticed this occurrence on my home installation. When I checked two of my customers installations they were behaving correctly. Surely the home license isn't behaving different than the business's that are paying full price. At home I'm using the Untangle full package that is 50$ per year or whatever. My customers are paying significantly more as they have 50-100 users or more.

Dmorris, Thanks for the input but I'm not sure how to respond. Are you saying there's something wrong with our specific installations? You told us how the categorization is designed to behave but we just caught it not doing that.

[Sam Graf]

I don't mean to minimize failures to properly categorize pornography, but that's relatively visible stuff. We have no idea if the reliability of the category service is better for more malicious, under-the-radar stuff. That's the truly creepy bit.

[BillyBob]

Dmorris,

I reread your post and I think you're saying that penthouse.com was mis-categorized and that the refresh corrected that?

[Sam Graf]

Are you using the home version of Untangle?

I am. I began using NGFW at a previous employer, and took advantage of the home license because of my appreciation for NGFW. So I can't concretely compare a commercial installation (except that in my experience even with the commercial product, making category suggestions isn't a very successful enterprise) anymore.

I doubt that Untangle treats home installations differently. I would think that would be counterproductive to their reason for offering the home license. But, I've been wrong before.

[dmorris]

Yes, penthouse.com was miscategorized. Once I saw the thread I submitted it to be fixed and it was fixed within 5 minutes.
Sorry about that. There is no need for you to take any action.

Our categorization provider made a mistake, if this is unacceptable to you then I understand.
Although of the url categorization providers we tested (and we've tested basically them all) there are only 2 that are decent IMO, and ours is one of them.

The home and business is the same. Its just a different license on the same product.

[Sam Graf]

Speaking for myself, mistakes happen. I get that.

I'm still unclear about the slow response of the cache refresh mechanism.

[sky-knight]

Speaking for myself, mistakes happen. I get that.

I'm still unclear about the slow response of the cache refresh mechanism.

Because it's DNS based, the upstream DNS servers you're using may have cached a lookup, and aren't bothering to ask the filter for an answer.

I had Google.com get categorized as a malware distribution point for one of my Untangle customers a few weeks ago, these things happen.