Caching proxy?

[rasqual]

Any prospect of that coming anytime soon?

I mean, c'mon. No one can tell me that wouldn't be a tres cool accoutrement for this thing.

I'm still in the honeymoon stage with Untangle, and it's quite a honeymoon. Thanks for bringing this product to market.

[dmorris]

Glad you like it! :D

We definitely think a web cacher would be cool.

What we are trying to evaluate is the saving actually to be had by implementing one. Modern browsers do quite a bit a caching on a per user basis, so significant savings only happen on very large networks.

Although if we could figure out a way to just plug in squid ;)

[rasqual]

Although if we could figure out a way to just plug in squid ;)

Right!

One place where the advantage is realized well before "large" numbers, is in education. Frequently, an entire lab full of students is doing the same thing under instruction. This is one case where caching would absolutely be of benefit.

Keep up the good work.

[PimpSel]

I've been testdriving your product and I like it, very much.. the only reason we're not implementing it as of now, is because of the lack of a caching proxy server..

We own a computer assembly / repair / service centre and we have one broadband pipe with contractual 40 GB /month limit. Currently we are using a transparent caching proxy server and we consume about 30 GB/ month, thanks to the proxy.. and this without recreational browsing, no peer to peer transfers.. VPN access, e-mail and various updates (Microsoft products, antiviral products) are the only traffic going in and out.

Without this cache, our contract would not be sufficient..

It is easy to understand to which extent a proxy greatly reduces traffic in our environment.. Just imagine how much updates freshly installed windows boxes and their antiviral products retreive.. Our own machines get updated via an internal wsus server and symantec enterprise, these however can not be (easily) used for updating our customer's machines..

We have an average of 15 to 20 freshly installed new or refurbished pc's a day, multiply this by the tremendous amount of updates, service packs, and you've got an idea what kind of traffic this creates..

Any ideas when this option will be implemented?

Keep up the good work,

PimpSel

[crab]

I personally think the easiest step here is a forward proxy (cache_peer).
That way I can still have a squid caching proxy running (with whatever complex squid rules I want) and it doesn't complicate matters for untangle. I don't think introducing squid into untangle will stick in line with its easy of use for newbies etc.
Having a forwarding proxy could also be the first step to installing squid on the untangle box and then having a few checkboxes to cache data etc.
I've been using ipcop and whilst it doesn't touch untangle it does allow me to forward to a proxy and this then gives me more control on my web traffic/

[dmorris]

I personally think the easiest step here is a forward proxy (cache_peer).
That way I can still have a squid caching proxy running (with whatever complex squid rules I want) and it doesn't complicate matters for untangle. I don't think introducing squid into untangle will stick in line with its easy of use for newbies etc.
Having a forwarding proxy could also be the first step to installing squid on the untangle box and then having a few checkboxes to cache data etc.
I've been using ipcop and whilst it doesn't touch untangle it does allow me to forward to a proxy and this then gives me more control on my web traffic/

hmm.. so you would suggest having the untangle box transparently catch port 80 traffic and send it through an explicit proxy?
seems like a pretty good idea :)

[crab]

hmm.. so you would suggest having the untangle box transparently catch port 80 traffic and send it through an explicit proxy?
seems like a pretty good idea :)

Ye something along those lines. I was thinking that you could chain it in a rack with web filtering. So you could send to an upstreaming proxy before or after webfiltering.
This would enable users who already have a squid caching solution to carry on and also allow us to poll up to a security proxy.

For built-in caching it should simply be a case of having a build of squid listening on 3128 or similar on the localhost. A couple of config options could allow you to reconfigure your squid run and I think it would be extremely importrant to include bypasses (intranet sites etc) that you don't wish to use your proxy server.

Maybe even open up the squid.conf for advanced users.

[dayala]

You can try to build an Update accelerator first. This would be a BOMB!

Imagine, being able to cache:

Windows, Linux, and Application updates such as Symantec, McAfee, etc.

This would help save bandwidth consumption.

Think about it.

Dennis

Glad you like it! :D

We definitely think a web cacher would be cool.

What we are trying to evaluate is the saving actually to be had by implementing one. Modern browsers do quite a bit a caching on a per user basis, so significant savings only happen on very large networks.

Although if we could figure out a way to just plug in squid ;)

[mcscotty]

If you want to use Squid to cache and untangle to filter, you could configure untangle as the default gateway for your squid box. Granted, this doesn't help you if you're trying to do this without touching the client configurations, but it can work well in an environment like mine, where I already have an explicit proxy in place, and block outbound ports 80 and 443 from my DHCP pool addresses to force users to use it.

[stlscott]

The Proxy Server feature is holding up several implementations for me. Currently, I have several hundred workstations configured for an existing firewall w/proxy (port 8080). Without this feature, I can't replace those firewalls with Untangle because I would have to touch all of those computers. No, they are not part of a domain so Group Policies can't be used. Further, when you have 30 or 40 users, the caching proxy filter truly reduces the bandwidth load, especially for "home pages" like MSN and CNN that load every time a user launches IE. When I put the last caching server in, I saw a 30% drop in bandwidth usage over prior usage for 4 months. Caching Proxy Servers work!

[tarrith]

The Proxy Server feature is holding up several implementations for me. Currently, I have several hundred workstations configured for an existing firewall w/proxy (port 8080). Without this feature, I can't replace those firewalls with Untangle because I would have to touch all of those computers. No, they are not part of a domain so Group Policies can't be used. Further, when you have 30 or 40 users, the caching proxy filter truly reduces the bandwidth load, especially for "home pages" like MSN and CNN that load every time a user launches IE. When I put the last caching server in, I saw a 30% drop in bandwidth usage over prior usage for 4 months. Caching Proxy Servers work!

I have the same setup as Stlscott, I serve out the internet via proxy on port 8080 as well.
Other than that I must say THIS THINGS GREAT!

Tarrith

[hescominsoon]

Glad you like it! :D

We definitely think a web cacher would be cool.

What we are trying to evaluate is the saving actually to be had by implementing one. Modern browsers do quite a bit a caching on a per user basis, so significant savings only happen on very large networks.

Although if we could figure out a way to just plug in squid ;)

IME squid saves between 25%-60% on bandwidth depending on traffic patterns. This is irregardless of size. You already have a proxy with the webscanner..just route the final scanned data through squid for caching.(take a look at the copfilter project which uses ipcop). Just make sure you don't have squid cache in ram much or your ram requirements go through the roof. I have had to tweak several squid configs to reduce squid memory footprint which can be rather large..:) I'm not a squid expert but if you would like i can post my tweaks that i normally use to reduce squid's memory footprint.

This looks like an excellent product. I have 10 firewalls i admin..some heavily modded ipcop machines and some astaro. Get the caching proxy server in there and i can begin evaluating replacing ALL of those machines with your product.

[woodrowbone]

Hi all!
I am new to Untangle and still learning, but I also miss the proxy feature.
Why not use something like advproxy?
http://www.advproxy.net/
I think this only works on IPCop and Smoothwall but it is an excellent feature.
It works more like a file server then a web proxy, saving all windows update and more.
I will attach a screen shot from my current Router software (endian) were you can see what programs it can save updates for.
As you can see I am only using it to cache windows update.

[hescominsoon]

Hi all!
I am new to Untangle and still learning, but I also miss the proxy feature.
Why not use something like advproxy?
http://www.advproxy.net/
I think this only works on IPCop and Smoothwall but it is an excellent feature.
It works more like a file server then a web proxy, saving all windows update and more.
I will attach a screen shot from my current Router software (endian) were you can see what programs it can save updates for.
As you can see I am only using it to cache windows update.

adv requires squid..which untangle currently lacks..:(

[dolson11]

I have been using untangle on a couple of small networks and I love it!!! But I agree a cache server would be great. Currently we use an Esoft Instagate EX2 and it is linux based I believe and it has a great web cache server built in that allows you to choose the cache size from 1-500mb. Helps a lot with updates and if we are having the whole class go to the same website!!!

Hope this will happen someday!!

Thanx for all the hard work!!!

[ItsOnlyMe]

What we are trying to evaluate is the saving actually to be had by implementing one. Modern browsers do quite a bit a caching on a per user basis, so significant savings only happen on very large networks.

I have 30%-35% cache hits in a small law-consulting company with 30 computers in it (the statistics is based on a WinRoute report). I think this is quite large amount... so the cache should be really efficient.

And one more thing to add. I think the cache efficiency should raise if there will be a good AD filter before it, because ADs usually loads dynamically and are incacheable. An example of such an AD filter is Mozilla's Adblock Plus plugin.

[MacMan]

Glad you like it! :D

We definitely think a web cacher would be cool.

What we are trying to evaluate is the saving actually to be had by implementing one. Modern browsers do quite a bit a caching on a per user basis, so significant savings only happen on very large networks.

Although if we could figure out a way to just plug in squid ;)

Every day we download a multi-megabyte pattern file to each computer. See below.

"Virus Pattern Files
Official Pattern Release 5.397.00
As of Jul 09, 2008, the latest pattern file number is 5.397.00.

The Official Pattern Release or OPR is Trend Micro's latest compilation of patterns for identified viruses. It is guaranteed to have passed a series of critical tests to ensure that customers get optimum protection from the latest virus threats.

lpt397.zip (Windows) 15.7MB"

Plus there are Java updates, windows updates, acrobat updates, updates to Firefox, and open office...

None of this benefits from the local browser cache!

[MacMan]

I have an enhancement request open on this.
If you want to help pile on the votes for this feature go to
:) http://bugzilla.untangle.com/show_bug.cgi?id=4015 :)
and vote for “Bug# 4015”

[freeside]

first, let me say what a good idea
this would be.

second, if you want a transparent proxy
outside somewhere ut 5.2 can do this already. portforward?

third, browsers DO know alot about caching, BUT
mostly, that's part of what you have to clean-up (disk-defrag) in XP once month.

i have another box (playing around) with opensuse linux 11
at it does the proxying for me at at the moment.
all my browsers have instruction to cache 0 MB. (less fragmentation and disk
usage)

fourthly? squid (at least) comes with a mechanism to
do "clustering" ... meaning you can add other squid proxy
boxes, it should ask before going for the source.

this is "very" similar" like TOR works (of course not encrypted, yet)

so maybe, -IF- you implement squid in UT, maybe have a page
where people can share their proxy, to form a big-o cluster.
benefit for privacy? anonymous

thanks ...

[datajockeys]

+1 for Squid

[Shadow_Knows]

I've only just found out about UT just a few days ago and now have v5.3 running on my home network. Not just 4-5 minutes after getting all of the racks setup - was I looking for squid. I run several CentOS machines and have Squid running in a few test networks right now. Amazingly easy to setup - and can see the benefits almost immediately. Hopefully - an optional RACK can be utilized in such a fashion that users wouldn't have to put in entries into their browsers, IE, Firefox, etc for a specific proxy server address. Having it transparent in the background would ease everyone's installation and likelihood of adaptation. I remember the old days in a large network with Novell's Border Mangler ...errr Border Manager....and setting up proxy addresses gets old after the first 2 machines.

TIA

[dolson11]

I have an enhancement request open on this.
If you want to help pile on the votes for this feature go to
http://bugzilla.untangle.com/show_bug.cgi?id=4015
and vote for “Bug# 4015”

[dolson11]

The Instagate EX2 was a great little box and it would let you choose the size of your transparent cache 0Mb - 500Mb. It is only like a 1.2ghz celeron and a 20gig HD.

It helps saves so much bandwidth for AV updates and windows updates and if I have a class all going to the same webpage !!!!

http://www.whystruggle.com/esoft/downloads/IGate_EX2_052103.pdf

[priteshvarsani]

You could just put the proxy server behind the Untangle computer ? That should work fine, just set up a bog standard linux/bsd server and install squid to it and your sorted, or with a little more work you can make it transparent as well, but most large networks manually specify a proxy server and do not have a WAN facing default gateway for clients.

I do however think caching should a package within untangles system, as it is especially useful on larger networks and networks that use the internet a lot

[dolson11]

"I do however think caching should a package within untangles system, as it is especially useful on larger networks and networks that use the internet a lot"


I agree it just would be a nice add for networks that use the internet a lot.

[goplaycheckers]

i would PAY for this feature!!!

[sky-knight]

Please don't mention Instagate in here again... I started to have waking nightmares and nearly went into convulsions... the proxy system in that thing was barely functional and didn't work with windows update AT ALL! The blasted thing was the source of constant pain and the customer that owned it just "loved" it. I'm just glad I don't work for them anymore.

[dolson11]

I still have four clients with Instagates and they all work flawlessly. When they die I would like to replace them with untanlge but for the last 4 years they have worked great!!!!

[mdh]

It shouldn't take too much creativity to fix that. :D

[Compstuff]

This is my 1st post so hello to all... anyway: I was sourcing my hardware and ready to setup a customer with 6 pro systems when I ran across this post about no cache(Squid) DAMN... I rely on SQUID quite heavily so this is a problem for me. I have been using IPCop transparent proxy for years and get about a 40% hit ratio plus I use the adv proxy for updates... this is VERY useful to us.

Until this feature is available am I correct that I can use IPCop behind UT?
UT---->IPCop with UT gateway----->Clients with IPCop gateway

[Compstuff]

It shouldn't take too much creativity to fix that. :D

Is to say that we can look forward to a transparent proxy and an advanced proxy for updates?

[goplaycheckers]

Is to say that we can look forward to a transparent proxy and an advanced proxy for updates?

I think he means replacing those Instagates with UT

[Compstuff]

I think he means replacing those Instagates with UT

my bad :o

[dolson11]

I wanna replace them as soon as the cache feature is there.

[sky-knight]

AHHHHHH!!

You said that "I" word again....

/me shudders in the corner...

[dolson11]

bump

[mdh]

The cache feature is not here now nor in the next version. Bumping just ain't do much right now.

[MacMan]

As this is the most popular requested feature, hopefully it wont take to much longer to implement! :rolleyes:

[mdh]

Tell that to the users who want dual WAN...

[Ryan Q]

Tell that to the users who want dual WAN...

Lucky for me I want both equally...

[MacMan]

Tell that to the users who want dual WAN...

In the feedback forum this thread has the most views 4000+ and the most replies.

In the Untangle Bugzilla, Enhancement 4015 has 20 votes! Almost double the next closest request!

http://bugzilla.untangle.com/index.cgi

Where are these dual WAN People?

Not in the "Top 10 enhancement requests" that I can see.

:D

[mdh]

So MacMan...How many things do you want that you don't let people know about? Between forum comments and support calls, there are more requests for dual WAN, failover and load balancing than there are for caching proxies. On the forums alone, there are probably more requests for caching proxies.

[MacMan]

As should be obvious when I want something I ask for it. How else will anyone know?

No I have no idea what support calls are fore Untangle, how would I?

If dual WAN is such a sought after item why has no one put in an Enhancement Request in to Bugzilla for it?

What is the purpose of Enhancement Requests if the most sought after Enhancement isn't even listed there?

:confused:

[mdh]

Most people don't bother to ask. I wouldn't expect you to know what support calls come in, nor did I ask. Dual WAN, load balancing and failover are all discussed, but not everything has a enhancement request. You decided that this was "the most sought after" request all on your own...I never said that. And, I am on my time, and I'm done for the day.

[MacMan]

there are more requests for dual WAN, failover and load balancing than there are for caching proxies.

Dual WAN, load balancing and failover are all discussed, but not everything has a enhancement request. You decided that this was "the most sought after" request all on your own...I never said that.

You didn't? :confused:

[mdh]

No, I didn't. More requests does not mean most sought after. You see what you read in the forums. I see what I read in the forums, what comes in via email, what I hear on support calls, and what is requested by resellers and managed service providers who echo the requests of their customers. There are variations.

[sky-knight]

There's the bottom line gentleman.. The people that "buy" the product are asking for dual wan. The people doing the open source bit are asking for proxy. The money makes the decision.

Besides, dual-wan is a feature that can't readily be added to a network if the edge router doesn't have the support. Proxy can be integrated with UT trivially.

[mdh]

I'm gonna be my own devil's advocate here though, and I want to be clear that this is my perception and my perception only. A caching proxy would likely be integration of an existing open source product balanced with optimization of system resources, while dual WAN would probably be a major networking project that takes a lot of human resources. There would be likely be tradeoffs. Bottom line is who knows!

[sky-knight]

Well that too, the open source community want's a caching proxy... but I don't see any of them stepping up to the plate, making it work, and posting how they did it... It is precisely the kind of improvement the community should handle, while the devs work on the other stuff that needs some major engineering.

[MacMan]

No, I didn't. More requests does not mean most sought after. You see what you read in the forums. I see what I read in the forums, what comes in via email, what I hear on support calls, and what is requested by resellers and managed service providers who echo the requests of their customers. There are variations.

Sorry, to me most requests and most sought after are the same thing. You seem to be walking a very fine semantic line here. I concede the point that you may be seeing more of the whole picture; my original statement should have been more specific and stated “As this is the most popular requested feature in the forums & Bugzilla”.

Still I am curious to know why this most requested feature isn’t even listed in Bugzliia? (Did I say it right this time?)

P.S. don’t you guys ever sleep?

[dolson11]

Wow so much drama over just a simple request. Who cares which one is the most wanted. I would love to have both of them. I would just think that since the Cache proxy is wanted the most on the forums and should be pretty easy to implement compared to the dual wan and load balancing that the administration would at least consider it. I have 10+ clients that would switch to it right away if it had a cache proxy.

[dmorris]

Still I am curious to know why this most requested feature isn’t even listed in Bugzliia? (Did I say it right this time?)


http://bugzilla.untangle.com/show_bug.cgi?id=1057

The reality is that caching is not easy to add and has arguable benefits. We don't try to author new technologies because we know we can't be the best at everything. In order to deliver a complete solution we must leverage where we can. Squid is the only viable solution and does not easily fit within the platform given its architecture. This leaves few options.

If you disagree - great! thats why we are open source. You are free to download the code and link squid into untangle. I'm sure people in the community would love you for it! :D

(as an aside, for dual wan we've already identified which technology we will probably leverage.)

[Silver Bullet]

I just wanted to throw in that implementing squid is no easy chore with Untangle. I tried it some time back and it will take a lot of work to make it work seamlessly.

:twocents:

[sky-knight]

I just wanted to throw in that implementing squid is no easy chore with Untangle. I tried it some time back and it will take a lot of work to make it work seamlessly.

:twocents:

Yes I just assumed that integrating the proxy would be a simpler task than wan load balancing.

[MacMan]

I just wanted to throw in that implementing squid is no easy chore with Untangle. I tried it some time back and it will take a lot of work to make it work seamlessly.

:twocents:

So why is it so difficult to implement? Both SmoothWall & IPCop have this feature, why not Untangle? Untangle is in my opinion a much better over all product :confused:

[hescominsoon]

The Instagate EX2 was a great little box and it would let you choose the size of your transparent cache 0Mb - 500Mb. It is only like a 1.2ghz celeron and a 20gig HD.

It helps saves so much bandwidth for AV updates and windows updates and if I have a class all going to the same webpage !!!!

http://www.whystruggle.com/esoft/downloads/IGate_EX2_052103.pdf
if you have a windows server you can install WSUS(windows software update services) for free which does a much better job at caching windows updates than any web proxy..:)

[bk1]

I also like the idea of a proxy because like a prior poster mentioned, lab settings in school often access the same site multiple times. A proxy should really speed that up.

[whatuusay1]

I'll throw in my hat as well.. Web Proxy get a big thumps up from me. I'd also like to see real time stats.. more so than the small graphs on the rack panel. I'd love to see a Stats rack item that displays the box stats (HD space, CPU, NIC traffic, SPAM, Spyware, Virus VPN connections etc. I love the reports but it would be amazing to ge a look at the realtime stats of your box.

[Compstuff]

I too would like to throw my hat in the ring for the proxy cache... even just an idea as to if this is being considered would be helpful since I am holding off implementing about 10 untangle boxes due to the lack of this feature... THANKS in advance

[wtangle]

I come across a platform that intergrates a proxy along it's other offerings. If only Untangle could add a proxy. It would be excellent since I consider UT far better than ebox-platform as a Network Gateway.

[MacMan]

If you want this feature go to

http://bugzilla.untangle.com/show_bug.cgi?id=4015

And vote for “Bug# 4015”

:D