Reports server

[Gremlin]

We currently run a Untangle 7 server (64bit) in our district's high school. While we still have plenty of issues that we are working with support with there is one thing we would like to see added to Untangle.

We are a large site and our web accessible logs fill in roughly 30 seconds on a average day. We would very much like to see a stand alone reports server. Basically a replication server that would hold its data for an extended period of time (months) and allow for reports to be run on it without affecting the main server. Also this would allow the long term storage of data for true historical reporting.

Having a separate backup/reports server would solve some of the issues we are experiencing with the new "lite" version of the reports module. Our older firewall used a similar setup which allowed us to dig into the logs while not impacting the performance of the device.

One more benefit would be a database or custom reports built-in or available on the forum/wiki. For instance you need to know every site a specific user/host has hit in the last 2 weeks or how many users have hit a specific site. Writing a quick query and posting it on the forums or wiki would allow someone to quickly access the data without having to learn the entire database structure. It would allow the administrator to choose what data he wants vs. wait for the end of day report or wait till it gets included in the reports module by default.

Having a standalone reports server/backup of event logs would defiantly add a significant amount of flexibility to Untangle's currently limited reports system. Does anyone know if this is feasible with Untangle's new database structure?

[dwasserman]

You can play with syslog and snmp traps , capturing all data and create your custom reports.
Is not easy, becuse the format is not standard, but is factible.
I know 2 products can use: GFI event log and manage engine event log , both are not free.
But any syslog server with a report generator module can do.

Please, share your experience here

[sky-knight]

Gremlin, I've been gathering information to develop exactly that for the last year. The issue preventing me from really pushing it?

I have no idea if Untangle themselves is ever going to do this. I've asked, and they say "it's on the table" and it's not wise for me to make something that competes directly with Untangle...

If you have someone around that is comfortable with SQL

http://forums.untangle.com/tip-day/1...-remotely.html

may be of interest to you. It is possible to get at the reports database manually, from there you're only a query away from getting the information yourself.

Mind you, if you're going into production you may want to research how to lock down postgres a bit more... that thread is very very insecure in the way it opens the door.

[Gremlin]

We have a SQL guy that lives and breaths the stuff; he is not me . I wanted to see if this idea has been passed around. We were thinking of a long term database since in our environment having month or more of info is needed at times. Storing that amount of data in a large database on the live server is going to cause issues when the nightly reports kick off never mind about the monthly reports. That and it seems like many people here would like to control what is in their reports. Giving them that ability on the live server is a recipe for trouble but it is a mute point on a standalone database box.

If it is "on the table" then the devs are aware so I'll drop it here.

[mrunkel]

You can play with syslog and snmp traps , capturing all data and create your custom reports.
Is not easy, becuse the format is not standard, but is factible.
I know 2 products can use: GFI event log and manage engine event log , both are not free.
But any syslog server with a report generator module can do.

Please, share your experience here

Syslog will get you system level info, but not the data out of the UVM. That isn't currently handled by syslog.

snmp traps will get you next to zero usable data.

[dwasserman]

Upsss
Wrong way

[Gremlin]

Just for kicks does anyone happen to know what version of PostgreSQL Untangle 7.1 uses? More importantly does untangle update the database engine regularly (not the database schema).

With the research I've dug up I'm pretty sure the reason we can't have stand alone report servers and a clusters of Untangle servers or anything else that would require real time database replication is because of PostgreSQL's limitations, it doesn't have a well designed replication mechanism.

PostgreSQL 9.0 "might" fix that but for now I guess I'm raining on my own parade.

[jtangle]

This is interesting:
Can anyone tell me just where the raw data used for the reports is located? If we can capture / save the data to another location we can transform it to a useful format and then dump it in to whatever database needed.

[bigdessert]

you can see some discussion about this here
http://forums.untangle.com/tip-day/1...live-data.html