How to Install all UT 7.3 -ntop, phpsysinfo, imspector, squid, monit, rsyslog, sqstat

[napa]

I TESTED EVERYTHING ON THIS POST, EVERYTHING WORKS AND DOES NOT AFFECT ANY OTHER MODULE OR CODE IN THE UNTANGLE SYSTEM, MY SERVER IS ON PRODUCTION WITH MORE THAN 150 ACTIVE USERS EVERYDAY AND PERFORMANCE IS EXCELLENT. USE IT AT YOUR OWN RISK.


Changes:
-Ntop: now it works fine, does not reboot everytime.
-Webmin: has no sense to install webmin, I never used to do nothing so is not installed now.
-Phpsysinfo: easier setup thanks to installed modules from UT 7.3
-Imspector: now it depends from phpsysinfo web not from webmin.
-Squid: improved security, just for users you want not to pass thru webfilter, and port not standard.
-Monit: it’s already installed on 7.3 so I just configured with our apps.
-Sqstat: changed SRG and Sarg by SQstat

Added
-start ssh everytime
-logs error messages from bootscreen
-Rsyslog alerts by mail credits to: bigdessert
-Remote access to UT database credits to: Sky-Knight
-Added cleaning files for database, files, and squid

Previous post:
http://forums.untangle.com/hacks/964...test-rels.html


1. Start SSH (start SSH Service)

rm /etc/ssh/sshd_not_to_be_run && /etc/init.d/ssh start

2. Logs de Boot (Logs all boot errors and information)

a. Edit /etc/default/bootlogd with: BOOTLOGD_ENABLE=Yes
b. Logs to: /var/log/boot.

3. Install NTOP

a) Increase the APT cache: /etc/apt/apt.conf or /etc/apt/apt.conf/70debconf

Add line: APT::Cache-Limit 30000000;

b) Uncomment lenny sources in /etc/apt/sources.list
c) Clean & Update Apt Cache

apt-get clean && apt-get update

d) Install ntop

apt-get install ntop

e) Set nTOP's admin password

ntop –A

f) Configure nTOP monitoring interfaces

echo 'USER="ntop"\nINTERFACES="eth0,eth1,eth2"' >| /var/lib/ntop/init.cfg

g) Configure nTOP to use TCP port 4000

echo 'GETOPT="-w 4000"' >> /etc/default/ntop

h) Start nTOP

/etc/init.d/ntop start

i) Confirm nTOP is listening on TCP port 4000

lsof -i | grep ntop

j) Configure a packet filter for nTOP (Packet Filter)

Port:4000 & Destination Local

k) Access NTOP

http://untangleip:4000

4. Install PHPSYSINFO

a) Edit the file /etc/apache2/sites-available/uvm to allow PHP and CGI files

<Directory /var/www>
Options +ExecCGI
AddHandler cgi-script .cgi
AddHandler fcgid-script .php
FCGIWrapper /usr/lib/cgi-bin/php5 .php
FileETag Mtime Size
</Directory>

b) Install PhpSysInfo

apt-get install phpsysinfo

c) Access PhpMyInfo: http://untangleip/phpsysinfo

5. Install Imspector

a) Install Dependencies

apt-get install make build-essential openssl libssl-dev

b) Install Imspector

cd /tmp
wget http://www.imspector.org/downloads/imspector-0.9.tar.gz
tar zxf imspector-0.9.tar.gz
cd imspector-0.9
make
make install
make install-cert

c) Create Folder

mkdir /etc/imspector

d) Create Dependency files
Add to start and create /etc/init.d/imspector with:

#! /bin/sh
# IMspector startup
PATH=/sbin:/bin:/usr/sbin:/usr/bin
echo "Starting IMspector..."

imspector -c /etc/imspector/imspector.conf
chmod 755 /etc/init.d/imspector
update-rc.d imspector defaults
mv /usr/etc/imspector/imspector.conf /etc/imspector/imspector.conf
mv /usr/etc/imspector/badwords.txt /etc/imspector/badwords.txt
mv /usr/etc/imspector/acl.txt /etc/imspector/acl.txt
/etc/init.d/imspector
cp /tmp/imspector-0.9/contrib/imspector.cgi to /usr/share/phpsysinfo/imspector.cgi

e) Restart Apache: /etc/init.d/apache2 restart
f) Access Imspector: http://untangleip/phpsysinfo/imspector.cgi
g) Rules in port Forwarding to Imspector

Configure a port forwarding in UT for all or users you want to monitor
Port:10000 & Destination Local
Source: Any or IP that you want.
Destination Port: 1863 (For MSN, Yahoo, Gmail, etc)
Protocol:TCP
New Destination: UT Internal eth1 IP
New Port. 16667
Configure a packet filter for squid (Packet Filter)
Port:3128& Destination Local

6. Install Squid (Just for power Users, It can’t be filtered)

a) Install squid

apt-get install squid

b) Configure squid
Edit /etc/squid/squid.conf with

acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
http_access allow all (define users who could use squid here)
acl_uses_indirect_client off
http_port 55555 transparent (uses no standar port so users can't find it)
cache_mem 32 MB
cache_dir ufs /var/spool/squid 5000 16 256
access_log /var/log/squid/access.log squid
cache_mgr admin@domain.com
httpd_suppress_version_string on

c) Reestart squid: /etc/init.d/squid reestart
d) Rules in port Forwarding & Packet Filter to Squid

Configure a port forwarding in UT for all or users you want to monitor
Source Address: any or IP you want to pass thru squid
Destination Port. 80
Protocol:TCP
New Address: INTERNALUTADDRESS
New Port. 55555
Configure a packet filter for squid (Packet Filter)
Port:55555& Destination Local

7. Install Monit (Just need to get configured with our info, it’s already installed on 7.3)

a) Add to file:/etc/untangle/monit.d/monit-base_all.conf

set daemon 120
set logfile /var/log/uvm/monit.log
set mailserver mail.server.com port 587 username "admin@domain.com" password "adminpwd"
set alert mail1@domain.com only on { timeout, nonexist, connection }
set alert mail2@domain.com only on { timeout, nonexist, connection }
set httpd port 2812 and use address localhost
allow 192.168.80.0/24
allow user:password
allow localhost


#MONITOR APACHE
check process apache with pidfile /var/run/apache2.pid
start program = "/etc/init.d/apache2 start"
stop program = "/etc/init.d/apache2 stop"
if failed host 192.168.200.254 port 80 protocol http for 2 cycles then restart
if 5 restarts within 10 cycles then timeout

#MONITOR NTOP
check process ntop with pidfile /var/run/ntop.pid
start program = "/etc/init.d/ntop start"
stop program = "/etc/init.d/ntop stop"
if failed host 192.168.200.254 port 4000 then restart
# if 5 restarts within 5 cycles then timeout

#MONITOR SSHD
check process sshd with pidfile /var/run/sshd.pid
start program "/etc/init.d/ssh start"
stop program "/etc/init.d/ssh stop"
if failed port 22 protocol ssh for 2 cycles then restart
if 5 restarts within 10 cycles then timeout

#MONITOR SQUID
check process squid with pidfile /var/run/squid.pid
start program "/etc/init.d/squid start"
stop program "/etc/init.d/squid stop"
if failed host 192.168.200.254 port 3128 then restart
if 5 restarts within 10 cycles then timeout

#MONITOR IMSPECTOR
check process imspector with pidfile /var/run/imspector.pid
start program "/etc/init.d/imspector start"
stop program "/etc/init.d/imspector stop"
if failed host 192.168.200.254 port 16667 then restart
if 5 restarts within 10 cycles then timeout

b) Restart /etc/init.d/untangle-vm (this restarts monit)

8. Install TCPTRACK & DSNIFF

apt-get install tcptrack
apt-get install dsniff

a) UT 7.3 has already iftop & jnettop

9. Install Rsyslog alerts (Just need to get configured with our info, it’s already installed on 7.3)
Credits for Bigdessert

a) Edit /etc/rsyslog.conf
Under #### MODULES ####

Add: $ ModLoad ommail

Under # provides UDP syslog reception, uncomment the following two lines:

#$ModLoad imudp
#$UDPServerRun 514

Under #### GLOBAL DIRECTIVES ####, add the following action
####ACTIONS####
##Note, smtp server must be able to relay mail!##

$ActionMailSMTPServer localhost
$ActionMailSMTPPort 25
$ActionMailFrom email@fromaddress.com
$ActionMailTo email@toaddress.com
$template mailSubject,"Untangle Alert On Server"
$template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"
$ActionMailSubject mailSubject
if $syslogtag contains 'Intrusion_Prevention' then :ommail:;mailBody

b) Restart rsyslog: /etc/init.d/rsyslog restart
c) Change in admin panel, under monitoring tab, enable syslog to localhost and port 514.

10. Install Sqstat for Squid (Live Stats)

Cd /tmp
wget http://samm.kiev.ua/sqstat/sqstat-1.20.tar.gz
tar xvzf sqstat-1.20.tar.gz
cp sqstat-1.20/config.inc.php.defaults sqstat-1.20/config.inc.php,
edit config.inc.php with Squid IP:127.0.0.1 and Port: 55555
mv sqstat-1.20 /usr/share/phpsysinfo/sqstat
view logs on http://untangleip/phpsysinfo/sqstat/sqstat.php

11. Allow remote access to UT database
Credits Sky-Knight

b) Add a packet filter rule with this:

Description: Pass Postgres
Action: Pass
Source Interface: Internal
Destined Local
Protocol: TCP
Destination Port: 5432

c) Stop postgress services:

/etc/init.d/untangle-vm stop
/etc/init.d/postgresql-8.3 stop

d) Edit file: /etc/postgresql/8.3/main/postgresql.conf with

listen_addresses = '*'

e) Edit file: /etc/postgresql/8.3/main/pg_hba.conf with

host all all 0.0.0.0/0 trust

f) Just restart the services.

/etc/init.d/postgresql-8.3 start
/etc/init.d/untangle-vm start

g) Get a copy of pgAdmin from www.pgadmin.org and connect using the following:

Name: Untangle
Host: Internal IP of your Untangle here
Port: 5432
SSL: allow
Maintenance DB: postgres
Username: postgres
Password: uvm

[napa]

12. Clean & Finish Setup

a) Comment Again lines from /etc/apt/sources.list
b) Clean APT: apt-get clean && apt-get update
c) Cleaning all system with 3 maintenance files:

Deletes all from database:
Cleandb.sh
/etc/init.d/untangle-vm stop && psql -U postgres uvm -c "drop schema events cascade;" && /etc/init.d/untangle-vm start

Messure the usage of any of the used directories, so we can delete something:
diskuse.sh
echo "Disk Free Status"
df -h
echo "Top 20 Tables"
psql -U postgres uvm -c "SELECT relname, reltuples, relpages FROM pg_class ORDER BY relpages DESC LIMIT 20 ;"
echo "Oldest record in database"
psql -U postgres uvm -q -t -c "SELECT time_stamp from pl_endp order by time_stamp limit 1;"
echo "Disk Use of /var/log - Top 10 files"
du -s /var/log/* | sort -g -r | head -n 10
echo "Disk Use of /var/log/uvm - Top 10 files"
du -s /var/log/uvm/* | sort -g -r | head -n 10
echo "Disk Use of postgres db files"
du -sh /var/lib/postgresql/*
echo "Disk Use of /usr/share/untangle/web/reports/ (old reports)"
du -sh /usr/share/untangle/web/reports
echo "Disk Use of /tmp"
du -sh /tmp
echo "Disk Use of /var"
du -sh /var/*

deletes and recreate the squid cache
Cleansquid.sh
squid -k shutdown
/usr/sbin/squid flush
rm -fr /var/spool/squid
mkdir /var/spool/squid
chmod 777 /var/spool/squid
squid -z
/etc/init.d/squid restart

[dmorris]

excellent write up - some very cool features!

also worth noting: if you do this TURN OFF auto-upgrade.
And also expect that you might have to reinstall if future upgrades fail.

[Mathiau]

Fantastic you took the time to do this for 7.3

Please let me know though if you are able to update your box with out it breaking, on past installs if i had ntop installed, updates would never work and i always had to redo my box as dmorris touched on.

[gotkimchi]

also, it is unsupported by the Untangle support team.

[gotkimchi]

chances are, if you are doing all those things, seems like you know what you are doing.

[Big D]

I know enough to be dangerous

[sky-knight]

I know enough to be dangerous

Best place to be and attitude to have IMHO.

[napa]

I take the latest smoothwall visor (is the same that came with standar package of imspector, and adapt the script to Untangle so know you can log and view msn and icq chats with all smilies!!

I leave you the link http://www.securitynation.com/imspec...or-untangle.7z

[proactivens]

great addition. Thanks for your time.

Why is it that these contribs. always seem to happen weeks or a month away from the next minor release? 7.4 is right around the corner. Who knows if these contirbs. will continue to work. It seems like a lot of work for a few weeks of use, unless you just stay at 7.3.1