Results 1 to 9 of 9
  1. #1
    Master Untangler
    Join Date
    Aug 2008
    Posts
    939

    Default Hard disk filling up, before 9.x had 80%+ free

    I believe it was 9.0, a SNMP monitoring bug caused the log files to fill up the drive. We worked with support and got a lot of it removed, but not all. Since that time, the drive has been filling up. Currently we are 31% free.

    Looking at /var/log I have removed the *.gz files.

    Looking at /var/log/uvm I find some VERY huge log files. Is this normal?

    -rw-r--r-- 1 root adm 5120068 2011-04-10 03:55 35.log.1
    -rw-r--r-- 1 root adm 5120081 2010-11-25 15:29 35.log.2
    -rw-r--r-- 1 root adm 5120021 2010-07-09 02:19 35.log.3
    -rw-r--r-- 1 root adm 5120009 2010-02-18 16:47 35.log.4
    -rw-r--r-- 1 root adm 5128441 2011-06-30 11:19 4.log.1
    -rw-r--r-- 1 root adm 5123460 2011-06-20 17:51 4.log.2
    -rw-r--r-- 1 root adm 5121435 2011-06-15 08:04 4.log.3
    -rw-r--r-- 1 root adm 5121773 2011-06-07 10:44 4.log.4
    -rw-r--r-- 1 root adm 5121407 2011-05-26 08:56 4.log.5
    -rw-r--r-- 1 root adm 5126183 2011-05-17 13:30 4.log.6
    -rw-r--r-- 1 root adm 5124978 2011-05-09 08:06 4.log.7
    -rw-r--r-- 1 root adm 5122951 2011-04-29 12:01 4.log.8
    -rw-r--r-- 1 root adm 5127143 2011-04-25 09:56 4.log.9
    -rw-r--r-- 1 root adm 5120051 2011-05-23 02:53 62.log.1
    -rw-r--r-- 1 root adm 5120523 2011-03-20 09:51 62.log.2
    -rw-r--r-- 1 root adm 5121163 2011-01-23 14:23 62.log.3
    -rw-r--r-- 1 root adm 108219 2011-11-30 10:19 apt.log
    -rw-r--r-- 1 root adm 476660 2011-11-30 10:21 console.log
    -rw-r----- 1 root adm 126787134 2011-11-11 09:35 events.log
    -rw-r--r-- 1 root adm 1749181522 2011-11-30 10:24 gc.log
    -rw-r--r-- 1 root adm 78454 2011-06-28 11:31 iptables.log
    -rw-rw-r-- 1 root adm 2863659 2011-11-30 10:24 monit.log
    -rw-r----- 1 root adm 743879 2011-11-30 08:11 node-13.log
    -rw-r----- 1 root adm 322 2011-10-14 13:08 node-17.log
    -rw-r----- 1 root adm 11105 2011-07-21 13:45 node-18.log
    -rw-r----- 1 root adm 8460 2011-07-17 19:34 node-19.log
    -rw-r----- 1 root adm 2598 2011-10-21 15:01 node-1.log
    -rw-r----- 1 root adm 131 2011-08-25 07:50 node-24.log
    -rw-r----- 1 root adm 6230688 2011-11-30 10:11 node-35.log
    -rw-r----- 1 root adm 771984 2011-11-30 08:12 node-36.log
    -rw-r----- 1 root adm 1394922 2011-11-29 07:55 node-38.log
    -rw-r----- 1 root adm 24464313 2011-11-29 07:55 node-39.log
    -rw-r----- 1 root adm 2957 2011-09-30 12:30 node-3.log
    -rw-r----- 1 root adm 768902 2011-11-29 19:31 node-40.log
    -rw-r----- 1 root adm 585451 2011-11-29 15:09 node-43.log
    -rw-r----- 1 root adm 739953 2011-11-30 05:20 node-44.log
    -rw-r----- 1 root adm 101207 2011-11-30 00:30 node-46.log
    -rw-r----- 1 root adm 368484 2011-11-30 03:31 node-47.log
    -rw-r----- 1 root adm 554064 2011-11-30 09:52 node-48.log
    -rw-r----- 1 root adm 69419419 2011-11-30 10:15 node-4.log
    -rw-r----- 1 root adm 2902 2011-11-14 14:41 node-51.log
    -rw-r----- 1 root adm 756261 2011-11-30 08:12 node-52.log
    -rw-r----- 1 root adm 1138 2011-10-06 13:06 node-54.log
    -rw-r----- 1 root adm 74871 2011-11-02 08:29 node-55.log
    -rw-r----- 1 root adm 47362 2011-11-21 10:40 node-56.log
    -rw-r----- 1 root adm 25851 2011-11-30 01:52 node-5.log
    -rw-r----- 1 root adm 16184 2011-11-30 07:59 node-60.log
    -rw-r----- 1 root adm 84989 2011-11-29 16:30 node-61.log
    -rw-r----- 1 root adm 872805 2011-11-30 05:45 node-62.log
    -rw-r----- 1 root adm 15654 2011-11-28 08:41 node-63.log
    -rw-r----- 1 root adm 88417 2011-08-30 08:01 node-7.log
    -rw-r----- 1 root adm 243 2011-10-14 13:08 node-81.log
    -rw-r----- 1 root adm 9639899808 2011-11-30 10:24 node-82.log
    -rw-r----- 1 root adm 516 2011-07-21 13:45 node-83.log
    -rw-r----- 1 root adm 308052 2011-11-30 08:03 node-84.log
    -rw-r----- 1 root adm 196 2011-07-05 15:15 node-85.log
    -rw-r----- 1 root adm 196 2011-07-05 15:16 node-86.log
    -rw-r----- 1 root adm 213 2011-09-22 16:49 node-89.log
    -rw-r----- 1 root adm 345 2011-09-22 16:28 node-90.log
    -rw-r----- 1 root adm 3959 2011-07-24 03:11 node-92.log
    -rw-r----- 1 root adm 10684524737 2011-11-30 10:24 nodes.log
    -rw-r--r-- 1 root adm 41764 2011-10-04 08:46 openvpn.log
    -rw-r--r-- 1 root adm 155133 2011-11-30 03:01 reporter.log
    -rw-r--r-- 1 root adm 512009 2009-09-14 02:08 reporter.log.2
    -rw-r--r-- 1 root adm 512019 2009-08-31 02:43 reporter.log.3
    -rw-r--r-- 1 root adm 512097 2009-08-16 02:28 reporter.log.4
    -rw-r--r-- 1 root adm 512122 2009-08-02 01:57 reporter.log.5
    -rw-r--r-- 1 root adm 512071 2009-07-18 02:32 reporter.log.6
    -rw-r--r-- 1 root adm 512032 2009-07-03 02:32 reporter.log.7
    -rw-r--r-- 1 root adm 119051 2010-03-11 15:38 report.log
    -rw-r--r-- 1 root adm 3245 2010-08-17 10:35 restore.log
    -rw-r--r-- 1 root adm 0 2011-07-25 01:43 schema.log
    -rw-r--r-- 1 root adm 182 2009-06-19 08:42 schema.log.0
    -rw-r--r-- 1 root adm 180 2009-06-19 08:42 schema.log.2
    -rw-r--r-- 1 root adm 574 2009-06-19 08:42 schema.log.3
    -rw-r--r-- 1 root adm 181 2009-06-19 08:42 schema.log.4
    -rw-r--r-- 1 root adm 178 2009-06-19 08:42 schema.log.5
    -rw-r--r-- 1 root adm 179 2009-06-19 08:42 schema.log.6
    -rw-r--r-- 1 root adm 179 2009-06-19 08:42 schema.log.7
    -rw-r--r-- 1 root adm 1065032 2009-06-19 08:42 schema.log.8
    -rw-r--r-- 1 root adm 14357665 2011-11-30 10:24 uvm.log
    -rw-r--r-- 1 root adm 526018 2011-06-29 07:32 uvm.log.1
    -rw-r--r-- 1 root adm 512094 2011-06-23 01:37 uvm.log.2
    -rw-r--r-- 1 root adm 512013 2011-06-14 00:39 uvm.log.3
    -rw-r--r-- 1 root adm 512097 2011-06-04 03:51 uvm.log.4
    -rw-r--r-- 1 root adm 512035 2011-05-25 02:04 uvm.log.5
    -rw-r--r-- 1 root adm 512072 2011-05-14 13:57 uvm.log.6
    -rw-r--r-- 1 root adm 512056 2011-05-05 11:10 uvm.log.7
    -rw-r--r-- 1 root adm 512091 2011-04-26 15:18 uvm.log.8
    -rw-r--r-- 1 root adm 512097 2011-03-22 17:50 uvm.log.9
    -rw-r--r-- 1 root adm 398169 2011-11-30 10:24 wrapper.log

  2. #2
    Master Untangler
    Join Date
    Aug 2008
    Posts
    939

    Default

    I took a peak into nodes.log (huge to look through), and I find endless entries that look like this:

    Jul 5 15:26:02 localhost node-82: [SplitDImpl] <NT1876832420> WARN No blinger found for interface: 2
    Jul 5 15:26:03 localhost node-82: [SplitDImpl] <NT1876832421> WARN No blinger found for interface: 2
    Jul 5 15:26:03 localhost node-82: [SplitDImpl] <NT1876832422> WARN No blinger found for interface: 2
    Jul 5 15:26:03 localhost node-82: [SplitDImpl] <NT1876832423> WARN No blinger found for interface: 2
    Jul 5 15:26:03 localhost node-82: [SplitDImpl] <NT1876832424> WARN No blinger found for interface: 2
    Jul 5 15:26:03 localhost node-82: [SplitDImpl] <NT1876832425> WARN No blinger found for interface: 2
    Jul 5 15:26:03 localhost node-82: [SplitDImpl] <NT1876832426> WARN No blinger found for interface: 2
    Jul 5 15:26:04 localhost node-82: [SplitDImpl] <NT1876832427> WARN No blinger found for interface: 2
    Jul 5 15:26:04 localhost node-82: [SplitDImpl] <NT1876832428> WARN No blinger found for interface: 2
    Jul 5 15:26:04 localhost node-82: [SplitDImpl] <NT1876832429> WARN No blinger found for interface: 2
    Jul 5 15:26:04 localhost node-82: [SplitDImpl] <NT1876832430> WARN No blinger found for interface: 2
    Jul 5 15:26:04 localhost node-82: [SplitDImpl] <NT1876832431> WARN No blinger found for interface: 2
    Jul 5 15:26:04 localhost node-82: [SplitDImpl] <NT1876832432> WARN No blinger found for interface: 2
    Jul 5 15:26:05 localhost node-82: [SplitDImpl] <NT1876832433> WARN No blinger found for interface: 2
    Jul 5 15:26:05 localhost node-82: [SplitDImpl] <NT1876832434> WARN No blinger found for interface: 2
    Jul 5 15:26:05 localhost node-82: [SplitDImpl] <NT1876832435> WARN No blinger found for interface: 2
    Jul 5 15:26:05 localhost node-82: [SplitDImpl] <NT1876832436> WARN No blinger found for interface: 2
    Jul 5 15:26:05 localhost node-82: [SplitDImpl] <NT1876832437> WARN No blinger found for interface: 2
    Jul 5 15:26:05 localhost node-82: [SplitDImpl] <NT1876832438> WARN No blinger found for interface: 2
    Jul 5 15:26:05 localhost node-82: [SplitDImpl] <NT1876832439> WARN No blinger found for interface: 2

  3. #3
    Untangle Ninja sky-knight's Avatar
    Join Date
    Apr 2008
    Location
    Phoenix, AZ
    Posts
    26,510

    Default

    Code:
    curl http://www.untangle.com/download/patches/generic/diskuse.sh | dash
    Past the output here.
    Rob Sandling, BS:SWE, MCP
    NexgenAppliances.com
    Phone: 866-794-8879 x201
    Email: support@nexgenappliances.com

  4. #4
    Master Untangler
    Join Date
    Aug 2008
    Posts
    939

    Default

    Thanks sky. Here is the result:

    ~ # curl http://www.untangle.com/download/pat...ric/diskuse.sh | dash
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed
    100 793 100 793 0 0 5685 0 --:--:-- --:--:-- --:--:-- 0
    Disk Free Status
    Filesystem Size Used Avail Use% Mounted on
    /dev/sda1 71G 45G 24G 66% /
    tmpfs 1.5G 0 1.5G 0% /lib/init/rw
    udev 10M 692K 9.4M 7% /dev
    tmpfs 1.5G 0 1.5G 0% /dev/shm
    /dev/sda5 16M 1.2M 14M 8% /data
    Top 20 Tables
    relname | reltuples | relpages
    ---------------------+-------------+----------
    pl_stats | 2.74612e+06 | 65532
    pl_endp | 2.76304e+06 | 61922
    sessions_2011_11_17 | 896808 | 35218
    n_firewall_evt | 2.45782e+06 | 34881
    sessions_2011_11_10 | 870432 | 34188
    sessions_2011_11_29 | 870455 | 33113
    sessions_2011_11_16 | 829379 | 32846
    sessions_2011_11_28 | 843072 | 32493
    sessions_2011_11_14 | 820523 | 32072
    sessions_2011_11_21 | 845622 | 31957
    sessions_2011_11_15 | 811962 | 31945
    sessions_2011_11_22 | 826167 | 31891
    sessions_2011_11_09 | 804723 | 31589
    sessions_2011_11_18 | 803483 | 31150
    sessions_2011_11_11 | 788747 | 31044
    sessions_2011_11_23 | 747033 | 30009
    sessions_2011_11_24 | 740797 | 29884
    sessions_2011_11_12 | 729990 | 29164
    pl_endp_sid_idx | 2.76304e+06 | 28988
    sessions_2011_11_13 | 739484 | 28973
    (20 rows)

    Oldest record in database
    2011-11-27 00:00:00.01

    Disk Use of /var/log - Top 10 files
    21949464 /var/log/uvm
    19840 /var/log/apache2
    5636 /var/log/auth.log.1
    3752 /var/log/atop.log.9
    3596 /var/log/atop.log.2
    3552 /var/log/atop.log.14
    3540 /var/log/atop.log.8
    3520 /var/log/atop.log.10
    3492 /var/log/atop.log.6
    3448 /var/log/atop.log.1
    Disk Use of /var/log/uvm - Top 10 files
    10459552 /var/log/uvm/nodes.log
    9438316 /var/log/uvm/node-82.log
    1712792 /var/log/uvm/gc.log
    123944 /var/log/uvm/events.log
    67956 /var/log/uvm/node-4.log
    23920 /var/log/uvm/node-39.log
    14056 /var/log/uvm/uvm.log
    6104 /var/log/uvm/node-35.log
    5024 /var/log/uvm/4.log.1
    5020 /var/log/uvm/4.log.9
    Disk Use of postgres db files
    13G /var/lib/postgresql/8.3
    Disk Use of /usr/share/untangle/web/reports/ (old reports)
    659M /usr/share/untangle/web/reports
    Disk Use of /tmp
    7.0M /tmp
    Disk Use of /var
    4.0K /var/agentx
    3.4M /var/backups
    8.6M /var/cache
    13G /var/lib
    4.0K /var/local
    12K /var/lock
    22G /var/log
    8.0K /var/mail
    4.0K /var/opt
    220K /var/run
    3.4G /var/spool
    4.0K /var/tmp
    43M /var/www

  5. #5
    Untangle Junkie dmorris's Avatar
    Join Date
    Nov 2006
    Location
    San Carlos, CA
    Posts
    17,486

    Default

    Hey far,

    give this a try: http://wiki.untangle.com/index.php/9...sive_logrotate

    I just wrote it up, but basically it just makes the logrotates much more often (since we don't really need that much data anyway)

    Please report back about how it works for you as this will probably be the new default in future versions.
    Attention: Support and help on the Untangle Forums is provided by volunteers and community members like yourself.
    If you need Untangle support please call or email support@untangle.com

  6. #6
    Master Untangler
    Join Date
    Aug 2008
    Posts
    939

    Default

    Hi Dirk. Sorry for the delay. I just upgraded to 9.1 (was on 9.01, and your patch required 9.02). The box was production and we had to wait for a downtime window for the upgrade.

    We are still seeing the disk fill up. Is there a new patch like the one above for 9.1?

    Thank You for your support

  7. #7
    Master Untangler deleted_account+118970@untangle.com's Avatar
    Join Date
    Feb 2009
    Posts
    193

    Default

    There is, on the wiki

  8. #8

  9. #9
    Untangle Junkie dmorris's Avatar
    Join Date
    Nov 2006
    Location
    San Carlos, CA
    Posts
    17,486

    Default

    Quote Originally Posted by far182 View Post
    Hi Dirk. Sorry for the delay. I just upgraded to 9.1 (was on 9.01, and your patch required 9.02). The box was production and we had to wait for a downtime window for the upgrade.

    We are still seeing the disk fill up. Is there a new patch like the one above for 9.1?

    Thank You for your support
    Hey, yeah just email me your UID and I'll take a look right now. (and turn on remote support if it isn't already)

    The more aggressive logrotate is built into 9.1, so if your disk is filling up its something else.
    It could be the captive portal issue if this server runs captive portal and has been installed for a long time. If not, its something else.
    Attention: Support and help on the Untangle Forums is provided by volunteers and community members like yourself.
    If you need Untangle support please call or email support@untangle.com

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

SEO by vBSEO 3.6.0 PL2