Page 1 of 2 12 LastLast
Results 1 to 10 of 11
  1. #1
    Newbie
    Join Date
    Sep 2021
    Posts
    4

    Default Logging / Reports stops peroiodically at midnight?

    Hi!

    I've got about 3 weeks into my Untangle NGFW experience and so far it's been great but I've found a weird issue I can't seem to figure out.

    Every few days - 2~4 but I haven't logged specifically how long, reporting seems to crash and nothing gets logged until I disable Reports app and restart it. If I don't catch it, it shows no data in the dashboard; if I do, it shows a flat line after midnight on all activity even though I know there is some.

    Data retention is set to 7 days.

    Any setting I should be checking on why this might happen?

    Thanks!

  2. #2
    Untangle Ninja sky-knight's Avatar
    Join Date
    Apr 2008
    Location
    Phoenix, AZ
    Posts
    26,241

    Default

    You should run some hardware diags, because that sounds like report's evening maintenance is causing the system to crash.

    This only happens if you have a defective disk, or defective RAM.
    Rob Sandling, BS:SWE, MCP
    NexgenAppliances.com
    Phone: 866-794-8879 x201
    Email: support@nexgenappliances.com

  3. #3
    Newbie
    Join Date
    Sep 2021
    Posts
    4

    Default

    Hmmm.. it's running in a Proxmox VM which is going to make it harder to pin down if it's hardware. Thanks for the insight!

  4. #4
    Untangler
    Join Date
    May 2008
    Posts
    428

    Default

    Quote Originally Posted by gbert View Post
    Hmmm.. it's running in a Proxmox VM which is going to make it harder to pin down if it's hardware. Thanks for the insight!
    Actually it is easier. Make sure your vm config is optimized. It would be nice if Untangle supported proxmox. Here is my vm.conf file. If anybody sees something I have missed please let us know.

    Code:
    root@pve:~# cat /etc/pve/qemu-server/707.conf
    #16.3.2
    #Proxmox nic disable firewal
    #cpu = host
    #ags=+ibpb,+amd-ssbd,+amd-no-ssb,+pdpe1gb,+aes
    #remove swap
    #    fix resume delay
    #    https://lists.debian.org/debian-user/2017/09/msg00866.html
    #
    agent: 1,fstrim_cloned_disks=1
    bios: seabios
    boot: order=virtio0;scsi2
    cores: 8
    cpu: host,flags=+ibpb;+pdpe1gb;+aes
    memory: 8192
    name: Untangle16.3.2
    net0: virtio=AE:CE:0B:E8:xx:51,bridge=vmbr1,queues=8
    net1: virtio=6E:6D:34:F4:xx:6A,bridge=vmbr0,queues=8
    net2: virtio=AA:B2:46:17:xx:9F,bridge=vmbr0,link_down=1,queues=8
    numa: 1
    onboot: 1
    ostype: l26
    rng0: source=/dev/urandom
    scsi2: none,media=cdrom
    scsihw: virtio-scsi-single
    smbios1: uuid=6ec1e32b-4986-43f8-9c31-xxx...
    sockets: 1
    startup: order=1
    virtio0: R10-4xssd:vm-707-disk-0,discard=on,iothread=1,replicate=0,size=32G
    vmgenid: 2a9be704-82fd-41f6-bfcd-d45fc9xxx..
    Give as much resources as you can. Set the cpu type host and as many cores as you can. Give as much memory as you can. Set the nics to virtio and set the multique to the number of cores. If you can give enough memory turn off swap. If it is needed proxmox will take care of it. I don't want both os's battling over swap. Can't be good.

    A few details can go a long way in getting help. Hardware and software are sometimes useful. Connections and # of users etc.

  5. #5
    Newbie
    Join Date
    Sep 2021
    Posts
    4

    Default

    Thanks for the suggestion. I think I'm pretty close to you in terms of how I've got my VM configured.

    Give as much resources as you can. Set the cpu type host and as many cores as you can. Give as much memory as you can. Set the nics to virtio and set the multique to the number of cores. If you can give enough memory turn off swap. If it is needed proxmox will take care of it. I don't want both os's battling over swap. Can't be good.

    A few details can go a long way in getting help. Hardware and software are sometimes useful. Connections and # of users etc.
    A few details on the hardware: Qotom Q375g4 box with i7-5550U (dual core), 4x Intel I211AT NICs, 8gb RAM, 1 mSATA drive with PROMOX running, 1 2.5" SSD with VMs. As you can see from the VM config below, I assigned 5gb of RAM to the Untangle VM with 80GB of drive space. Untangle itself reports 2.89 GB ram used, with Proxmox reporting 4.5GB in use. Looking at the CPU, RAM, SWAP, and disk usage logs in Untangle, nothing is spiking when it crashes.

    Connections wise, typically 25 active users, max I've seen is 35 active users, <50 known devices.
    Sessions wise, worst case peaks on the log are 17,000, averages around 10,000 total.

    Looking back at the log, it's definitely crashing after 2 days of logging. Not sure where to go from here... assign more resources?


    Code:
    boot: order=scsi0;ide2
    cores: 2
    cpu: host,flags=+pcid;+aes
    ide2: local:iso/ngfw-untangle-16.3.2-amd64.iso,media=cdrom
    memory: 5120
    name: untangle
    net0: virtio=4E:76:68:19:D2:5C,bridge=vmbr1,firewall=1
    net1: virtio=96:D1:1C:D9:CF:53,bridge=vmbr2,firewall=1
    numa: 0
    onboot: 1
    ostype: l26
    scsi0: vmdrive:vm-100-disk-0,size=80G,ssd=1
    scsihw: virtio-scsi-pci
    smbios1: uuid=85cdbcd5-a771-4b36-a4e4-93d6f28778a8
    sockets: 1
    vmgenid: edccfa9c-897c-482c-af93-0d0bb90bf16a

  6. #6
    Untangler
    Join Date
    May 2008
    Posts
    428

    Default

    Proxmox 7? The main difference I see is numa = 1. Not sure if that would make a difference. The other thing I would do is setup "virtio rng" if you haven't already. Take a snapshot and try disabling swap.

    Performance might improve if on the nics you set mulique to the number of core you give the vm. Probably nothing to do with the crash though.

  7. #7
    Newbie
    Join Date
    Sep 2021
    Posts
    4

    Default

    Thanks @donhwyo ! I'll give that a try and report back. Worst case, I try making a new VM and seeing whether it makes a difference.

    Stupid question: is disabling swap just running swapoff and removing it from fstab or do I have to do something else within Untangle?

  8. #8
    Untangler
    Join Date
    May 2008
    Posts
    428

    Default

    Quote Originally Posted by gbert View Post
    Stupid question: is disabling swap just running swapoff and removing it from fstab or do I have to do something else within Untangle?
    Yes that is the way. If you get a delay on booting look at this.
    Code:
    swap fix resume delay https://lists.debian.org/debian-user/2017/09/msg00866.html
    Other optimizations. Proxmox nic disable firewall. Not sure about wan nic might be good? I did disable on the wan too so it is your call. I also install "qemu-guest-agent mlocate and mtr" just wget and dpkg -i. mtr is a better traceroute.

  9. #9
    Untangler
    Join Date
    Mar 2018
    Location
    Toronto, Ontario
    Posts
    59

    Default

    Quote Originally Posted by gbert View Post
    Thanks @donhwyo ! I'll give that a try and report back. Worst case, I try making a new VM and seeing whether it makes a difference.

    Stupid question: is disabling swap just running swapoff and removing it from fstab or do I have to do something else within Untangle?
    Don't recommend to disable swap. It's there for a reason, and if by a one time event that physical RAM is exhausted and you don't have swap, Linux will invoke OOM(Out of memory killer) and kill some random process which will make your untangle in an unstable state.

    I've personally seen Untangle take 800MB of swap few months ago for version 16.1.x as a one time event. Have not seen that issue for the latest 16.3.x.

    A better way is to change Untangle swappiness behavior from default 60 to say 10. Which means swap will be invoked only when RAM is 90% full. In /etc/sysctl.conf file, add the following line: vm.swappiness=10

  10. #10
    Untangler
    Join Date
    May 2008
    Posts
    428

    Default

    Quote Originally Posted by balrog View Post
    Don't recommend to disable swap. It's there for a reason, and if by a one time event that physical RAM is exhausted and you don't have swap, Linux will invoke OOM(Out of memory killer) and kill some random process which will make your untangle in an unstable state.

    I've personally seen Untangle take 800MB of swap few months ago for version 16.1.x as a one time event. Have not seen that issue for the latest 16.3.x.

    A better way is to change Untangle swappiness behavior from default 60 to say 10. Which means swap will be invoked only when RAM is 90% full. In /etc/sysctl.conf file, add the following line: vm.swappiness=10
    Are you using Untangle as a proxmox vm? If it was bare metal I would not disable swap. The proxmox optimizations I have read are the reason I disable swap to keep the os's from swapping twice. I have plenty of ram and give 8GB to Untangle. I have never seen it much above 4GB. Proxmox memory usage is about %70 and swap usage usually shows %0 or sometimes a few megabytes. So it works for me. Your mileage may differ.

    I like the swappiness sugestion, thanks.

Page 1 of 2 12 LastLast

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

SEO by vBSEO 3.6.0 PL2