AES-NI and AVX instructions not detected correctly, no QAT support

[ensnare]

I've noticed some low-hanging fruits that can greatly increase Untangle performance.

First, on Untangle 16.2 AES and AVX instructions are not detected correctly.

[root @ gw01] ~ # dmesg | grep AES
[   74.330866] AVX or AES-NI instructions are not detected.
[   74.350067] AVX or AES-NI instructions are not detected.

I am running a Intel(R) Atom(TM) CPU C3758 @ 2.20GHz

Second, QuickAssist accelerators are detected but not enabled in OpenSSL, OpenVPN, or StrongSwan:

[root @ gw01] ~ # dmesg | grep qat
[    6.127840] c3xxx 0000:01:00.0: firmware: direct-loading firmware qat_c3xxx_mmp.bin
[    6.129491] c3xxx 0000:01:00.0: firmware: direct-loading firmware qat_c3xxx.bin
[    6.386857] c3xxx 0000:01:00.0: qat_dev0 started 6 acceleration engines

[root @ gw01] ~ # openssl engine
(rdrand) Intel RDRAND engine
(dynamic) Dynamic engine loading support

[root @ gw01] ~ # openssl speed -elapsed -evp aes-128-cbc
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-cbc for 3s on 16 size blocks: 54138103 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 64 size blocks: 21740710 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 6167867 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 1645871 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 209545 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 104911 aes-128-cbc's in 3.00s
OpenSSL 1.1.1d  10 Sep 2019
built on: Mon Dec  7 20:44:45 2020 UTC
options:bn(64,64) rc4(16x,int) des(int) aes(partial) blowfish(ptr)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-CKx7Fo/openssl-1.1.1d=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc     288736.55k   463801.81k   526324.65k   561790.63k   572197.55k   572953.94k

Enabling these two features could greatly increase performance with existing hardware.

[jcoffin]

AES-NI is enable but maybe not for your processor. But enabling Intel AES-NI engine in the SSL engine is an idea worth looking into.

[root @ untangle] ~ # lscpu | grep Model
Model: 63
Model name: Intel(R) Xeon(R) CPU E5-2609 v3 @ 1.90GHz

[root @ untangle] ~ # sort -u /proc/crypto | grep module
module : aesni_intel
module : aes_x86_64
module : crc32c_intel
module : crc32_pclmul
module : crct10dif_pclmul
module : cryptd
module : ghash_clmulni_intel
module : kernel

[ensnare]

It would be fantastic to have OpenSSL support for QuickAssist and AES-NI as these are the two hardware accelerators integrated in most server-grade intel processors.

This appears to be an issue with the older 4.19 kernel. What happens if I manually update kernel to 5.10 -- understood this is an unsupported configuration, but would it work? Any special handling? Thank you.

[sky-knight]

Untangle has a custom kernel, you can't just compile a new one. I have no idea if Untangle's customizations are compatible with the 5.0 kernel either, more than likely not.

Debian has said the 5.x kernels will be in Debian 11. So as far as this specific issue is concerned, you'll have to wait for A.) Debian to release Bullseye, and B.) Untangle to update to it.

[ensnare]

I have no idea

Me neither. I found an old guide on rebuilding the kernel. Would be great to have an updated one! https://wiki.untangle.com/index.php/...ntangle_kernel

[donhwyo]

You (or Untangle) may want to look at what proxmox does. It is also built on Debian buster. They use a ubuntu kernel and add there modifications to that. Seems to be stable. Ubuntu does update the kernel frequently even on the lts versions. I have seen a few other projects do similar.
Linux 5.4.78-2-pve #1 SMP PVE 5.4.78-2 (Thu, 03 Dec 2020 14:26:17 +0100)

FYI