40GBE Mellanox conectX-3 supported? Other QSFP+ NIC alternatives?

[Pyroteq]

Does Untangle support 40 gigabit QSFP+ cards such as Mellanox conectX-3?

eBay has a tonne of these cards: Mellanox ConnectX-3 CX354A that support dual 40GBE QSFP+ ports which can be had for cheap.

I figure this would be a great way to get cheap 10GBE connectivity to my servers with the QSFP+ to SFP+ breakaway cables.

Otherwise, is the CX311A single port 10GBE SFP+ version supported?

I've seen these cards mentioned frequently on places like /r/homelab and ServeTheHome but doing a search on these forums only brings up a post from 4 years ago suggesting the ConnectX-3 may not have worked at the time. Surely such a popular card would be in high demand to work with Untangle?

I'd also be interested in knowing if any special configuration would be required to work? Eg, would the QSFP+ port just appear in Untangle as 4 separate NIC's?

[sky-knight]

If it's got an Intel chip, it'll most likely work because Intel piles most of their fiber stuff into the same driver.

For the rest, you'll have to lookup what Linux kernel module the hardware requires, and then pull up an Untangle via SSH and use modinfo to see if that module is present, and what version it is. If it's there, and of sufficient version, yes it'll work.

[Pyroteq]

Chip is Mellanox pretty sure.

https://forums.untangle.com/hardware...supported.html

According to this thread another user was able to get their ConnectX-3 card to show in SSH, but nothing in Untangle.

User wasn't specific in which model card they had, regardless this thread was from 2018 and on an older version of Debian, so things might have changed since then as Untangle now runs on a later Debian version.

Hoping to get a confirmation before I pull the trigger.

[dashpuppy]

Why do you need 40gig ? Or you just doing this for fun to see if it will work ?

[sky-knight]

And yet, that other thread didn't post any detail. What did it mean? Show in SSH?

lspci can list it, but that's just a PCI Device list, if you lack the appropriate kernel module it's useless. There is nothing special to get the NIC working in Untangle, Linux just needs to be able to use it. That means possibly compiling a kernel module, or you wait until we're on the next version of Debian for more options.

I told you above how to "confirm" the card's compatibility. That's the best answer you're going to get, as there is no official HCL for Debian.

[balrog]

Does Untangle support 40 gigabit QSFP+ cards such as Mellanox conectX-3?

eBay has a tonne of these cards: Mellanox ConnectX-3 CX354A that support dual 40GBE QSFP+ ports which can be had for cheap.

I figure this would be a great way to get cheap 10GBE connectivity to my servers with the QSFP+ to SFP+ breakaway cables.

Otherwise, is the CX311A single port 10GBE SFP+ version supported?

I've seen these cards mentioned frequently on places like /r/homelab and ServeTheHome but doing a search on these forums only brings up a post from 4 years ago suggesting the ConnectX-3 may not have worked at the time. Surely such a popular card would be in high demand to work with Untangle?

I'd also be interested in knowing if any special configuration would be required to work? Eg, would the QSFP+ port just appear in Untangle as 4 separate NIC's?

Untangle is supported on official hardware, otherwise no. However, connectx-3 is supported in Linux. That out of the way, Current untangle is based on Debian 10. The kernel module drivers is there (mlx4_* and ib_*) , but not enabled.

You'll have to update the connectx-3 to the latest firmware and choose the route of going for ethernet or infiniband(default).

IMHO the card is cheap, i get that. But then you you can't reach 10Gb ethernet or 40Gb infiniband (both needs tuning), and you need beefy hardware to support the speed, kernel modules may not be there on future untangle, add cables, dacs and the switch... invested time and money adds up.

If you really want to do it, use proxmox + connectx-3 and virtualize untangle on it and call it a day.

[Pyroteq]

IMHO the card is cheap, i get that. But then you you can't reach 10Gb ethernet or 40Gb infiniband (both needs tuning), and you need beefy hardware to support the speed, kernel modules may not be there on future untangle, add cables, dacs and the switch... invested time and money adds up.

Already have a Juniper EX3300-48P I scored off eBay coming in the next couple of days with 48 PoE+ ports and 4 10Gbit SFP+ slots.

My router is a Xeon E3-1220 v3 @ 3.1Ghz with 8GB RAM so hardware shouldn't be an issue.

If you really want to do it, use proxmox + connectx-3 and virtualize untangle on it and call it a day.

Don't like virtualising my router. I tried a few years ago with eSXi but it was really slow to boot up and made managing it much harder when the entire network depended on it being up for DHCP, DNS, etc.

Why do you need 40gig ? Or you just doing this for fun to see if it will work ?

More interested in the 10gbit actually.

In future my office is moving downstairs, so I didn't want multiple PC's + wireless devices fighting over a 1gbit backhaul to the server rack. The ability to add 10G to my NAS would be a bonus too.

If what I'm trying to do isn't possible at this time or just too many hoops to jump through to activate the drivers in the kernal, is the HP 560SFP+ supported by Untangle?

There's second hand ones available on eBay with dual SFP+ slots for $120 AUD

https://www.ebay.com.au/itm/16499429...frcectupt=true

I think that uses the Intel x560 chipset?

I've seen lots of fakes from China on eBay so going for the HP versions seems like I'm more likely to get a real one.

[defcomllc]

Yes just use the HP Intel 560 dual SFP+ cards...I use them on lots on UT boxes when I need SFP+

[jcoehoorn]

Don't like virtualising my router.

I totally understand this. Virtualization does have some downsides, especially on shared hardware and especially for the router. It's so easy for an unrelated VM to have a busy period where the resource use can start to impact Untangle, and now suddenly your internet throughput is effected in a bad way.

But there are still benefits if you virtualize on dedicated hardware, where the Untangle VM is the only VM. This protects you from resource exhaustion caused by other VMs, it helps with hardware compatibility, backups, fleet management, etc. Really the only downside is getting the NICs assigned correctly, and that's something you usually only need to care about once, during the initial setup.

[dashpuppy]

I totally understand this. Virtualization does have some downsides, especially on shared hardware and especially for the router. It's so easy for an unrelated VM to have a busy period where the resource use can start to impact Untangle, and now suddenly your internet throughput is effected in a bad way.

But there are still benefits if you virtualize on dedicated hardware, where the Untangle VM is the only VM. This protects you from resource exhaustion caused by other VMs, it helps with hardware compatibility, backups, fleet management, etc. Really the only downside is getting the NICs assigned correctly, and that's something you usually only need to care about once, during the initial setup.

If Untangle is the only VM on the server whats the point ? Other then backups. Wouldn't it be 100% better to have it on bare metal dedicated..