Comware

Hi All,

I am having a network issue when I am installing kernel-5.4.35 in rhel7 OS running on top of the physical server DL360 Gen10.

The network interface is configured as bond0 in the server.

The interface is working with kernel-4.4 but showing offline when I am installing kernel-5.4 on top of rhel7.

~]$ ifconfig -a
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 1500
inet 10.159.90.48 netmask 255.255.254.0 broadcast 10.159.91.255
inet6 fe80::8230:e0ff:fe31:f528 prefixlen 64 scopeid 0x20<link>
ether 80:30:e0:31:f5:28 txqueuelen 1000 (Ethernet)
RX packets 12661144901 bytes 15943452290966 (14.5 TiB)
RX errors 59420 dropped 6556849 overruns 59420 frame 0
TX packets 29050276631 bytes 42082191901534 (38.2 TiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:c7:85:f4:e4 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eno1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 80:30:e0:41:43:c0 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16

eno2: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 80:30:e0:41:43:c1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17

eno3: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 80:30:e0:41:43:c2 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16

eno4: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 80:30:e0:41:43:c3 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17

eno5: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether 80:30:e0:31:f5:28 txqueuelen 1000 (Ethernet)
RX packets 12651354173 bytes 15942631431595 (14.4 TiB)
RX errors 59420 dropped 0 overruns 59420 frame 0
TX packets 29050276631 bytes 42082191901534 (38.2 TiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 34 memory 0xe7800000-e7ffffff

eno6: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether 80:30:e0:31:f5:28 txqueuelen 1000 (Ethernet)
RX packets 9790728 bytes 820859371 (782.8 MiB)
RX errors 0 dropped 6556849 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 125 memory 0xe8800000-e8ffffff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 823770 bytes 136031216 (129.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 823770 bytes 136031216 (129.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

I checked the kernel config file and all the config parameter enabled in the config file.

CONFIG_NET_VENDOR_BROADCOM=y
CONFIG_HAS_DMA=y
#CONFIG_BCM63XX is not set
CONFIG_SSB_POSSIBLE=y
CONFIG_HAS_IOMEM=y
CONFIG_PCI=y

5d:00.0 Ethernet controller: Broadcom Limited NetXtreme II BCM57810 10 Gigabit Ethernet (rev 10)
Subsystem: Hewlett-Packard Company FlexFabric 10Gb 2-port 534FLR-SFP+ Adapter
Kernel driver in use: bnx2x
Kernel modules: bnx2x
5d:00.1 Ethernet controller: Broadcom Limited NetXtreme II BCM57810 10 Gigabit Ethernet (rev 10)
Subsystem: Hewlett-Packard Company FlexFabric 10Gb 2-port 534FLR-SFP+ Adapter
Kernel driver in use: bnx2x
Kernel modules: bnx2x

Please help us to sole the issue on kernel-5.4

Thanks in advance,

Sandeep Kumar Kar

Hi! what are you trying to do is fully custom.

AFAIK RHEL 7 isn't based on any 4.x or 5.x Linux Kernel, instead is based on the 3.10.0 Kernel branch (with backported fixes from upstream Vanilla) and RHEL 8 is based on 4.18.0 Kernel branch (with backported fixes from upstream Vanilla).

See here and here

Given that, the HP FlexFabric 10Gb 2-port 534FLR-SFP+ Adapter uses official and supported (thus certified) device drivers for RHEL 7 and 8, respectively called, HPE QLogic NX2 10/20 GbE Multifunction Drivers for Red Hat Enterprise Linux 7 x86_64 and HPE QLogic NX2 10/20 GbE Multifunction Drivers for Red Hat Enterprise Linux 8 being supported, respectively, by RHEL 7.6 and 7.7 for the RHEL 7.x device driver, and RHEL 8.0 for the RHEL 8.x device driver. Plus you have to consider the relationship between Adapter, Firmware, Device Driver and OS. Actually the HPE QLogic NX2 Online Firmware Upgrade Utility for Linux x86_64 firmware - compatible (also) with the HP FlexFabric 10Gb 2-port 534FLR-SFP+ Adapter - supports RHEL OS up to 7.7 (7 Update 7) and 8.0 (8 Update 0)...so, to stay on the safe side, you should use RHEL 7 (up to 7.7) or RHEL 8 (up to 8.0) with official HPE supported device driver and Firmware.

I think you missed the point that there is no warranty that the same device driver (or another Vanialla one) would work on newer Kernels (or Vanilla ones): it could work or not...and diagnosing why it doesn't work as expected (or why it is not stable)...could be a little bit out-of-scope here...because you're essentially running a Custom Kernel (well two custom kernels 4.4 and 5.4) on a commercial Linux distribution like RHEL 7 - so created an unsupported configuration/scenario - and thus results can be really unpredictables.

So, in other terms, your Server is OK, the Network Adapter is OK, supported Device Drivers are OK, Network Adapter Firmware and OS are OK...BUT the scenario you built is not entirely OK because you're not playing with propers software components arranged in the way they were/are supported.

Thank you for your reply.

Can anyone tell me if the version is compatible in rhel-7.5

Working version on rhel-7.5 with kernel-4.4

#modinfo bnx2x

filename:       /lib/modules/4.4.194-1.el7.x86_64/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko

firmware:       bnx2x/bnx2x-e2-7.12.30.0.fw

firmware:       bnx2x/bnx2x-e1h-7.12.30.0.fw

firmware:       bnx2x/bnx2x-e1-7.12.30.0.fw

version:        1.712.30-0

license:        GPL

description:    QLogic BCM57710/57711/57711E/57712/57712_MF/57800/57800_MF/57810/57810_MF/57840/57840_MF Driver

author:         Eliezer Tamir

srcversion:     57E2F5329D2A3E5469B1075

alias:          pci:v000014E4d0000163Fsv*sd*bc*sc*i*

alias:          pci:v000014E4d0000163Esv*sd*bc*sc*i*

alias:          pci:v000014E4d0000163Dsv*sd*bc*sc*i*

alias:          pci:v00001077d000016ADsv*sd*bc*sc*i*

alias:          pci:v000014E4d000016ADsv*sd*bc*sc*i*

alias:          pci:v00001077d000016A4sv*sd*bc*sc*i*

alias:          pci:v000014E4d000016A4sv*sd*bc*sc*i*

alias:          pci:v000014E4d000016ABsv*sd*bc*sc*i*

alias:          pci:v000014E4d000016AFsv*sd*bc*sc*i*

alias:          pci:v000014E4d000016A2sv*sd*bc*sc*i*

alias:          pci:v00001077d000016A1sv*sd*bc*sc*i*

alias:          pci:v000014E4d000016A1sv*sd*bc*sc*i*

alias:          pci:v000014E4d0000168Dsv*sd*bc*sc*i*

alias:          pci:v000014E4d000016AEsv*sd*bc*sc*i*

alias:          pci:v000014E4d0000168Esv*sd*bc*sc*i*

alias:          pci:v000014E4d000016A9sv*sd*bc*sc*i*

alias:          pci:v000014E4d000016A5sv*sd*bc*sc*i*

alias:          pci:v000014E4d0000168Asv*sd*bc*sc*i*

alias:          pci:v000014E4d0000166Fsv*sd*bc*sc*i*

alias:          pci:v000014E4d00001663sv*sd*bc*sc*i*

alias:          pci:v000014E4d00001662sv*sd*bc*sc*i*

alias:          pci:v000014E4d00001650sv*sd*bc*sc*i*

alias:          pci:v000014E4d0000164Fsv*sd*bc*sc*i*

alias:          pci:v000014E4d0000164Esv*sd*bc*sc*i*

depends:        mdio,libcrc32c,ptp

retpoline:      Y

intree:         Y

vermagic:       4.4.194-1.el7.x86_64 SMP mod_unload modversions

parm:           num_queues: Set number of queues (default is as a number of CPUs) (int)

parm:           disable_tpa: Disable the TPA (LRO) feature (int)

parm:           int_mode: Force interrupt mode other than MSI-X (1 INT#x; 2 MSI) (int)

parm:           dropless_fc: Pause on exhausted host ring (int)

parm:           mrrs: Force Max Read Req Size (0..3) (for debug) (int)

parm:           debug: Default debug msglevel (int)

#ll /lib/firmware/bnx2x/
total 8756
-rw-r--r--. 1 root root 151568 Feb 20 2018 bnx2x-e1-6.0.34.0.fw
-rw-r--r--. 1 root root 151680 Feb 20 2018 bnx2x-e1-6.2.5.0.fw
-rw-r--r--. 1 root root 151688 Feb 20 2018 bnx2x-e1-6.2.9.0.fw
-rw-r--r--. 1 root root 161144 Feb 20 2018 bnx2x-e1-7.0.20.0.fw
-rw-r--r--. 1 root root 161248 Feb 20 2018 bnx2x-e1-7.0.23.0.fw
-rw-r--r--. 1 root root 161368 Feb 20 2018 bnx2x-e1-7.0.29.0.fw
-rw-r--r--. 1 root root 164392 Feb 20 2018 bnx2x-e1-7.10.51.0.fw
-rw-r--r--. 1 root root 170192 Feb 20 2018 bnx2x-e1-7.12.30.0.fw
-rw-r--r--. 1 root root 170096 Feb 20 2018 bnx2x-e1-7.13.1.0.fw
-rw-r--r--. 1 root root 162128 Feb 20 2018 bnx2x-e1-7.2.16.0.fw
-rw-r--r--. 1 root root 162152 Feb 20 2018 bnx2x-e1-7.2.51.0.fw
-rw-r--r--. 1 root root 163576 Feb 20 2018 bnx2x-e1-7.8.17.0.fw
-rw-r--r--. 1 root root 163592 Feb 20 2018 bnx2x-e1-7.8.19.0.fw
-rw-r--r--. 1 root root 163352 Feb 20 2018 bnx2x-e1-7.8.2.0.fw
-rw-r--r--. 1 root root 210776 Feb 20 2018 bnx2x-e1h-6.0.34.0.fw
-rw-r--r--. 1 root root 210824 Feb 20 2018 bnx2x-e1h-6.2.5.0.fw
-rw-r--r--. 1 root root 211000 Feb 20 2018 bnx2x-e1h-6.2.9.0.fw
-rw-r--r--. 1 root root 168552 Feb 20 2018 bnx2x-e1h-7.0.20.0.fw
-rw-r--r--. 1 root root 168544 Feb 20 2018 bnx2x-e1h-7.0.23.0.fw
-rw-r--r--. 1 root root 168680 Feb 20 2018 bnx2x-e1h-7.0.29.0.fw
-rw-r--r--. 1 root root 173016 Feb 20 2018 bnx2x-e1h-7.10.51.0.fw
-rw-r--r--. 1 root root 178984 Feb 20 2018 bnx2x-e1h-7.12.30.0.fw
-rw-r--r--. 1 root root 178992 Feb 20 2018 bnx2x-e1h-7.13.1.0.fw
-rw-r--r--. 1 root root 169896 Feb 20 2018 bnx2x-e1h-7.2.16.0.fw
-rw-r--r--. 1 root root 169960 Feb 20 2018 bnx2x-e1h-7.2.51.0.fw
-rw-r--r--. 1 root root 171920 Feb 20 2018 bnx2x-e1h-7.8.17.0.fw
-rw-r--r--. 1 root root 171920 Feb 20 2018 bnx2x-e1h-7.8.19.0.fw
-rw-r--r--. 1 root root 171368 Feb 20 2018 bnx2x-e1h-7.8.2.0.fw
-rw-r--r--. 1 root root 247008 Feb 20 2018 bnx2x-e2-6.0.34.0.fw
-rw-r--r--. 1 root root 247232 Feb 20 2018 bnx2x-e2-6.2.5.0.fw
-rw-r--r--. 1 root root 247504 Feb 20 2018 bnx2x-e2-6.2.9.0.fw
-rw-r--r--. 1 root root 290952 Feb 20 2018 bnx2x-e2-7.0.20.0.fw
-rw-r--r--. 1 root root 289448 Feb 20 2018 bnx2x-e2-7.0.23.0.fw
-rw-r--r--. 1 root root 289848 Feb 20 2018 bnx2x-e2-7.0.29.0.fw
-rw-r--r--. 1 root root 321456 Feb 20 2018 bnx2x-e2-7.10.51.0.fw
-rw-r--r--. 1 root root 321320 Feb 20 2018 bnx2x-e2-7.12.30.0.fw
-rw-r--r--. 1 root root 320936 Feb 20 2018 bnx2x-e2-7.13.1.0.fw
-rw-r--r--. 1 root root 295072 Feb 20 2018 bnx2x-e2-7.2.16.0.fw
-rw-r--r--. 1 root root 295264 Feb 20 2018 bnx2x-e2-7.2.51.0.fw
-rw-r--r--. 1 root root 310032 Feb 20 2018 bnx2x-e2-7.8.17.0.fw
-rw-r--r--. 1 root root 310440 Feb 20 2018 bnx2x-e2-7.8.19.0.fw
-rw-r--r--. 1 root root 304664 Feb 20 2018 bnx2x-e2-7.8.2.0.fw

NonWorking bnx2x

version:        1.712.30-0

The solution to the issue is

Before upgrading the kernel 5.4 in the server  

Upgrade the linux-firmware in the OS

#yum update linux-firmware*

after that upgrade the kernel-5.4

and reboot the server 

It will fix the issue