Wired Intelligent Edge

ProVision to Comware trunk - port unselected

Hello

I have a trunk with 2 ports between a Aruba 5412Rzl2 and a HPE FLEXFABRIC 5710-48XGT.

The trunk on the 5710 / Comware have been created exactly like advised here: https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c03503743

On the 5412 / ProVision the trunk are also created.

The problem are that one of the ports are always down / unselected in the trunk and if I remove the cable in the active port the connection fails. All hardware and cables are ok - tried change tranceivers, cables, modules and even the 5710 switch.

I've spend days on this problem and have no idea how to solve it.

Below a piece of the configuration from both switches and some info:

******************************************************************
***** Comware configuration and some informations *****
******************************************************************

#
stp global enable
#
interface Bridge-Aggregation103
port link-type trunk
port trunk permit vlan all
link-aggregation mode dynamic
#
interface NULL0
#
interface Vlan-interface1
ip address 10.1.1.12 255.255.255.0
#
interface FortyGigE1/0/49
port link-mode bridge
port link-type trunk
port trunk permit vlan all
port link-aggregation group 103
#
interface FortyGigE1/0/50
port link-mode bridge
port link-type trunk
port trunk permit vlan all
port link-aggregation group 103
#
#

<dk-switch11>disp link-aggregation verbose
Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing
Port Status: S -- Selected, U -- Unselected, I -- Individual
Port: A -- Auto port, M -- Management port, R -- Reference port
Flags:  A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,
        D -- Synchronization, E -- Collecting, F -- Distributing,
        G -- Defaulted, H -- Expired

Aggregate Interface: Bridge-Aggregation103
Aggregation Mode: Dynamic
Loadsharing Type: Shar
Management VLANs: None
System ID: 0x8000, 4cae-a3ac-44d7
Local:
  Port                Status   Priority Index    Oper-Key               Flag
  FGE1/0/49(R)        S        32768    1        1                      {ACDEF}
  FGE1/0/50           U        32768    2        1                      {ACG}
Remote:
  Actor               Priority Index    Oper-Key SystemID               Flag
  FGE1/0/49           0        257      1062     0xae00, 0897-34d7-ae00 {ACDEF}
  FGE1/0/50           32768    0        0        0x8000, 0000-0000-0000 {DEF}

<dk-switch11>display interface brief | include UP
InLoop0              UP   UP(s)    --
NULL0                UP   UP(s)    --
REG0                 UP   --       --
Vlan1                UP   UP       10.1.1.12
BAGG103              UP   40G(a)  F(a)   T    1
FGE1/0/49            UP   40G(a)  F(a)   T    1
FGE1/0/50            UP   40G(a)  F(a)   T    1

*******************************************************************
***** ProVision configuration and some informations *****
*******************************************************************

trunk I2,J2 trk101 lacp
#
spanning-tree Trk101 priority 4
#
vlan 50
name "CLIENT"
tagged Trk101
#
#

DK1-SWITCH01# sh int br | i Trk101
  I1-Trk101    QSFP+SR4   | No        Yes     Up     40GigFD    NA   off  0
  J1-Trk101               | No        Yes     Down   .               off  0

DK1-SWITCH01# sh lacp peer | i Trk101
  I1     Trk101 4caea3-ac44d7     1     32768     1       Active   Slow
  J1     Trk101 000000-000000     2     32768     1       Active   Slow

Hello @lasse-k are you able to show us sanitized running configuration of the Aruba 5412R zl2 (sanitized = obfuscated MAC addresses, Serial Numbers and other sensitive details of your devices/networks)?

To complete you would provide the outputs of show modules, show tech transceviers, show interface ethernet x (with x=I1, J1, I2 and J2) and show vlan port trk101 detail (and trk102, eventually) too, it would help a lot to exactly understand what is going on.

Also, there is something strange on DK1-SWITCH01: you reported that port I2 and port J2 are members of Trk101 (the command reported is indeed trunk I2,J2 trk101 lacp) but, on the next show lacp command's output, the Trk101 logical interface looks like was made of port I1 and port J1 instead of I2 and J2. Isn't it?

Side note: is the Aruba 5412R zl2 running an updated version of its ArubaOS-Switch?

Would be nice to also have a clear picture of what are materials (SKUs) actually involved...are you using QSFP+ Transceivers or DACs? Are you able to report us back the SKUs involved on both sides (HPE 5710 and Aruba 5412R zl2)?

Aruba 5412 zl2
SN info erased

QSFP+ tranceivers for Aruba
JH231A

HPE 5710

QSFP+ tranceivers for HPE
JG325B

Hello @parnassus 

Having some problems adding the info to this thread - working on it.

Trk101 are of course port J1,I1

BTW when rebooting the HPE 5710 switch the interface thats up changed to port I1 !?

Hello @parnassus 

Sorry but all my posts gets rejected or marked as spam

Hello @parnassus 

Below the show modules

DK1-SWITCH01# sh modules

 Status and Counters - Module Information

  Chassis: 5412Rzl2 J9851A         Serial Number:   ---

  Allow V2 Modules:   Yes

                                                                       Core  Mod
  Slot  Module Description                     Serial Number  Status   Dump  Ver
  ----- -------------------------------------- -------------- -------- ----- ---
  MM1   HP J9827A Management Module 5400Rzl2   ---                        Active   YES   1
  MM2   HP J9827A Management Module 5400Rzl2   -                       Standby  YES   1


  A     HP J9986A 24p 10/100/1000BASE-T PoE... -Up       YES   3
  B     HP J9986A 24p 10/100/1000BASE-T PoE... -Up       YES   3
  C     HP J9986A 24p 10/100/1000BASE-T PoE... -Up       YES   3
  D     HP J9986A 24p 10/100/1000BASE-T PoE... -Up       YES   3
  E     HP J9986A 24p 10/100/1000BASE-T PoE... -Up       YES   3
  F     HP J9986A 24p 10/100/1000BASE-T PoE... -Up       YES   3
  G     HP J9986A 24p 10/100/1000BASE-T PoE... -Up       YES   3
  H     HP J9986A 24p 10/100/1000BASE-T PoE... -Up       YES   3
  I     HP J9996A 2p 40GbE QSFP+ v3 zl2 Mod    -Up       YES   3
  J     HP J9996A 2p 40GbE QSFP+ v3 zl2 Mod    -Up       YES   3
  K     HP J9993A 8p 1G/10GbE SFP+ v3 zl2 Mod  -Up       YES   3
  L     HP J9993A 8p 1G/10GbE SFP+ v3 zl2 Mod  -Up       YES   3

Hello @parnassus 

Here are the show tech transceviers

DK1-SWITCH01# show tech trans
Cmd Info : show tech transceivers

transceivers

Transceiver Technical Information:
 Port # | Type      | Prod #     | Serial #         | Part #
 -------+-----------+------------+------------------+----------
 I1     |    ??     |       ?? B | unsupported      | 1990-4557
 J1     |    ??     |       ?? B | unsupported      | 1990-4557
 K1     | SFP+SR    |     J9150D | -| 1990-4391
 K2     | SFP+SR    |     J9150D | -| 1990-4391
 K8     | SFP+SR    |     J9150D | -| 1990-4391
 L1     | SFP+SR    |     J9150D | -| 1990-4391
 L7     | SFP+SR    |     J9150D | -       | 1990-4391
 L8     | SFP+SR    |     J9150D | -| 1990-4391


show time
Wed Jun  3 16:14:45 2020

Hello @parnassus 

Now the running conf from the Aruba - sorry for all the posts but they get marked as spam if to much tech info inside

; J9851A Configuration Editor; Created on release #KB.16.05.0007
; Ver #12:08.1d.fb.7f.bf.bb.ff.7c.59.fc.7b.ff.ff.fc.ff.ff.3f.ef:f6
hostname "DK1-SWITCH01"
module A type j9986a
module B type j9986a
module C type j9986a
module D type j9986a
module E type j9986a
module F type j9986a
module G type j9986a
module H type j9986a
module I type j9996a
module J type j9996a
module K type j9993a
module L type j9993a
console idle-timeout 600
console idle-timeout serial-usb 600
trunk K1,L8 trk1 lacp
trunk K2,L7 trk2 lacp
trunk I1,J1 trk101 lacp
trunk B21-B22,F21-F22 trk130 lacp
power-over-ethernet pre-std-detect ports A1-A24,B1-B24,C1-C24,D1-D24,E1-E24,F1-F24,G1-G24,H1-H24
timesync sntp
sntp unicast
sntp 360
sntp server priority 1 185.23.153.237
sntp server priority 2 157.7.154.29
sntp server priority 3 157.7.208.12
no telnet-server
time daylight-time-rule western-europe
time timezone 60
ip access-list extended "xxxxxx"
     10 deny ip 0.0.0.0 255.255.255.255 10.1.67.2 0.0.0.0
     20 permit ip 0.0.0.0 255.255.255.255 10.1.67.0 0.0.0.255
     30 permit tcp 0.0.0.0 255.255.255.255 10.1.11.177 0.0.0.0 eq 8080
     40 permit tcp 0.0.0.0 255.255.255.255 10.1.11.177 0.0.0.0 eq 8443
     50 permit tcp 0.0.0.0 255.255.255.255 10.1.11.177 0.0.0.0 eq 8880
     60 permit tcp 0.0.0.0 255.255.255.255 10.1.11.177 0.0.0.0 eq 8843
     65 permit tcp 0.0.0.0 255.255.255.255 10.1.11.177 0.0.0.0 eq 22
     70 deny ip 0.0.0.0 255.255.255.255 10.0.0.0 0.255.255.255
     80 permit ip 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255
   exit
ip directed-broadcast
ip route 0.0.0.0 0.0.0.0 10.1.10.32
ip route 10.9.0.0 255.255.0.0 10.1.254.1
ip route 10.10.0.0 255.255.0.0 10.1.254.1
ip route 10.18.0.0 255.255.0.0 10.1.254.1
ip route 10.19.0.0 255.255.0.0 10.1.254.1
ip route 10.20.0.0 255.255.0.0 10.1.254.1
ip route 10.40.0.0 255.255.0.0 10.1.254.1
ip routing
ip udp-bcast-forward
ip source-interface sntp vlan 20
ipv6 nd suppress-ra
snmp-server community "public" unrestricted
snmp-server contact "xxxxxx@xxxxxx" location "xxxxxx"
oobm
   ip address dhcp-bootp
   exit
vlan 1
   name "DEFAULT_VLAN"
   no untagged A1,A3-A12,A17-A24,B1-B10,B18-B20,B23-B24,C1-C9,E1,E3-E12,E17-E24,F1-F10,F18-F20,F23-F24,G1-G24,H1-H24
   untagged A2,A13-A16,B11-B17,C10-C24,D1-D24,E2,E13-E16,F11-F17,I2,J2,K3-K8,L1-L6,Trk1-Trk2,Trk101,Trk130
   ip address 10.1.1.2 255.255.255.0
   exit
vlan 2
   name "DMZ"
   tagged B18,F18,Trk130
   no ip address
   exit
vlan 3
   name "WAN_LINK"
   untagged B23-B24,F23-F24
   tagged Trk1-Trk2
   no ip address
   exit
vlan 10
   name "NETWORK"
   untagged B18,F18
   tagged Trk1
   ip address 10.1.10.13 255.255.255.0
   exit
vlan 11
   name "SECURE"
   untagged A3-A6,B19-B20,E3-E5,F19-F20
   tagged Trk1,Trk130
   ip address 10.1.11.2 255.255.255.0
   ip helper-address 10.1.20.13
   ip helper-address 10.1.20.14
   exit
vlan 14
   name "ADMIN_WIFI"
   untagged C5-C6
   tagged Trk1-Trk2
   ip address 10.1.14.2 255.255.255.0
   ip helper-address 10.1.20.13
   ip helper-address 10.1.20.14
   exit
vlan 20
   name "HW_SERVER"
   untagged A17-A24,B1-B10,E17-E24,F1-F10
   tagged A16,B12-B13,E16,F12-F13,Trk1-Trk2,Trk130
   ip address 10.1.20.2 255.255.255.0
   ip helper-address 10.1.20.13
   ip helper-address 10.1.20.14
   ip forward-protocol udp 10.1.23.255 2712
   ip forward-protocol udp 10.1.50.255 9
   exit
vlan 23
   name "VIRT_SERVER"
   tagged A13-A16,B11-B14,E13-E16,F11-F14,Trk1,Trk130
   ip address 10.1.23.2 255.255.255.0
   ip forward-protocol udp 10.1.20.255 2712
   ip forward-protocol udp 10.1.20.255 6801
   ip forward-protocol udp 10.1.20.255 4530
   ip forward-protocol udp 10.1.50.255 9
   exit
vlan 24
   name "NLS"
   tagged Trk130
   ip address 10.1.24.2 255.255.255.0
   exit
vlan 25
   name "AX_CLUSTER"
   tagged Trk130
   ip address 10.1.25.2 255.255.255.0
   ip helper-address 10.1.20.13
   ip helper-address 10.1.20.14
   exit
vlan 26
   name "DK_VIRT_CLUST_02"
   untagged A7-A9
   no ip address
   exit
vlan 27
   name "DK_VIRT_CLUST_03"
   untagged A10-A12
   no ip address
   exit
vlan 28
   name "DK_VIRT_CLUST_01"
   untagged C7-C9
   no ip address
   exit
vlan 30
   name "PRINTER"
   untagged G1-G24,H11-H12,H14-H24
   tagged Trk1-Trk2
   ip address 10.1.30.2 255.255.255.0
   ip helper-address 10.1.20.13
   ip helper-address 10.1.20.14
   exit
vlan 40
   name "IPT"
   untagged C1-C4
   tagged Trk1-Trk2
   ip address 10.1.40.2 255.255.255.0
   exit
vlan 50
   name "CLIENT"
   untagged E6-E12,H1-H10,H13
   tagged B14,F14,Trk1-Trk2,Trk101,Trk130
   ip address 10.1.50.2 255.255.255.0
   ip helper-address 10.1.20.14
   ip helper-address 10.1.20.13
   ip helper-address 10.1.23.230
   ip forward-protocol udp 10.1.20.255 2712
   ip forward-protocol udp 10.1.20.255 1604
   ip forward-protocol udp 10.1.23.255 2712
   exit
vlan 55
   name "CLIENT_WLAN"
   tagged C5-C6,Trk1-Trk2
   ip address 10.1.55.2 255.255.255.0
   ip helper-address 10.1.20.13
   ip helper-address 10.1.20.14
   ip forward-protocol udp 10.1.20.255 2712
   ip forward-protocol udp 10.1.20.255 1604
   exit
vlan 67
   name "MOBILE_WIFI"
   tagged C5-C6,Trk1-Trk2
   ip access-group "xxxxxx" vlan-in
   ip address 10.1.67.2 255.255.255.0
   ip helper-address 10.1.20.13
   ip helper-address 10.1.20.14
   exit
vlan 254
   name "MPLS"
   untagged A1,E1
   ip address 10.1.254.2 255.255.255.0
   exit
spanning-tree Trk1 priority 4
spanning-tree Trk2 priority 4
spanning-tree Trk101 priority 4
spanning-tree Trk130 priority 4
no tftp server
no autorun
no dhcp config-file-update
no dhcp image-file-update
password manager
password operator

Hello @parnassus 

Not sure about the SKU - the full model or the serial numbers?

Hello @parnassus 

Full model name for HPE switch:

HPE 5710 48XGT 6QS+/2QS28

Hello @parnassus 

Now since yesterday I was not able to get the connection up. Now today I removed all tranceivers in both switches and rebooted the HPE 5710. When the switch was up running again I attached all tranceivers / cables. Now 1 of the two connections are up - and still the other port are unselected. See below output from the 2 switches. Also keeps getting these info on the HPE 5710 console - see below.

Aruba 5412

DK1-SWITCH01# sh tech tran
Cmd Info : show tech transceivers

transceivers

Transceiver Technical Information:
 Port # | Type      | Prod #     | Serial #         | Part #
 -------+-----------+------------+------------------+----------
 I1     |    ??     |       ?? B | unsupported      | 1990-4557
 J1     | QSFP+SR4  |     JH231A | PH90H7G075       | 1990-4557
 K1     | SFP+SR    |     J9150D | CN88KBV1NC       | 1990-4391
 K2     | SFP+SR    |     J9150D | CN88KBV56P       | 1990-4391
 K8     | SFP+SR    |     J9150D | CN88KBV1XN       | 1990-4391
 L1     | SFP+SR    |     J9150D | CN88KBV5BD       | 1990-4391
 L7     | SFP+SR    |     J9150D | CN88KBV3RR       | 1990-4391
 L8     | SFP+SR    |     J9150D | CN88KBV3RZ       | 1990-4391


show time
Thu Jun  4 09:41:45 2020


=== The command has completed successfully. ===

HPE 5710
<dk-switch11>disp link-aggregation verbose
Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing
Port Status: S -- Selected, U -- Unselected, I -- Individual
Port: A -- Auto port, M -- Management port, R -- Reference port
Flags:  A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,
        D -- Synchronization, E -- Collecting, F -- Distributing,
        G -- Defaulted, H -- Expired

Aggregate Interface: Bridge-Aggregation103
Aggregation Mode: Dynamic
Loadsharing Type: Shar
Management VLANs: None
System ID: 0x8000, 4cae-a3ac-44d7
Local:
  Port                Status   Priority Index    Oper-Key               Flag
  FGE1/0/49           U        32768    1        1                      {ACG}
  FGE1/0/50(R)        S        32768    2        1                      {ACDEF}
Remote:
  Actor               Priority Index    Oper-Key SystemID               Flag
  FGE1/0/49           32768    0        0        0x8000, 0000-0000-0000 {DEF}
  FGE1/0/50           0        289      1062     0xae00, 0897-34d7-ae00 {ACDEF}

HPE 5710

<dk-switch11>%Jan  1 00:14:06:967 2011 dk-switch11 STP/6/STP_NOTIFIED_TC: Instance 0's port Bridge-Aggregation103 was notified a topology change.
%Jan  1 00:14:09:945 2011 dk-switch11 STP/6/STP_NOTIFIED_TC: Instance 0's port Bridge-Aggregation103 was notified a topology change.
%Jan  1 00:15:52:946 2011 dk-switch11 STP/6/STP_NOTIFIED_TC: Instance 0's port Bridge-Aggregation103 was notified a topology change.
%Jan  1 00:15:55:942 2011 dk-switch11 STP/6/STP_NOTIFIED_TC: Instance 0's port Bridge-Aggregation103 was notified a topology change.
%Jan  1 00:15:56:947 2011 dk-switch11 STP/6/STP_NOTIFIED_TC: Instance 0's port Bridge-Aggregation103 was notified a topology change.
%Jan  1 00:15:59:942 2011 dk-switch11 STP/6/STP_NOTIFIED_TC: Instance 0's port Bridge-Aggregation103 was notified a topology change.

The fact that QSFP+ MPO SR Transceiver is not recognized on Port I1 (Aruba 5412R zl2: Module I Port 1) is not a good sign.

Your Aruba 5400R zl2 Switch should be running using the ArubaOS-Switch KB.16.05.0007 (Mid 2018)...not new but also not too old. Definitely not ancient. Using a newer version would probably be better but, technically speaking, used modules and QSFP+ MPO Transceviers are supported.

Is your Aruba 5400R zl2 operating in v3 only mode or is operating in Compatibility mode?

You have to know the Aruba 5400R zl2 is able to operate in two modes:

1. Compatibility mode: This is the default mode and it allows the Aruba 5400R zl2 chassis to run both v2 zl and v3 zl2 modules concurrently, but the software feature set will be equivalent to v2 zl (as example it cannot enable any v3 zl2 specific software feature).
2. V3 only mode: Some features are supported in v3 zl2 only mode so you need to “turn off” the v2 zl modules. When moving from the default v2-compatible mode to v3-only mode, the switch reboot is required.

Check this. Not sure about the impact of runnning in Compatibility mode (v2 zl/v3 zl2) since the J9996A is for sure v3 zl2 but, AFAIK, it's recognized on your chassis so you should be OK. Nevermind, do a check.

The only QSFP+ Transceiver I see recognized by your Aruba 5400R zl2 is the one inserted into Module J Port 1 and it is exactly a JH231A with 4x4 1999-4554 (That's the SKU).

As of now the Aruba 5400R zl2 supports those QSFP+/DAC once installed on QSFP+ ports of modules with SKU J9996A (it's your case) and J9992A (it's not your case):

1. HPE X142 40G QSFP+ MPO SR4 Transceiver (JH231A)
2. HPE X142 40G QSFP+ LC LR4 SM Transceiver (JH232A)
3. HPE X142 40G QSFP+ MPO eSR4 300M Transceiver (JH233A)
4. Aruba 40G QSFP+ LC Bidirectional 150m MMF 2-strand Transceiver (JL308A)
5. HPE X242 40G QSFP+ to QSFP+ 1m Direct Attach Copper Cable (JH234A)
6. HPE X242 40G QSFP+ to QSFP+ 3m Direct Attach Copper Cable (JH235A)
7. HPE X242 40G QSFP+ to QSFP+ 5m Direct Attach Copper Cable (JH236A)

So you're dealing with an MPO SR4 QSFP+ Transceiver on the Aruba side.

On the HPE FlexFabric 5710 you're correctly using the corresponding JG325B MPO SR4 Transceiver, which is supported on the HPE 5710:

1. HPE X140 40G QSFP+ MPO SR4 Transceiver (JG325B)
2. HPE X140 40G QSFP+ MPO MM 850nm CSR4 300m Transceiver (JG709A)
3. HPE X140 40G QSFP+ LC BiDi 100m MM Transceiver (JL251A)
4. HPE X140 40G QSFP+ LC LR4 SM 10km 1310nm Transceiver (JG661A)
5. HPE X140 40G QSFP+ LC LR4L 2km SM Transceiver (JL286A)
6. HPE QSFP/SFP+ Adapter Kit (655874-B21)
7. HPE FlexNetwork X240 40G QSFP+ QSFP+ 1m Direct Attach Copper Cable (JG326A)
8. HPE FlexNetwork X240 40G QSFP+ QSFP+ 3m Direct Attach Copper Cable (JG327A)
9. HPE FlexNetwork X240 40G QSFP+ QSFP+ 5m Direct Attach Copper Cable (JG328A)
10. HPE FlexNetwork X240 40G QSFP+ to 4x10G SFP+ 1m Direct Attach Copper Splitter Cable (JG329A)
11. HPE FlexNetwork X240 40G QSFP+ to 4x10G SFP+ 3m Direct Attach Copper Splitter Cable (JG330A)
12. HPE FlexNetwork X240 40G QSFP+ to 4x10G SFP+ 5m Direct Attach Copper Splitter Cable (JG331A)
13. HPE X2A0 40G QSFP+ to QSFP+ 7m Active Optical Cable (JL287A)
14. HPE X2A0 40G QSFP+ to QSFP+ 10m Active Optical Cable (JL288A)
15. HPE X2A0 40G QSFP+ to QSFP+ 20m Active Optical Cable (JL289A)

Couldn't be an issue related to the MPO cabling used? are you going directly (FO patch) between two Switches?

If you’re going QSFP+ MPO to QSFP+ MPO directly, are you using an MPO Female to MPO Female crossover fiber optic patch cable or what?

Is it a Key-Up to Key-Up crossover MPO 12-strand cable?

Other question: could you provide (R|M)STP - Spanning Tree - status on both ends (Aruba/HPE)?

Also, can you post details about Comware V7 version currently running on your HPE FlexNetwork 5710 (SKU: JL586A) Switch? <- use the display version CLI command.

Are you sure that, HPE 5710 side, the six QSFP+ ports are all in their default state (not splitted 40Gbps -> 4x10Gbps)?

I noticed there is this particular note about JG325B but it looks not too relevant.

Edit: ironically the latest (6th May 2020) HPE Comware 40G Transceriver doesn't list the HPE FlexNetwork 5710 (JL586A) Switch among those supporting the QSFP+ Transceiver JG325B...the (4th May 2020) HPE FlexNewtork 5710 QuickSpecs instead correctly reports the JG325B as normally supported.

Hello

Sorry for the late reply - been out of office for a few days.

I will find answers for all questens right away.

Swapped all 4 tranceivers and cables and rebooted HPE switch. Afterwared attached cables / tranceivers at the excat same time in both switches and now connection seems fine !?!?

Aruba 5400 zl2 switch runs Compatibility mode - default not changed

Switches are connected directly using QSFP+ MPO to QSFP+ MPO with an MPO Female to MPO Female fiber optic cable QK729A Key-Up to Key-Up.

All six QSFP+ ports on HPE 5710 side are in default state.

HPE 5710

<dk-switch11>disp version
HPE Comware Software, Version 7.1.070, Release 2612P05
Copyright (c) 2010-2019 Hewlett Packard Enterprise Development LP
HPE 5710 48XGT 6QS+/2QS28 Switch uptime is 0 weeks, 0 days, 2 hours, 26 minutes
Last reboot reason : User reboot

Boot image: flash:/5710-cmw710-boot-r2612p05.bin
Boot image version: 7.1.070, Release 2612P05
  Compiled Feb 25 2019 11:00:00
System image: flash:/5710-cmw710-system-r2612p05.bin
System image version: 7.1.070, Release 2612P05
  Compiled Feb 25 2019 11:00:00


Slot 1:
Uptime is 0 weeks,0 days,2 hours,26 minutes
5710 48XGT 6QS+/2QS28 Switch with 2 Processor
BOARD TYPE:         5710 48XGT 6QS+/2QS28 Switch
DRAM:               4096M bytes
FLASH:              1024M bytes
PCB 1 Version:      VER.A
PCB 2 Version:      VER.A
FPGA Version:       NONE
Bootrom Version:    234
CPLD 1 Version:     002
CPLD 2 Version:     002
Release Version:    HPE 5710 48XGT 6QS+/2QS28 Switch-2612P05
Patch Version:      None
Reboot Cause:       UserReboot
[SubSlot 0] 48XGT+6QSFP Plus/2QSFP28


<dk-switch11>disp link-aggregation verbose
Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing
Port Status: S -- Selected, U -- Unselected, I -- Individual
Port: A -- Auto port, M -- Management port, R -- Reference port
Flags:  A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,
        D -- Synchronization, E -- Collecting, F -- Distributing,
        G -- Defaulted, H -- Expired

Aggregate Interface: Bridge-Aggregation103
Aggregation Mode: Dynamic
Loadsharing Type: Shar
Management VLANs: None
System ID: 0x8000, 4cae-a3ac-44d7
Local:
  Port                Status   Priority Index    Oper-Key               Flag
  FGE1/0/49(R)        S        32768    1        1                      {ACDEF}
  FGE1/0/50           S        32768    2        1                      {ACDEF}
Remote:
  Actor               Priority Index    Oper-Key SystemID               Flag
  FGE1/0/49           0        257      1062     0xae00, 0897-34d7-ae00 {ACDEF}
  FGE1/0/50           0        289      1062     0xae00, 0897-34d7-ae00 {ACDEF}

Aruba 5400 

DK1-SWITCH01# sh tech trans
Cmd Info : show tech transceivers

transceivers

Transceiver Technical Information:
 Port # | Type      | Prod #     | Serial #         | Part #
 -------+-----------+------------+------------------+----------
 I1     | QSFP+SR4  |     JH231A | PH90H7G075       | 1990-4557
 J1     | QSFP+SR4  |     JH231A | PH90H7G06P       | 1990-4557
 K1     | SFP+SR    |     J9150D | CN88KBV1NC       | 1990-4391
 K2     | SFP+SR    |     J9150D | CN88KBV56P       | 1990-4391
 K8     | SFP+SR    |     J9150D | CN88KBV1XN       | 1990-4391
 L1     | SFP+SR    |     J9150D | CN88KBV5BD       | 1990-4391
 L7     | SFP+SR    |     J9150D | CN88KBV3RR       | 1990-4391
 L8     | SFP+SR    |     J9150D | CN88KBV3RZ       | 1990-4391


show time
Mon Jun  8 13:01:34 2020

(R|M)STP - Spanning Tree - status on both ends (Aruba/HPE) below. Not sure thats its the right info you need? Else let me know what you need.

HPE 5710

<dk-switch11>disp stp interface Bridge-Aggregation 103
 ----[CIST][Port1615(Bridge-Aggregation103)][FORWARDING]----
 Port protocol       : Enabled
 Port role           : Root Port (Boundary)
 Port ID             : 128.1615
 Port cost(Legacy)   : Config=auto, Active=1
 Desg.bridge/port    : 32768.2c23-3abd-3f06, 128.683
 Port edged          : Config=disabled, Active=disabled
 Point-to-Point      : Config=auto, Active=true
 Transmit limit      : 10 packets/hello-time
 TC-Restriction      : Disabled
 Role-Restriction    : Disabled
 Protection type     : Config=none, Active=none
 MST BPDU format     : Config=auto, Active=802.1s
 Port Config-
 Digest-Snooping     : Disabled
 Rapid transition    : True
 Num of VLANs mapped : 2
 Port times          : Hello 2s MaxAge 20s FwdDelay 15s MsgAge 0s RemHops 20
 BPDU sent           : 2
          TCN: 0, Config: 0, RST: 0, MST: 2
 BPDU received       : 2634
          TCN: 0, Config: 0, RST: 0, MST: 2634

Aruba 5400

DK1-SWITCH01(config)# show spanning-tree detail

 Multiple Spanning Tree (MST) Information

  STP Enabled   : No

What I'am trying to set are two rack switches for servers using 2 x HPE 5710 switches. The switches will both connect to the main switch Aruba 5400. All server NIC's will have connections to both HPE 5710 switches.

Both HPE 5710 switches have 2 cables connected to the Aruba 5400 switch in a trunk.

Somehow I got the trunk up running no on the first switch and will now try configure the other one.

I'am a bit in doubt about the spanning-tree setup on these switches so any suggestion will be appreciated a lot. Might be the whole issue / problem that the spanning-tree setup are done wrong?

Hi! Glad you fixed (just reseating QSFP+ Transcerivers on the Aruba 5400R zl2?) the LAGs that are now working on both peers (at least from your Aruba 5400R zl2 to one of your HPE FlexNetwork 5710).

Just a question: why not deploying IRF with your two HPE FlexNetwork 5710? that way you would use Port Trunking (LACP) on Aruba 5400R zl2 with member ports going from the Aruba chassis to both 5710 chassis (so you will "distribute" the physical links to both your 5710 concurrently)...one single Trk<x> on the Aruba side and one single Bridge Aggregation interface (LAG) on the HPE 5710 IRF side.

The same could be done (should be done, at that point) between the Server and the HPE 5710 IRF pair.

Spanning Tree should be configured accordingly (in any case)...so you should decide what Spanning Tree protocol you want (RSTP/MSTP) and then set both peers (here I'm supposing you would go with the IRF approach...so the IRF become a single logical peer for the Aruba 5400R zl2 despite is made of two physical switches).

Thanks for your help.

The IRF setup will be a task for me later as I need to read more about this too ...

Lets consider I will do a "normal" trunk setup with the 2 switches as shown in the small diagram. Can I them just like the IRF setup make 1 trunk on the Aruba switch for all connections from both switches? 

And I assume that spanning tree need to be enabled on all 3 switches?

Sorry for all the questions but this is my first redundant switch setup.

---

@lasse-k wrote: Lets consider I will do a "normal" trunk setup with the 2 switches as shown in the small diagram. Can I them just like the IRF setup make 1 trunk on the Aruba switch for all connections from both switches?

If your HPE 5710's pair will remain made of separated (not uplinked together) switches it will just a matter of setting up two separated trunks (as per your network diagram) with links egressing from the Aruba 5400R zl2 and ingressing, respectively, into each one HPE 5710.

Say:

Aruba 5400R zl2 trk<x> (2 links) <===> (2 links) HPE 5710 (East)

Aruba 5400R zl2 trk<y> (2 links) <===> (2 links) HPE 5710 (West)

Instead of (IRF case):

Aruba 5400R zl2 trk<z> (e.g. 4 links) <===> IRF (say 2 links on 5710 East and the others 2 links on the 5710 West)

With regards to Spanning Tree: yes, even if the network topology hasn't any loop (your one, as per diagram, hasn't <- provided that the Server is not going to bridge its ports), it's best practice to enable and configure Spanning Tree accordingly because STP kicks in to protect your network from loops.

Hello @parnassus 

I will definitely look into the spanning tree now and later on the IRF.

Thanks a lot for your help and also giving me some insights to some usefull network stuff.

---

@parnassus wrote:

---

With regards to Spanning Tree: yes, even if the network topology hasn't any loop (your one, as per diagram, hasn't <- provided that the Server is not going to bridge its ports), it's best practice to enable and configure Spanning Tree accordingly because STP kicks in to protect your network from loops.

---

Just to be sure. My server will indeed have 2 bridged ports and supposed to have 1 cable in each switch. If this setup have to run I will need the Spanning tree setup - right? Putting the Aruba 5400 as the root bridge and the others with lower priority. And will the Aruba be ok to "speak" with the HPE devices?

There shouldn't be issues between Aruba and HPE in terms of STP. Let me check if I'm able to find some documents about STP interoperability between ArubaOS-Switch (ProVision) based devices and Comware based devices.

With regards to the Server: I mean...if the two ports are set as switched on the very same VLAN the Server is going to form (and close) a loop on that VLAN so I expect STP to kick in and disable one offending link (between the Server and the IRF). Isn't it?

Yes the Windows server will have a NIC team af 2 physical ports on the same VLAN. I think (will investigate - not 100% sure) that Windows NIC teaming can handle spanning tree issues by itself. But as you say one of the links might will be closed by one of the switches / spanning tree on the switch.

If the NIC ports are "bonded" together (via "LACP" = Dynamic Trunking or via "Non Protocol" = Static Trunking) [*] then STP will see those ports as one entity, the logical interface represeting the Link Aggregation Group (LAG) so, correctly, it will not kick in since the Switch will see one aggregated port and not two physical interfaces (from the STP standpoint).

But you wrote that NIC ports will be "bridged" and I associate the bridging action with something which is not the aggregation we are here referring to so my note above about the possible loop.

[*] in your scenario, not having an IRF deployed, you can't "bond" NIC's ports together and terminate their links against different standalone 5710...so, at best, in this conditions, each NIC port can only be configured to be simply "standalone" (without any traffic bridging between the them).

Hello

Hmm - ok. Seems I need to go the IRF way then.

Tried to make a diagram below. When setting up a IRF I assume the one switch can break down and I will still be up running? Do one of the switches act as primary and if this breaks down the second switch will also be down?

I will then connect the switches like below - will this be possible? To ensure redundancy.

The issue about the NIC / LACP / static trunking will then be gone I assume as both switches will then act as one?

Is really easier if you have a look at the HPE FlexFabric 5710 Switch Series Virtual Technologies Configuration Guide (valid for R2702 and newer). It's dedicated to IRF, concepts, deployment and management of such a technology.

Basically the two (chained) IRF Members start to act as a single logical identity (and they're seen that way from peers), the IRF is going to have a Commander member and a Standby member (that's on the simple two members IRF case).

Peer devices (your Server and/or your other Aruba 5400 Switch) are going to be connected by using LAG LACP (port bonding/aggregation with LACP as the link aggregation control protocol). If the Commander became faulty the Standby member continues the operation. Clearly each LAG leg need to depart from the peer and terminate on a IRF Member...that's to ensure that a leg/interface/IRF member fault doesn't cause the entire LAG to go down.

Hello @parnassus 

Just looked at the documentation and I will set up IRF using it. Seems quite understandably.

Thanks a lot for your help and great support. I really appreciate all your help

Hi @lasse-k, I'm glad to have helped!