Wired Intelligent Edge

Just noticed that default MTU value on all interfaces is set at 1500 (!).

What is the best practice with regard to MTU value to adopt on physical interfaces members specifically used into the VSX ISL LAG (considering that those ones are generally capable of 40 Gbps)?

Once the VSX ISL LAG (or, more broadly, on any type of LAG) is set:

1. Will it be possible to change MTU value? if so should this happen at LAG level or at each single member physical interface level [*]?
2. Will it cause ISL interruption? in other terms...will the VSX ISL be totally disrupted (LAG where member interfaces have concurrently differents MTS values for a limited time) causing a VSX Split Brain situation that will force the Secondary to shutdown its VSX LAGs?

[*] I'm referring to the issue we experienced in trying to change MTU on a yet defined (VSX/Multi-Chassis) LAG - per LAG - and this issue  is actually documented on latest ArubaOS-CX 10.01.0020 (concurrently it appears as an Enhancement "Added support for MTU on LAG interfaces" and also as a known issue CR_31631 "Cannot change the MTU if port is a member of a LAG.": it's declared that you finally can manage MTU at LAG level...but, de-facto, you still can't - I wasn't able to find the command [**] - and there is a workaround to overcome this issue...basically moving out a LAG member interface out of its LAG, change MTU and then moving it in again).

[**] Note:

!Version ArubaOS-CX TL.10.01.0020

Aruba-8320-2(config)# interface lag
  <1-256>  LAG number ranges from 1 to 256

Aruba-8320-2(config)# interface lag 1

Aruba-8320-2(config-lag-if)#
  apply          Apply a configuration record
  arp            Configure ARP commands
  bfd            Set BFD configuration
  description    Add an interface description
  end            End current mode and change to enable mode
  exit           Exit current mode and change to previous mode
  hash           The type of hash algorithm used for aggregated port
                 (Default:l3-src-dst)
  ip             IP information
  ipv6           Error: Help strings does not match for the identical tokens
  l3-counters    Enable Rx and Tx L3 counters
  lacp           Configure LACP parameters
  list           Print command list
  loop-protect   Configure loop protection
  mvrp           Enable the Multiple VLAN Registration Protocol (MVRP)
  no             Negate a command or set its defaults
  qos            Quality of Service configuration
  rate-limit     Apply a rate-limit to a specific traffic type for this port
  routing        Configure interface as L3
  sflow          Enable sFlow
  shutdown       Enable/disable a LAG
  spanning-tree  Spanning-tree configuration
  track          Track information
  vlan           VLAN configuration
  vrf            VRF Configuration
  vrrp           VRRP information
  vsx-sync       Enable VSX config sync for specific interface associations

Aruba-8320-2(config-lag-if)# mtu
% There is no matched command.

Aruba-8320-2(config-lag-if)# ip
  address           Set IP address
  bootp-gateway     Interface IP used as source IP for forwarding DHCP requests
  forward-protocol  Configure a forward-protocol on the interface
  helper-address    Configure the helper-address for DHCP relay
  igmp              IGMP Configurations
  irdp              Configure ICMP Router Discovery Protocol
  mtu               Configure the IP MTU for the interface
  ospf              OSPF interface commands
  pim-sparse        Configure the PIM-SM protocol
  proxy-arp         Enable proxy ARP

Correct, the current fix is the following:

say you have int 1/1/49 and 1/1/50 in lag1

on both switches

-1- Enter config context in int 1/1/49

-2- shut  (VSX ISL still up due to 1/1/50 up)

-3- no lag 1

-4- mtu 9198 (example)

-5- lag 1

-6- no shut

-7- show vsx status / show int brief

-8- Enter config context in int 1/1/50

-9- shut  (VSX ISL still up due to 1/1/49 up)

-10- no lag 1

-11- mtu 9198

-12- lag 1

-13- no shut

-14- show vsx status / show int brief

You should have no ISL interruption.

Hello Giles, thanks for your input.

The procedure you detailed above is exactly the same I used for all of our VSX LAGs (a grand total of 22 + 22 interfaces - excluding 4 + 4 of VSX ISL LAG - reconfigured on the fly to change MTU vlaue from default 1500 to 9198)...I was scared [*] to let interfaces with different MTU values (say 1500 and 9198) to coexist on the very same VSX LAG, even if this coexistence lasts for the minimum time needed to apply changes to all LAG member interfaces' MTUs.

[*] to say the whole truth, I'm still scared about reconfiguring MTU values on interfaces that are members of the VSX ISL LAG...now that we left our staging area for full production.

Done. Pretty flawless.

Clearly Aruba VSX nodes are reporting a Warning log similar to:

"Aruba-8320-1 lacpd[1403]: Event|1310|LOG_WARN|AMM|-|Partner is out of sync for interface 1/1/53 LAG sport: 1. Actor state: ALFOX, partner state PSFO"

but having more than one interface belogning to the VSX ISL LAG the VSX ISL didn't lose its sync status.

in 10.2, you can set MTU on interface belonging to LAG.