Wired Intelligent Edge

Scenario: During a PXE boot DHCP transaction, the client will loop through DHCP discover and offer transactions for approx 3 minutes/180 seconds if dot1x is enabled on the port (even though MAC auth is used for this device at this point). After the delay it will then complete successfully.

This delay was not experienced on our pre CX network, and if we disable aaa authentication port-access dot1x authenticator on the port, there is never a three minute delay. This delay is not experienced every time, but more often than not.

Has anyone had a similar experience? We have tried a lot of troubleshooting options (dhcp server options, changing ip helper, ipxe tftp server locations, scripts, different devices, different switches, VLANs, tunneled/locally switched etc), but fundamentally the delay can be triggered on and off by removing the dot1x port config, which shouldn't be involved in the transaction. The discover and offers can be seen on both the DHCP server logs and on wireshark captures of the ports. There are no differences between the first DHCP discover and the one that is finally used in the completed DORA exchange after three minutes.

Hello

I can only make guesses as you did not share your configuration.

AOS-CX switches have an ordered sequence of the authentication types by default. First 802.1.X is tried and after that MAC-Auth. With default timers the switch takes about 160s before MAC authentication kicks in. The default of AOS-S switches is different. They do a parallel authentication instead.

The easiest way would be to configure parallel authentication on AOS-CX switches too, which is archived with the following interface configuration

interface x/y/z
  port-access onboarding-method concurrent enable

Other options would be to change order of authentication types (MAC-auth first , 802.1X second) or to modify the 802.1X timers

Regards

Holger

Scenario: During a PXE boot DHCP transaction, the client will loop through DHCP discover and offer transactions for approx 3 minutes/180 seconds if dot1x is enabled on the port (even though MAC auth is used for this device at this point). After the delay it will then complete successfully.

This delay was not experienced on our pre CX network, and if we disable aaa authentication port-access dot1x authenticator on the port, there is never a three minute delay. This delay is not experienced every time, but more often than not.

Has anyone had a similar experience? We have tried a lot of troubleshooting options (dhcp server options, changing ip helper, ipxe tftp server locations, scripts, different devices, different switches, VLANs, tunneled/locally switched etc), but fundamentally the delay can be triggered on and off by removing the dot1x port config, which shouldn't be involved in the transaction. The discover and offers can be seen on both the DHCP server logs and on wireshark captures of the ports. There are no differences between the first DHCP discover and the one that is finally used in the completed DORA exchange after three minutes.

Hello Holger, thanks for taking the time to reply,

• We have the port-access onboarding-method concurrent enable command in place.
• I have tried altering the 802.1x timers (max-eapol-requests, max-retries), but this doesn't have any effect on the intermittent lag.
• I have also added the command aaa authentication port-access auth-priority mac-auth dot1x again without effect.

The port configuration is as below, not including the changes attempted above.

interface 3/1/22
    description 030/361
    no shutdown
    no routing
    vlan trunk native 101
    vlan trunk allowed 101
    spanning-tree bpdu-guard
    spanning-tree root-guard
    spanning-tree tcn-guard
    spanning-tree port-type admin-edge
    port-access onboarding-method concurrent enable
    aaa authentication port-access allow-lldp-bpdu
    no aaa authentication port-access allow-lldp-auth
    aaa authentication port-access client-limit 5
    aaa authentication port-access radius-override enable
    port-access allow-flood-traffic enable
    aaa authentication port-access dot1x authenticator
        enable
    aaa authentication port-access mac-auth
        enable
    loop-protect
    exit

Thanks!

Hello

I can only make guesses as you did not share your configuration.

AOS-CX switches have an ordered sequence of the authentication types by default. First 802.1.X is tried and after that MAC-Auth. With default timers the switch takes about 160s before MAC authentication kicks in. The default of AOS-S switches is different. They do a parallel authentication instead.

The easiest way would be to configure parallel authentication on AOS-CX switches too, which is archived with the following interface configuration

interface x/y/z
  port-access onboarding-method concurrent enable

Other options would be to change order of authentication types (MAC-auth first , 802.1X second) or to modify the 802.1X timers

Regards

Holger

Scenario: During a PXE boot DHCP transaction, the client will loop through DHCP discover and offer transactions for approx 3 minutes/180 seconds if dot1x is enabled on the port (even though MAC auth is used for this device at this point). After the delay it will then complete successfully.

This delay was not experienced on our pre CX network, and if we disable aaa authentication port-access dot1x authenticator on the port, there is never a three minute delay. This delay is not experienced every time, but more often than not.

Has anyone had a similar experience? We have tried a lot of troubleshooting options (dhcp server options, changing ip helper, ipxe tftp server locations, scripts, different devices, different switches, VLANs, tunneled/locally switched etc), but fundamentally the delay can be triggered on and off by removing the dot1x port config, which shouldn't be involved in the transaction. The discover and offers can be seen on both the DHCP server logs and on wireshark captures of the ports. There are no differences between the first DHCP discover and the one that is finally used in the completed DORA exchange after three minutes.

I would check the port-access status (detail) on the port when you see this issue. The port is probably in a state that blocks the DHCP requests or the responses (do you have dhcp snooping enabled maybe??).

The vlan trunk native & allowed is also a bit strange, as the default port mode is access, and you did not have vlan access 101 on the port. So the client with MAC authentication, if no VLAN is returned (or role with VLAN), the client may end up in VLAN1. The show port-access status detail will tell you the VLAN, ACLs, etc, so good point to start with.

Hello Holger, thanks for taking the time to reply,

• We have the port-access onboarding-method concurrent enable command in place.
• I have tried altering the 802.1x timers (max-eapol-requests, max-retries), but this doesn't have any effect on the intermittent lag.
• I have also added the command aaa authentication port-access auth-priority mac-auth dot1x again without effect.

The port configuration is as below, not including the changes attempted above.

interface 3/1/22
    description 030/361
    no shutdown
    no routing
    vlan trunk native 101
    vlan trunk allowed 101
    spanning-tree bpdu-guard
    spanning-tree root-guard
    spanning-tree tcn-guard
    spanning-tree port-type admin-edge
    port-access onboarding-method concurrent enable
    aaa authentication port-access allow-lldp-bpdu
    no aaa authentication port-access allow-lldp-auth
    aaa authentication port-access client-limit 5
    aaa authentication port-access radius-override enable
    port-access allow-flood-traffic enable
    aaa authentication port-access dot1x authenticator
        enable
    aaa authentication port-access mac-auth
        enable
    loop-protect
    exit

Thanks!

Hello

I can only make guesses as you did not share your configuration.

AOS-CX switches have an ordered sequence of the authentication types by default. First 802.1.X is tried and after that MAC-Auth. With default timers the switch takes about 160s before MAC authentication kicks in. The default of AOS-S switches is different. They do a parallel authentication instead.

The easiest way would be to configure parallel authentication on AOS-CX switches too, which is archived with the following interface configuration

interface x/y/z
  port-access onboarding-method concurrent enable

Other options would be to change order of authentication types (MAC-auth first , 802.1X second) or to modify the 802.1X timers

Regards

Holger

Scenario: During a PXE boot DHCP transaction, the client will loop through DHCP discover and offer transactions for approx 3 minutes/180 seconds if dot1x is enabled on the port (even though MAC auth is used for this device at this point). After the delay it will then complete successfully.

This delay was not experienced on our pre CX network, and if we disable aaa authentication port-access dot1x authenticator on the port, there is never a three minute delay. This delay is not experienced every time, but more often than not.

Has anyone had a similar experience? We have tried a lot of troubleshooting options (dhcp server options, changing ip helper, ipxe tftp server locations, scripts, different devices, different switches, VLANs, tunneled/locally switched etc), but fundamentally the delay can be triggered on and off by removing the dot1x port config, which shouldn't be involved in the transaction. The discover and offers can be seen on both the DHCP server logs and on wireshark captures of the ports. There are no differences between the first DHCP discover and the one that is finally used in the completed DORA exchange after three minutes.

Hi Herman,

We have made changes to the config to reflect your comments.

• Turned DHCP snooping off - No change in behaviour.
• Removed the vlan trunk configuration,  but this also did not solve the issue.

It looks like a bug, but we have also tried the latest firmware, and have had no success.

 I have attached some port access debugging logs, one for when dot1x is enabled and one without. You can see the 3 minute lag quite clearly within this too. During that period the only message logged are as below:

I would check the port-access status (detail) on the port when you see this issue. The port is probably in a state that blocks the DHCP requests or the responses (do you have dhcp snooping enabled maybe??).

The vlan trunk native & allowed is also a bit strange, as the default port mode is access, and you did not have vlan access 101 on the port. So the client with MAC authentication, if no VLAN is returned (or role with VLAN), the client may end up in VLAN1. The show port-access status detail will tell you the VLAN, ACLs, etc, so good point to start with.

Hello Holger, thanks for taking the time to reply,

• We have the port-access onboarding-method concurrent enable command in place.
• I have tried altering the 802.1x timers (max-eapol-requests, max-retries), but this doesn't have any effect on the intermittent lag.
• I have also added the command aaa authentication port-access auth-priority mac-auth dot1x again without effect.

The port configuration is as below, not including the changes attempted above.

interface 3/1/22
    description 030/361
    no shutdown
    no routing
    vlan trunk native 101
    vlan trunk allowed 101
    spanning-tree bpdu-guard
    spanning-tree root-guard
    spanning-tree tcn-guard
    spanning-tree port-type admin-edge
    port-access onboarding-method concurrent enable
    aaa authentication port-access allow-lldp-bpdu
    no aaa authentication port-access allow-lldp-auth
    aaa authentication port-access client-limit 5
    aaa authentication port-access radius-override enable
    port-access allow-flood-traffic enable
    aaa authentication port-access dot1x authenticator
        enable
    aaa authentication port-access mac-auth
        enable
    loop-protect
    exit

Thanks!

Hello

I can only make guesses as you did not share your configuration.

AOS-CX switches have an ordered sequence of the authentication types by default. First 802.1.X is tried and after that MAC-Auth. With default timers the switch takes about 160s before MAC authentication kicks in. The default of AOS-S switches is different. They do a parallel authentication instead.

The easiest way would be to configure parallel authentication on AOS-CX switches too, which is archived with the following interface configuration

interface x/y/z
  port-access onboarding-method concurrent enable

Other options would be to change order of authentication types (MAC-auth first , 802.1X second) or to modify the 802.1X timers

Regards

Holger

Scenario: During a PXE boot DHCP transaction, the client will loop through DHCP discover and offer transactions for approx 3 minutes/180 seconds if dot1x is enabled on the port (even though MAC auth is used for this device at this point). After the delay it will then complete successfully.

This delay was not experienced on our pre CX network, and if we disable aaa authentication port-access dot1x authenticator on the port, there is never a three minute delay. This delay is not experienced every time, but more often than not.

Has anyone had a similar experience? We have tried a lot of troubleshooting options (dhcp server options, changing ip helper, ipxe tftp server locations, scripts, different devices, different switches, VLANs, tunneled/locally switched etc), but fundamentally the delay can be triggered on and off by removing the dot1x port config, which shouldn't be involved in the transaction. The discover and offers can be seen on both the DHCP server logs and on wireshark captures of the ports. There are no differences between the first DHCP discover and the one that is finally used in the completed DORA exchange after three minutes.