Security

 View Only
Expand all | Collapse all

AOS10 Framed IP Missing From Accounting Start

This thread has been viewed 68 times
  • 1.  AOS10 Framed IP Missing From Accounting Start

    Posted 23 days ago

    Hello All,

    I have AOS10 setup in Old Central on version 10.4.xx. When wireless device auths, accounting start data is sent to ClearPass 6.12.7 without the IP. This data is then discarded by my firewall for lack of IP. The interim packet is then sent after timer expiration setup on the WLAN security configuration. I have Enforce DHCP enabled. I am beating my head against the wall wondering how to force the accounting start data to wait for the IP before sending. The firewall is using this data for group-based policy, so users have to wait minutes to gain access when roaming or reconnecting to wireless for any reason. Anyone have a solution or recommendation for this?

    ClearPass 6.12.7
    Aruba Central AOS10 APs on version 10.8

    Gateway Cluster on version 10.4.1.12



  • 2.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 22 days ago

    so you have your clearpass configured for radius proxy to your firewall? 
    also have you configured interim radius accounting? what is that value set yo? since wireless auth happens before the device gets an ip address, it is the interim accounting that will update the radius server with the IP address of the client.  



    ------------------------------
    If my post was useful accept solution and/or give kudos.
    Any opinions expressed here are solely my own and not necessarily that of HPE or Aruba.
    ------------------------------



  • 3.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 22 days ago

    Thanks for the reply, AP,

    Yes, ClearPass is proxying the accounting data to the firewall. The firewall ignores the accounting start as there is no IP. 5 minutes later the interim accounting is sent with the framed IP, the firewall creates the user group association. The initial wait is not great but also not the end of the world. The major problem comes with anyone roaming and requiring firewall GBP to access secure resources. Every roam is an accounting stop, start, 5-minute wait and interim to recreate the group association and allow access through the firewall.

    I had considered setting the interim accounting as low as it gets, 1 minute, that would still not be ideal and I am concerned with 10 thousand devices sending accounting every minute.




  • 4.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 21 days ago

    what firewall is this? perhaps ClearPass has a direct integration with rather than using Radius proxy. Also you could enforce policy enforcement directly on the AOS10 APs. 



    ------------------------------
    If my post was useful accept solution and/or give kudos.
    Any opinions expressed here are solely my own and not necessarily that of HPE or Aruba.
    ------------------------------



  • 5.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 19 days ago
    Edited by SC-e61e78 19 days ago

    FortiGate, for which there are integrations, however, we do not have, nor want to pay for if avoidable, FortiManager. My findings indicate this is required middleware for FSSO.

    For our environment, the FortiGate must remain the point of authority for secure access, so AP level policy enforcement will not be an option for us.

    What I have changed since we started this conversation to help improve the existing workflow:
    Firewall side, changes marked by +:
     config user radius
         edit "RSSO Agent"
             set rsso enable
             set rsso-radius-response enable
             set rsso-validate-request-secret enable
             set rsso-secret FortinetPasswordMask
             set rsso-endpoint-attribute User-Name
    +        set rsso-flush-ip-session disable
    +        set rsso-ep-one-ip-only enable
    +        set rsso-context-timeout 28800
    +        set rsso-log-period 60
    +        set rsso-log-flags protocol-error profile-missing accounting-stop-missed accounting-event endpoint-block radiusd-other
         next
     end

    Classic Central side VIA CAS:

    radius-interim-accounting-interval 1

    After some consideration and bouncing specs off of AI, I went ahead and ran a test and then distributed to all AP Groups, modifying the WLAN to reduce accounting interim to 1 minute. So far, the sum of these changes has improved usability to the point where people don't notice the difference as much, but there is still that 1-minute delay upon authentication completion. 

    If there are integrations that support ClearPass direct to FortiGate for a more immediate result, I would be interested to hear about them!




  • 6.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 18 days ago

    Im happy to give my feedback although I cant review this week. I have used RSSO with fortigate in a few different scenarios. Clearpass has been the sender of radius accounting messages and I have also used freeradius 3.0 as the sender. In both of these cases it was MM/MD design with aruba controllers with CAP's. I don't recall issues when the user performed auth and having to wait. 

    I was going to generally suggest to lower the interim accounting updates which you have done. I would have to also check the fortigate cli documentation; it does look like you had adjusted timeouts so the user sessions will not get purged prematurely. 

    I have not done any debugging with the framed-ip-address attribute on Campus AP's managed by central. We have been using 10.7 for code for the longest time (past 1+ year). There was a dhcp bug on early 10.8 that impacted us and it seems to be addressed now. I am happy to see what I can find out with framed-ip-address on 10.8 code

    Im not sure if you have grabbed a pcap of the accounting-start packet in comparison to when dhcp takes place. Im also not sure if you known the length of time from when the radius-accept packet is received by AP vs when the accounting-start is send to clearpass? I recall watching debugs in freeradius 3 when I setup the home server and I recall a small gap of time. Maybe it was 5 seconds after the accept message was received. It was enough time for dhcp to take place and the accounting to be started. 

    I did a quick look and dont recall seeing any settings to delay accounting on my controller configs. I would need to go back to baseline before questioning software version. If it was me writing code. Having a small gap from the radius-accept before starting the accounting would make sense. 




  • 7.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 18 days ago

    Another possibility could be to add a more restricted firewall policy after the policy that is requiring rsso authentication. This would provide a fallback option while waiting for the framed-ip to become present. Since fw rules are top->down, the client will hit the proper policy once the sessions clears and inspection for the 5-tuples happens again. If sessions are fast-pathed due to matching the current 5-tuples. They will remain on the old policy. This is the approach I would take if fortigate auth was required in fw policies. 




  • 8.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 12 days ago

    That is a good suggestion, and I do have fall-thru policies for a lot of CAT-2 and below destinations, but that's still of minimal help, I have found.




  • 9.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 12 days ago

    Hey JK, thanks for the reply,

    Are you using the same implementation within Aruba Central now successfully? The AOS8 Mobility Conductor days had a knob for this, where Aruba Central does not.

    I had thought about taking a PCAP for that timing comparison, however, I determined there was nothing I could do with the results at this point without something to adjust in response to it.




  • 10.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 11 days ago
    Edited by JK-90898a 11 days ago

    Im assuming you are using tunneled WLAN to the gateways? As you did list a gateway cluster in your opening message. There are ways to get configs on the gateways through alternate methods. I had used a web based tool to push an incorrect license on a VGW in azure. I believe this would take CLI commands. I can dig up the link although I think its classic central only. 

    Controller-less AP's really just stem from the IAP code chain and I doubt they would have settings to optimize this. I would have to look at the cli guide to see if anything exists. 

    Tunneled AP's with radius proxy to the gateway. I would expect you could perform the similar operations as 8.x controllers. At this point the radius packet is still being passed through the gateway and attributers can be updated. 

    I can provide feedback of my findings when I enable RSSO. My assumption is the standalone AP will not have the ability to perform this. I also know they are changing workflows in CNX right now. My guess is any feature request will be on the back burner. Since there is a fortinet / aruba relationship with this integration. I would also expect that this would like to be preserved and if a feature request needs to be added it would be honored. Not sure if the sales side at fortinet could be pushed to engage in aruba on this behalf as well?




  • 11.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 11 days ago

    Q> "I'm assuming you are using tunneled WLAN to the gateways"?
    A> Correct, for all considerations in this discussion.

    My thought process took me down the same road of intercepting the packet at the Gateway, however, I did not find an available CLI command for accounting.


    I would be very interested in your RSSO findings, especially if you discover a way include IP in acc start.


    I have put a bug in the ear of my SE and the upstream PLM about this; we shall see if anything comes of it.




  • 12.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 11 days ago

    I looked quickly at fortinet documentation. It looks outside the RSSO there is an context endpoint connector (api). Both would require the IP to be present before sending data properly. Since we have multiple fortigates, I most likely will send accounting to FSSO collector vs the firewalls themselves. Depending what I find, I can also send something to PLM. I think my target use case would be windows 11 that are intune managed. 




  • 13.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 11 days ago
    Edited by JK-90898a 11 days ago

    I did a quick debug on this today. I found the radius accounting start had the framed IP. I have listed 2 ways on how you can debug this below. The accounting start was 1 second after the radius accept. I just wanted to provide a way you can debug this as a starting point and provide a baseline. 

    Code: 10.8.1.0
    Design: SD-Branch Gateway on Classic Central. AP is in same group config with tunneled WLAN to the single gateway. 

    Debugging Options

    • AP - show ap onboarding event (this command you can obtain the authtrace-buf output)
    • GW - logging levels can be enabled with "disaster-recover on" 
      • logging security process dot1x-proc level debugging (not needed)
      • logging security process authmgr level debugging (used when not proxy for tunneled WLAN)
      • logging security process radproxy level debugging (new with proxy for tunneled WLAN)
      • logging security subcat aaa level debugging (used in all versions from 6.x)
    • GW PCAP (probably to difficult to do on local system as you have to convert the hex)
      • packet-capture controlpath udp 1812,1813
      • packet-capture destination local-filesystem 
        • show packet-capture controlpath-pcap
      • packet-capture destination ip-address <client-ipv4>

    The pcap is an easy win to send to your pc, although the logging method would be easiest. The Authtrace-buf on gateway with proxy will not show the same as the AP. The AP onboarding command is great and its what is sent to CNX for client connectivity stats. You wont see the framed IP in the onboarding event, although it will show data on when accounting-start happened vs the wpa 4-way handshake. You can see the time stamp of the dhcp, dns, assoc, auth. You can then compare the time to the accounting-start. 

    show ap onboarding event 

    • get client mac or see the events you want on a per AP level
    • use client mac to get the event ID's.
    • Then look at the event ID directly

    show ap onboarding event client-mac 4a:e2:28:c7:fb:d3
    show ap onboarding event client-mac 4a:e2:28:c7:fb:d3 1785278024375

    Gateway Logging

    ** I put this in text to remove some fields and remove some lines. I did not modify the time stamps. I can see the framed IP's listed here just fine in the accounting start. I did change the framed-ip and the vlan-id.  I just didn't feel like sharing corp info. 

    logging security process radproxy level debugging
    logging security subcat aaa level debugging

    show log security 1000 (or what ever number you need for history)




  • 14.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 10 days ago

    That's interesting, I do not know the ramifications of enabling disaster-recovery on my cluster managed GW, so I'm going to avoid that. Documentation describes it similar to "aruba-central support-mode", for switches, but I will have to test that first on a managed cluster.

    Thank you for providing the commands you used. My results from the AP are different from yours and consistent with what I am seeing behavior wise:

    Jul 29 08:47:31.347  rad-acct-start

    DHCP time         : Jul 29 08:47:31.370

    AP-Mname# show ap onboarding event client-mac 00:de:ad:be:Ef:00 1785340051097
     
    Onboarding Event Table
    ----------------------
    mac                bssid              status   phaseID  rcode  ip-ready  arp      dns      start-timestamp      session-id
    ---                -----              ------   -------  -----  --------  ---      ---      ---------------      ----------
    00:de:ad:be:Ef:00  44:12:44:77:8c:90  success  N/A      0      success   success  success  Jul 29 08:47:31.097  1785340051097
     
    Start time        : Jul 29 08:47:31.097
    Assoc resp time   : Jul 29 08:47:31.106
    Auth done time    : Jul 29 08:47:31.343
    DHCP time         : Jul 29 08:47:31.370
    ARP time          : Jul 29 08:47:31.379
    DNS time          : Jul 29 08:47:31.414
    Jul 29 08:47:31.347  rad-acct-start   ->                  00:de:ad:be:Ef:00  44:12:44:77:8c:90/__gw_ip-adres###  -   -     00:de:ad:be:Ef:00



  • 15.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 10 days ago

    It looks as you have 50ms between the dhcp and radius accounting. 

    I have been using DR mode for the last 6 years. It just means config syncs will not be pushed from mgmt until DR mode is turned off.  I just didn't feel like looking in the web for the logging options. When DR mode is turned off the settings you made will be removed. Its device dependent. If you are working on a cluster you may need to debug all devices unless you are positive that traffic is only tunneled to the one node. 

    If you dont want to enable / disable the logs in the web to debug this, you can send a remote PCAP to your machine. I typically use OSX so I can see the GRE packets of the remote pcap. I provided the control-plane options and I have been using it on remote branches for years. I do think its easier to read log file vs radius pcap.

    In my case I was using a 635. I setup a new config and I checked the AP was tunneled to the gateway via the following commands. "show datapath tunnel" or "show tunnelmgr tunnel-list".

    Since the framed-ip is part of the radius start message. My guess is it may be HW dependent or a possible software hiccup. Im happy to share configs or something you may want more of. Just PM me. Im also happy to provide info to ERT if you think its a bug. Just PM me or add me to the case. 




  • 16.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 9 days ago

    I chosen an integration that seems to work for us. The clearpass connector via fortimanager will be used. 

    Due to the setup that we have with multiple firewalls, it did not make sense to use RSSO directly. We had FSSO already in place and accounting sent there. It just was not the most user friendly in all situations as we had to append the domain to the username so FSSO could capture the AD groups. I am not sure if you if your solution was to hard code a class to match in fortigate or to use %{Tips:Role} in some way. I do know that the accounting proxy has limited config for each service. 

    Since we are migrating to windows 11 that is a pure cloud service. We struggle today with FSSO not working. Due to these variables it seemed to make sense to move to the context server option with API. We still have the challenge with needing the group info for users to accesses services vs a unified role of employee, byod, etc. I use the intune extension, and due to processing overhead I use mac caching with boolean value of future date time stamp and a couple services. Since we fetch the intune authorization every 2 days we can save any updated groups to endpoints and use that data in real time to send to fortinet. 

    The clearpass connector via fortinet fabric uses the tips roles and can be linked to local user groups. The cppm context server action is applied on the enforcement rules that you wish to match. 

    This solution was applicable for the following reasons. Multiple Firewalls, using intune/entra groups, leveraging tips roles to create user groups in fortinet, leveraging fortimanager for a central environment. 




  • 17.  RE: AOS10 Framed IP Missing From Accounting Start
    Best Answer

    Posted 6 days ago

    Your two traces point at the gateway. JK's start waited for DHCP behind a single 10.8.1.0 gateway; yours fired 23 ms early through a cluster on 10.4.1.12. On a tunneled WLAN the accounting is generated gateway-side, and radproxy changed plenty between 10.4 and 10.8.

    Quick isolation: one test AP tunneled to a 10.8.x gateway, same WLAN, rerun the onboarding trace. If the start waits there, the fix is upgrading the cluster, not finding a knob.

    Either way those two traces side by side are an ERT case: same trigger, one has Framed-IP, one doesn't, on shipping code. That's a defect, not a feature request.



    ------------------------------
    Dustin Burns

    Lead Mobility Engineer @Worldcom Exchange, Inc.

    ACCX 1271| ACMX 509| ACSP | ACDA | MVP Guru 2022-2023
    If my post was useful accept solution and/or give kudos
    ------------------------------



  • 18.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 5 days ago

    Thanks JK and DB,

    DB, that's a sharp catch lining up the two traces against the gateway versions rather than just the AP model. JK's gateway was on 10.8.x and mine's sitting on 10.4.1.12 for the cluster, so the radproxy change between those versions tracks with what we saw. I noticed that version diff myself but hadn't concluded it was connected it to the trace behavior until you laid it out, thanks for the push!

    We already have a maintenance window scheduled for the gateway cluster, so I'll get it upgraded to 10.8.1 there and rerun the same onboarding trace for comparison. If the accounting-start waits for DHCP on the upgraded code like JK saw, that confirms it's a version issue rather than something I can tune around, and once the results are in, I'll open the ERT case with both traces side by side, DB, like you suggested.

    Will report back once the upgrade's done and I have a fresh trace either way. Appreciate you both digging into this with me.




  • 19.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 2 days ago

    Ok so after updating to the GW Cluster 10.8.1.0, we now have the framed IP in the accounting start packet!!

    This is a huge win for us; I can now turn the interim updates back up to 5 minutes and continue testing.

    Thank you for all the effort and insight JK, and for the ultimate outside observation that helped solve it DB!




  • 20.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 2 days ago

    Thats good news. Thanks for the update.


    The whole 10.4 LSR reminded me of the early versions of 8.x when I was in the VAR space. It was a little bumpy as I ran into issues in 10.6 + 10.7 and TAC was always the same. Glad to hear your issues were already resolved and waiting for you in a new release :-D 




  • 21.  RE: AOS10 Framed IP Missing From Accounting Start

    Posted 2 days ago

    Yeah, this is a relief for sure! Hopefully once we get the rest of our AP groups over to 10.8, we will stop getting random AP kernel panics.