Wireless Access

Hello,

We have 2 x virtual Mobility Masters tha tare working just fine.  We also have 14 local controller (2 controllers per site/location) that communciate with the MMM with an IP sec tunnel.

Today, I noticed that 2 different local controllers (at different locations) lost its communication with the MM.  In the past we needed to remove the IPsec confioguration on both sides (MD & The MM) and then re-add them and that has fixed the problem.

My question is.. Is tehre a way to find out when the MM stopped commnciating with the rouge devices?  I know they use port 4500; but, the questionis when exactly did the controlelrs stop communicating with the MM?

So far the only thing I can find is if I logon to the Web UI of the local controllers and navigate to the 'Dashbouard' - 'Controller' section.  Then display readsd when the controller last commhnicated with teh license server (MM V.I.P. is the license server) ; but, it does not provide a specifci time.  Only the date is provided.

Is there a log file tht can privide the specific time it lost communication with each other?  Perhaps on the MM?  Or is that something that is provide din Airwave or ClearPass?

Show log security all

show log system all

Run both of these commands on the MM and MD.

On the MM, this is what you will see when the MD stops communicating with it in the security log:

Oct 21 09:59:28 :103103: <5900> <WARN> |ike| DPD PEER DEAD: peer 192.168.1.7 port:4500
Oct 21 09:59:28 :103103: <5900> <WARN> |ike| IPSec SA Deletion: IPSEC_delSa SPI:25da3600 OppSPI:851e5d00 Dst:192.168.1.7 Src:192.168.1.5 flags:1001 dstPort:0 srcPort:0
Oct 21 09:59:28 :103103: <5900> <WARN> |ike| IKE SA Deletion: IKE2_delSa peer:192.168.1.7:4500 id:2434186925 errcode:ERR_IKESA_CLEARED saflags:0x10000029 arflags:0x200
Oct 21 09:59:31 :199801: <20225> <INFO> |sshd| Accepted password for admin from 192.168.1.237 port 49486 ssh2

On the MD this is what you will see in the security log:

Oct 21 09:59:04 :103103:  <8556> <WARN> |ike|   IKE SA Deletion: IKE2_delSa peer:192.168.1.248:4500 id:2974080627 errcode:ERR_IKESA_CLEARED saflags:0x10000009 arflags:0x5
Oct 21 09:59:30 :103103:  <8556> <WARN> |ike|   DPD PEER DEAD: peer 192.168.1.5 port:4500
Oct 21 09:59:30 :103103:  <8556> <WARN> |ike|   IPSec SA Deletion: IPSEC_delSa SPI:851e5d00 OppSPI:25da3600 Dst:192.168.1.5 Src:192.168.1.7 flags:19 dstPort:0 srcPort:0
Oct 21 09:59:30 :103103:  <8556> <WARN> |ike|   IKE SA Deletion: IKE2_delSa peer:192.168.1.5:4500 id:2974080581 errcode:ERR_IKESA_CLEARED saflags:0x1100002d arflags:0x200

it appears that the logs from teh MM only go back like 5 minutes with the command : show log security all<enter>

I am looking for a couple of day s back.  5 mnites does not seem lik e along time for me.

You should do a "show log all | include <ip address of md>" to see if there is any information in there.  If not, the logs have already rolled and the information is gone.

The logs do roll after retaining a certain amount of information, so configuring a syslog server is the only way to ensure everything is kept indefinitely.

"Show crypto ipsec sa peer <ip address of md>" is a way of showing how long an ipsec tunnel from and MD has been up.

>#show log all | include <IP_address> 

Appears to give some information.  BUt I am not abel to see a different between the controlelrs that are conencted correctly and teh ones that are not.  

Except the md's that are not connected keep displaying <debug> emessages while the other controlelrs do not.  Perhaps Aruba SUpport can decifer the log inforamtion better?

"show log security all | include ike" would narrow it down.

From this web site: https://community.arubanetworks.com/t5/Controller-Based-WLANs/Understanding-and-Troubleshooting-Master-Local-controller-issues/ta-p/239727

I found the command:  show master-local stats

If ececuted from the MM it proivdes missed Heartbeat from local controllers, teh columns are:

Missed -> HB Req from Local(s)

IP Address, HB Req,  HB Resp,  Cfg Terminate, 

Peer Reset,  Total Missed,  Last Sent Missed,

Last Synced/First Missed

The key column to see is the last: Last synched/First Missed

I see the controlelrs that I wanted information on state:

Fri Oct 18 16:58:38 2019/Sat Oct 19 07:25:05 2019

So something must have happened Saturday, 10/19 at 7:25 am their local time.