Hi,
I have an IAP-325 that has been unstable for some time, and seems to repeatedly run into memory issues. It is usually in a cluster with an IAP-315 and an IAP-225, which have not experienced this issue. I first noticed the issue in 2018 and have kept the 325 sitting around in monitor mode since then.
The IAP-325 used to serve clients fine for a bit, but appeared to eventually start dropping packets due to low memory (despite the cluster never having more than 15 simultaneous clients). This is an excerpt of `show tech-suport` from June 2018:
show log driver
520.115122] aruba_mods_ether_sendup : pkt drop due to lowmemi am a potential master now
[75529.629024] i am a master now
[75529.651986] asap_set_swarm_state: 4 callbacks suppressed
[75529.651986] (17:50:04) !!! Pot-Master ---> Master
[75529.708249] Del bridge entry for magic vlan GW, 40:e3:d6:c6:dd:76
[75529.781164] asap_send_elected_master: sent successfully
[75541.271355] Sun Jun 24 17:50:15 2018:2457 75452.130914 Client Match: density report of wifi1 is not updated for a long time.
[75566.568449] aruba_mods_ether_sendup : pkt drop due to lowmem
[75567.569917] aruba_mods_ether_sendup : pkt drop due to lowmem
[75568.572729] aruba_mods_ether_sendup : pkt drop due to lowmem
[75569.668793] aruba_mods_ether_sendup : pkt drop due to lowmem
[75572.271011] Sun Jun 24 17:50:46 2018:2458 75483.130570 Client Match: density report of wifi1 is not updated for a long time.
[75579.973229] aruba_mods_ether_sendup : pkt drop due to lowmem
[75581.444144] aruba_mods_ether_sendup : pkt drop due to lowmem
More recently (this April), after some firmware upgrades, I did a factory reset on the 325 and took another look. I found it was continually rebooting after "out of memory" kernel panics. The full console logs from one boot-to-kernel-panic cycle are attached, but here are the highlights:
APBoot 1.5.5.5 (build 55373)
Built: 2016-06-09 at 11:36:40
Model: AP-32x
< . . . >
Hit <Enter> to stop autoboot: 0
Booting OS partition 0
Checking image @ 0x0
Copying image from 0x44000000
Image is signed; verifying checksum... passed
Signer Cert OK
Policy Cert OK
RSA signature verified.
[ 0.000000]
[ 0.000000] Aruba Networks
[ 0.000000] ArubaOS Version 8.6.0.3-8.6.0.3 (build 74788 / label #74788)
[ 0.000000] Built by p4build@pr-hpn-build10 on 2020-03-20 at 17:22:14 UTC (gcc version 4.6.3 20120201 (prerelease) (Linaro GCC 4.6-2012.02) )
[ 0.000000] CPU: ARMv7 Processor [512f04d0] revision 0 (ARMv7), cr=10c5387d
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[ 0.000000] Machine: IPQ806X wave-2 board
[ 0.000000] Flash variant: default
[ 0.000000] msm_reserve_memory: 0x44600000, 0x200000
[ 0.000000] msm_reserve_memory: 0x44800000, 0x200000
[ 0.000000] Memory policy: ECC disabled, Data cache writealloc
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 57642
[ 0.075570] CPU1: Booted secondary processor
[ 0.102280] clk_tbl_nss - loaded
[ 0.212339] PCI: enabling device 0000:02:00.0 (0140 -> 0143)
[ 0.212527] PCI: enabling device 0000:00:00.0 (0140 -> 0143)
[ 11.962011] [sched_delayed] sched: RT throttling activated
[ 15.659481] m25p80 spi5.0: found mx25u3235f, expected s25fl512s
[ 18.002561] Found AT97SC3203 on i2c-gpio0
[ 18.062293] no pmic restart interrupt specified
[ 18.701749]
[ 18.701749] Starting Kernel SHA1 KAT ...Completed Kernel SHA1 KAT
[ 18.780943] Starting Kernel HMAC-SHA1 KAT ...Starting Kernel DES KAT ...Completed Kernel DES KAT
[ 18.887222] Starting Kernel AES KAT ...Completed Kernel AES KAT
[ 18.930959]
[ 18.976819] Starting Kernel AESGCM KAT ...Completed Kernel HMAC-SHA1 KAT
[ 19.071727] Completed Kernel AESGC�Thu Jan 1 00:00:00 PST 1970
< . . .>
[ 140.472477] USB Serial support registered for cp210x
[ 140.579287] usbcore: registered new interface driver cdc_eem
Reboot caused by kernel panic: Out of memory
shutting down watchdog process (nanny will restart it)...
Completed SW FIPS KAT test
<<<<< Welcome to the Access Point >>>>>
Power supply mode is DC, USB Modem is not present.
User: [ 144.279943] Starting Kernel HMAC SHA1 FIPS KAT ...
[ 144.354545] Completed Kernel HMAC SHA1 FIPS KAT
[ 144.930365] Kernel watchdog refresh ended on core 0.
< . . . >
[ 148.299906] Starting Kernel HMAC SHA384 FIPS KAT ...
[ 148.351171] Completed Kernel HMAC SHA384 FIPS KAT
[ 150.277944] Starting Kernel HMAC SHA512 FIPS KAT ...
[ 150.328803] Completed Kernel HMAC SHA512 FIPS KAT
[ 168.980912] VAP device aruba015 created osifp: (c5c55540) os_if: (c5178000)
[ 169.082474] VAP device aruba115 created osifp: (c66a5540) os_if: (c3848000)
[ 169.395907] ieee80211_ioctl_dbgreq received fips event 2 c50296c0 length 16
Firmware AES-CCM Known Answer Test Passed
[ 170.526866] wlan_mlme_app_ie_delete: appie is NULL. Do nothing.
[ 171.654764] wlan_mlme_app_ie_delete: appie is NULL. Do nothing.
[ 179.391908] i am master now
[ 179.412777] (22:44:41) !!! Init ---> Master
[ 179.462855] asap_send_elected_master: sent successfully
[ 181.554170] asap_send_elected_master: sent successfully
[ 214.100687] cli invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
< . . .>
[ 239.052015] Kernel panic - not syncing: Out of memory
< . . . >
[ 244.144079] Rebooting in 2 seconds..
[ 246.138363] Going down for restart now
Has anyone encountered something similar? Is this AP toast, or is there something else I should look at/try to change? I have a few pairs of tech support dumps (for the 325 and 225, both when 325 was successfully serving clients and when it was not) that I could post as well, if there's value in it.
Thanks for looking