Developer

I'm looking through the list of devices in the IOT protobuf files and grepped out a consolidated list that shows when things appeared:

//
// This is the list as it appeared in 8.7
//
unclassified                            = 0;
arubaBeacon                             = 1;
arubaTag                                = 2;
zfTag                                   = 3;
stanleyTag                              = 4;
virginBeacon                            = 5;
enoceanSensor                           = 6;
enoceanSwitch                           = 7;
iBeacon                                 = 8;
allBleData                              = 9;
RawBleData                              = 10;
eddystone                               = 11;
assaAbloy                               = 12;
arubaSensor                             = 13;
abbSensor                               = 14;
wifiTag                                 = 15;
wifiAssocSta                            = 16;
wifiUnassocSta                          = 17;
mysphera                                = 18;
sBeacon                                 = 19;
wiliot                                  = 20;
ZSD                                     = 21;
serialdata                              = 22;
exposureNotification                    = 23;

//
// These were added in 8.8
//
exposureNotification = 23;
onity = 24;
minew = 25;
google = 26;
polestar = 27;
blyott = 28;
diract = 29;
gwahygiene = 30;

//
// This one was added in 8.9
//
noneBleData = 31;

I'm wondering if somebody has an explanation of what all of them are, and which ones are open (like iBeacon and Eddystone) versus proprietary.  I know for instance that polestar and minew are beacon manufacturers.  I'm guessing those are proprietary formats.  The appearance of "google" in there is harder to figure out - the only google beacon format I know of is Eddystone.

I'd also like more insight into what noneBleData is.  Previously we discussed that future versions might have a way to forward raw advertisement data in the event that the format is unknown to AOS.  Is that what this is?  I'm a little hamstrung at the moment because we're just getting 8.7 now, so I probably can't use that (yet).  But, I'd like my software to be as prepared for it as possible.  The biggest challenge that I have right now is wanting to be able to send extra data to make signed/secure beacons.  I haven't yet really figured out a great way to do that.  Ideally I'd be able to send some other kind of advertisement that the IOT service in AOS would just forward on to me as an opaque string of bytes.  I'm under the impression it doesn't really do that, though.


------------------------------
Christopher Piggott
------------------------------

Hi Chrisopher,

These definition are not really advertisement formats but device classes AOS can identify. This usually happens based on BLE vendor identifiers. Some of these device classes like iBeacon or Eddystone can be decoded by AOS while others can't e.g. minew. 

Pre-AOS 8.9 these device classes the only way to enable BLE data forwarding for custom advertisments formats. Starting for AOS 8.9 or higher this possible by using the new generic device class filters.

Take a look here for some information about these device class filters in AOS 8.8: Device class filters

Unfortunately, I haven't had the time to update the documenation for 8.9 yet but you can find information about the generic device class filters in the 8.9 user guide IoT chapter.

Pre-AOS 8.9 you can get BLE data (opaque sting) beeing forward only for known device classes. Starting with AOS 8.9 you can get BLE data form almost all BLE devices. You just have to device a generic device class filter e.g. using a vendor identifier in the AOS configuration for the devices you are interested it. This should also support your secure/signed BLE devices use case.

I have to check what "noneBleData" is used for.

Regards,

Jens

------------------------------
Jens Fluegel
------------------------------

Interesting, thank you.

So here's what I'm seeing.  I created a beacon with this content:

[23, 255, 111, 253, 143, 180, 144, 132, 160, 49, 117, 99, 127, 124, 106, 57, 167, 152, 107, 208, 28, 156, 22, 253]

Length = 23, type = 0xFF (which means MANUFACTURER SPECIFIC), and the rest is just gibberish.  It seems to work.  I receive this from AOS IOT:

advertisement,addrType=private_non_resolvable,mac=3dc862fe8b82,reporter=fc7ff1c0fd38,reporterName=DAZZLE rssi=-88i,data="02011a03036ffd17166ffd9ffff79a6a6b9b24236cfb0e863db477dafcbbf9" 1641307666076341322

That's in my own format (really it's Influx Line Protocol) but the gist is that the rssi and data both came from bleData.  This device isn't sending out any other beacons / advertisements at all, so it SHOULD be something that AOS doesn't recognize.  My ageFilter is set to 120 seconds, so while this same device may have sent an iBeacon while I was testing last week, it hasn't done so in at least that long.

So this is actually doing exactly what I want it to do.  IWhat worries me is that you have told me a couple of times:

> Pre-AOS 8.9 these device classes the only way to enable BLE data forwarding for custom advertisments formats. Starting for AOS 8.9 or higher this possible by using the new generic device class filters.

but this seems to be working for me on 8.7:

ArubaOS (MODEL: Aruba9012-US), Version 8.7.1.7

I am thinking maybe what I am receiving through bleData is not the same thing you are talking about, because it looks like my 8.7 is doing somethign you said folds in with 8.9.

Currently, I have deviceClassFilter set to all.

Is this expected behavior?

--Chris







​

------------------------------
Christopher Piggott
------------------------------

Original Message:
Sent: Jan 04, 2022 03:48 AM
From: Jens Fluegel
Subject: IOT device types in protobufs

Hi Chrisopher,

These definition are not really advertisement formats but device classes AOS can identify. This usually happens based on BLE vendor identifiers. Some of these device classes like iBeacon or Eddystone can be decoded by AOS while others can't e.g. minew. 

Pre-AOS 8.9 these device classes the only way to enable BLE data forwarding for custom advertisments formats. Starting for AOS 8.9 or higher this possible by using the new generic device class filters.

Take a look here for some information about these device class filters in AOS 8.8: Device class filters

Unfortunately, I haven't had the time to update the documenation for 8.9 yet but you can find information about the generic device class filters in the 8.9 user guide IoT chapter.

Pre-AOS 8.9 you can get BLE data (opaque sting) beeing forward only for known device classes. Starting with AOS 8.9 you can get BLE data form almost all BLE devices. You just have to device a generic device class filter e.g. using a vendor identifier in the AOS configuration for the devices you are interested it. This should also support your secure/signed BLE devices use case.

I have to check what "noneBleData" is used for.

Regards,

Jens

------------------------------
Jens Fluegel
------------------------------

Hi Christopher,

Please check how your device is classified by the AP, that may explain why you get BLE data with 8.7.1.7 using the following command:

show ap debug ble-table ap-name <ap-name> all

The device class "all" is a little bit confusing and is removed in 8.9 and onwards. The "all" basically means all known device classes, as explained in the documentation liked in my previous post. But in 8.7.x BLE data forwarding was only available for specific devices classes and is not configurable.

With AO 8.8. we added the per packet device class filter feature including the capability to forward ble data frames form clients that at least transmit advertisements of a known devices class in parallel to other unknown advertisements formats. This means if BLE e.g. transmits an iBeacon advertisement, a known devices class, and other formats and you enabled BLE data. all advertisements including the custom formats would be forwarded. With 8.7.x this would be unexpected.  

By the way, the ageFilter and others do not apply to BLE data forwarding, just to BLE telemetry.
BLE telemetry is enabled by default and cannot be disabled. This means that you will always get BLE telemetry at the configured report interval and BLE data in addition. 

Regards,

Jens




------------------------------
Jens Fluegel
------------------------------

Original Message:
Sent: Jan 04, 2022 09:56 AM
From: Christopher Piggott
Subject: IOT device types in protobufs

Interesting, thank you.

So here's what I'm seeing.  I created a beacon with this content:

[23, 255, 111, 253, 143, 180, 144, 132, 160, 49, 117, 99, 127, 124, 106, 57, 167, 152, 107, 208, 28, 156, 22, 253]

Length = 23, type = 0xFF (which means MANUFACTURER SPECIFIC), and the rest is just gibberish.  It seems to work.  I receive this from AOS IOT:

advertisement,addrType=private_non_resolvable,mac=3dc862fe8b82,reporter=fc7ff1c0fd38,reporterName=DAZZLE rssi=-88i,data="02011a03036ffd17166ffd9ffff79a6a6b9b24236cfb0e863db477dafcbbf9" 1641307666076341322

That's in my own format (really it's Influx Line Protocol) but the gist is that the rssi and data both came from bleData.  This device isn't sending out any other beacons / advertisements at all, so it SHOULD be something that AOS doesn't recognize.  My ageFilter is set to 120 seconds, so while this same device may have sent an iBeacon while I was testing last week, it hasn't done so in at least that long.

So this is actually doing exactly what I want it to do.  IWhat worries me is that you have told me a couple of times:

> Pre-AOS 8.9 these device classes the only way to enable BLE data forwarding for custom advertisments formats. Starting for AOS 8.9 or higher this possible by using the new generic device class filters.

but this seems to be working for me on 8.7:

ArubaOS (MODEL: Aruba9012-US), Version 8.7.1.7

I am thinking maybe what I am receiving through bleData is not the same thing you are talking about, because it looks like my 8.7 is doing somethign you said folds in with 8.9.

Currently, I have deviceClassFilter set to all.

Is this expected behavior?

--Chris







​

------------------------------
Christopher Piggott
------------------------------


Original Message:
Sent: Jan 04, 2022 03:48 AM
From: Jens Fluegel
Subject: IOT device types in protobufs

Hi Chrisopher,

These definition are not really advertisement formats but device classes AOS can identify. This usually happens based on BLE vendor identifiers. Some of these device classes like iBeacon or Eddystone can be decoded by AOS while others can't e.g. minew. 

Pre-AOS 8.9 these device classes the only way to enable BLE data forwarding for custom advertisments formats. Starting for AOS 8.9 or higher this possible by using the new generic device class filters.

Take a look here for some information about these device class filters in AOS 8.8: Device class filters

Unfortunately, I haven't had the time to update the documenation for 8.9 yet but you can find information about the generic device class filters in the 8.9 user guide IoT chapter.

Pre-AOS 8.9 you can get BLE data (opaque sting) beeing forward only for known device classes. Starting with AOS 8.9 you can get BLE data form almost all BLE devices. You just have to device a generic device class filter e.g. using a vendor identifier in the AOS configuration for the devices you are interested it. This should also support your secure/signed BLE devices use case.

I have to check what "noneBleData" is used for.

Regards,

Jens

------------------------------
Jens Fluegel

I guess the only  mystery to me, then, is why do I sometimes get bleData.  A few times I've gone looking for that MAC in the debug table (show ble table ...) and these senders just don't appear.

nbTopic == NbTopic_bleData
Reported = nil

in BleData the BleFrameType is: BleFrameType_adv_nonconn_ind (2)  and the address type is MacAddrType_addr_type_private_non_resolvable (2)


In this case the contents are

02 01 1a    (incomplete list of 16-bit UUIDs?)
03 03 6f fd  (incomplete list of 16-bit UUIDs?)
17 16 6f fd 3a 28 79 6e 32 d9 f4 0e ac 2e 51 2c d3 21 17 50 ec b4 ca fe  (public address?)


so this is a pretty strange message.  It actually looks like the first should be 0x03 0x01 0x1a which would be flags in a normal advertisement.


So I still feel like something strange is going on here, and in light of this source address not being classified I'm still wondering why I get bleData at all.  I wonder if the fact that the address type is private/non-resolvable is what's making the difference here as I notice none of those types of addressess show up in "show ap debug ble-table ap-name DAZZLE all" 

Hi Christopher,

These bleData pakets you see are very likely exposure-notifications. You ore someone else in the area may have a COVID-19 app on their smartphone sending these exposure-notifications.

https://www.google.com/covid19/exposurenotifications/
https://developer.apple.com/documentation/exposurenotification

We have added support for exposure-notifications with 8.7.0.0 and these are just forwarded and are not saved in the APs BLE table because this would fill up the tables quickly because of randomized MAC addresses and so on.

Because you enabled device calls "all" exposure-notifications are a know device class and forwarded by using BLE data only, not via BLE telemetry.

If you change your configuration to a device device class e.g. iBeacon only, these BLE advertisements should no longer be forwarded.

Regards,

Jens





------------------------------
Jens Fluegel
------------------------------

Original Message:
Sent: Jan 06, 2022 08:08 PM
From: Christopher Piggott
Subject: IOT device types in protobufs

I guess the only  mystery to me, then, is why do I sometimes get bleData.  A few times I've gone looking for that MAC in the debug table (show ble table ...) and these senders just don't appear.

nbTopic == NbTopic_bleData
Reported = nil

in BleData the BleFrameType is: BleFrameType_adv_nonconn_ind (2)  and the address type is MacAddrType_addr_type_private_non_resolvable (2)


In this case the contents are

02 01 1a    (incomplete list of 16-bit UUIDs?)
03 03 6f fd  (incomplete list of 16-bit UUIDs?)
17 16 6f fd 3a 28 79 6e 32 d9 f4 0e ac 2e 51 2c d3 21 17 50 ec b4 ca fe  (public address?)


so this is a pretty strange message.  It actually looks like the first should be 0x03 0x01 0x1a which would be flags in a normal advertisement.


So I still feel like something strange is going on here, and in light of this source address not being classified I'm still wondering why I get bleData at all.  I wonder if the fact that the address type is private/non-resolvable is what's making the difference here as I notice none of those types of addressess show up in "show ap debug ble-table ap-name DAZZLE all" 

We have added support for exposure-notifications with 8.7.0.0 and these are just forwarded and are not saved in the APs BLE table because this would fill up the tables quickly because of randomized MAC addresses and so on.


You are so valuable to the community - THANK YOU.   This all makes sense now.


So back to the original discussion about device classifiers: what if a device sends a recognized beacon format like an iBeacon, then in a subsequent advertisement sends something proprietary.  The ble-table will have an entry for that device and it will be classified as an iBeacon - do I understand from a previous post that it will then forward other advertisements from that MAC address (at least for as long as it's still in the AP's ble-table) even if the contents are proprietary?  Or is the answer still "no, unless you're in 8.9"  ?




------------------------------
Christopher Piggott
------------------------------

Original Message:
Sent: Jan 07, 2022 06:18 AM
From: Jens Fluegel
Subject: IOT device types in protobufs

Hi Christopher,

These bleData pakets you see are very likely exposure-notifications. You ore someone else in the area may have a COVID-19 app on their smartphone sending these exposure-notifications.

https://www.google.com/covid19/exposurenotifications/
https://developer.apple.com/documentation/exposurenotification

We have added support for exposure-notifications with 8.7.0.0 and these are just forwarded and are not saved in the APs BLE table because this would fill up the tables quickly because of randomized MAC addresses and so on.

Because you enabled device calls "all" exposure-notifications are a know device class and forwarded by using BLE data only, not via BLE telemetry.

If you change your configuration to a device device class e.g. iBeacon only, these BLE advertisements should no longer be forwarded.

Regards,

Jens





------------------------------
Jens Fluegel
------------------------------


Original Message:
Sent: Jan 06, 2022 08:08 PM
From: Christopher Piggott
Subject: IOT device types in protobufs

I guess the only  mystery to me, then, is why do I sometimes get bleData.  A few times I've gone looking for that MAC in the debug table (show ble table ...) and these senders just don't appear.

nbTopic == NbTopic_bleData
Reported = nil

in BleData the BleFrameType is: BleFrameType_adv_nonconn_ind (2)  and the address type is MacAddrType_addr_type_private_non_resolvable (2)


In this case the contents are

02 01 1a    (incomplete list of 16-bit UUIDs?)
03 03 6f fd  (incomplete list of 16-bit UUIDs?)
17 16 6f fd 3a 28 79 6e 32 d9 f4 0e ac 2e 51 2c d3 21 17 50 ec b4 ca fe  (public address?)


so this is a pretty strange message.  It actually looks like the first should be 0x03 0x01 0x1a which would be flags in a normal advertisement.


So I still feel like something strange is going on here, and in light of this source address not being classified I'm still wondering why I get bleData at all.  I wonder if the fact that the address type is private/non-resolvable is what's making the difference here as I notice none of those types of addressess show up in "show ap debug ble-table ap-name DAZZLE all" 

I am happy to help. You can support my work with Kudos ;-).

Yes, forwarding proprietary advertisements as you describe it would work but only if you are using AOS/Instant OS 8.8 (per frame filter disabled).
With 8.7.x or lower this is not possible, unfortunately.
With 8.9 you don't need this approach anymore because you can use the generic filters to classify you devices on your own.

------------------------------
Jens Fluegel
------------------------------

Original Message:
Sent: Jan 07, 2022 09:29 AM
From: Christopher Piggott
Subject: IOT device types in protobufs

We have added support for exposure-notifications with 8.7.0.0 and these are just forwarded and are not saved in the APs BLE table because this would fill up the tables quickly because of randomized MAC addresses and so on.


You are so valuable to the community - THANK YOU.   This all makes sense now.


So back to the original discussion about device classifiers: what if a device sends a recognized beacon format like an iBeacon, then in a subsequent advertisement sends something proprietary.  The ble-table will have an entry for that device and it will be classified as an iBeacon - do I understand from a previous post that it will then forward other advertisements from that MAC address (at least for as long as it's still in the AP's ble-table) even if the contents are proprietary?  Or is the answer still "no, unless you're in 8.9"  ?




------------------------------
Christopher Piggott
------------------------------


Original Message:
Sent: Jan 07, 2022 06:18 AM
From: Jens Fluegel
Subject: IOT device types in protobufs

Hi Christopher,

These bleData pakets you see are very likely exposure-notifications. You ore someone else in the area may have a COVID-19 app on their smartphone sending these exposure-notifications.

https://www.google.com/covid19/exposurenotifications/
https://developer.apple.com/documentation/exposurenotification

We have added support for exposure-notifications with 8.7.0.0 and these are just forwarded and are not saved in the APs BLE table because this would fill up the tables quickly because of randomized MAC addresses and so on.

Because you enabled device calls "all" exposure-notifications are a know device class and forwarded by using BLE data only, not via BLE telemetry.

If you change your configuration to a device device class e.g. iBeacon only, these BLE advertisements should no longer be forwarded.

Regards,

Jens





------------------------------
Jens Fluegel


Original Message:
Sent: Jan 06, 2022 08:08 PM
From: Christopher Piggott
Subject: IOT device types in protobufs

I guess the only  mystery to me, then, is why do I sometimes get bleData.  A few times I've gone looking for that MAC in the debug table (show ble table ...) and these senders just don't appear.

nbTopic == NbTopic_bleData
Reported = nil

in BleData the BleFrameType is: BleFrameType_adv_nonconn_ind (2)  and the address type is MacAddrType_addr_type_private_non_resolvable (2)


In this case the contents are

02 01 1a    (incomplete list of 16-bit UUIDs?)
03 03 6f fd  (incomplete list of 16-bit UUIDs?)
17 16 6f fd 3a 28 79 6e 32 d9 f4 0e ac 2e 51 2c d3 21 17 50 ec b4 ca fe  (public address?)


so this is a pretty strange message.  It actually looks like the first should be 0x03 0x01 0x1a which would be flags in a normal advertisement.


So I still feel like something strange is going on here, and in light of this source address not being classified I'm still wondering why I get bleData at all.  I wonder if the fact that the address type is private/non-resolvable is what's making the difference here as I notice none of those types of addressess show up in "show ap debug ble-table ap-name DAZZLE all" 

devices for industrial use are often deployed in places without an ethernet or WiFi connection and have to use a mobile broadband connection. To reduce traffic costs, it is required to keep the amount of data that has to be transmitted as low as possible.

------------------------------
Gunner Kreiger
Liteblue

------------------------------