Skip to content

RaftBLEManagerSysMod

Jump to bottom
Rob Dobson edited this page Oct 25, 2026 · 8 revisions

BLEManager SysMod

Drives the device's Bluetooth Low Energy stack on top of NimBLE. Provides a BLE peripheral that exposes Raft's command/response GATT service (and optional standard services like Battery / Device Info / Heart Rate), and an optional BLE central role for scanning advertisements (e.g. BTHome) and feeding them into the device's bus subsystem.

Source: RaftSysMods/components/BLEManager/

Overview

BLEManager wraps the NimBLE host that ships with ESP-IDF and integrates it with the rest of Raft:

  • It is a SysMod, so it participates in the standard setup() / loop() cycle and is configured via SysType JSON under the key BLEMan.
  • It registers a Comms Channel so command/response traffic from a connected BLE central is exchanged with the rest of the device using the same RICREST framing as WiFi/Serial.
  • It runs its own GAP/GATT event handling (advertise → connect → MTU/PHY exchange → connection-parameter update) and exposes named values and REST endpoints for control.

The two roles can be enabled independently:

Role Config Purpose
Peripheral peripheral: 1 (default) Advertise; serve one central (phone / browser / dongle) at a time, with a second connection accepted while one is connected (see Connections); expose the command-response GATT service.
Central central: 1 Scan continuously for advertisements (passive or active); decode known formats (e.g. BTHome) and forward to a Bus named by busConnName. Outbound connections from the central are not currently supported.

SysType configuration

Full reference: BLE Manager Settings.

Common keys at a glance:

"BLEMan": {
  "enable": 1,
  "peripheral": 1,
  "central": 0,

  // Throughput / latency
  "mtuSize": 512,
  "maxPktLen": 500,
  "connIntvPrefMs": 15,
  "sendUseInd": 1,
  "outQSize": 30,
  "pubQSize": 30,
  "pubQMaxBytes": 32768,

  // Advertising (manufacturer data: the company ID then the end of the serial number, 8 bytes in all - only
  // advertised when advManufValueBytes is set, and the device has a serial number)
  "advIntervalMs": 200,
  "advManufID": "0x02e5",
  "advManufValue": "serialNo",
  "advManufValueBytes": 8,

  // Filter UUID (used by web BLE / RaftJS to find this device)
  "uuidCmdRespService":  "aa76677e-9cfd-4626-a510-0d305be57c8d",
  "uuidCmdRespCommand":  "aa76677e-9cfd-4626-a510-0d305be57c8e",
  "uuidCmdRespResponse": "aa76677e-9cfd-4626-a510-0d305be57c8f"
}

The uuidCmdResp* triple defines the command/response service and characteristics (see GATT layout). Leave them at their defaults unless you have a reason to change them — RaftJS and the example WebUIs use the defaults.

Peripheral mode (default)

In peripheral mode BLEManager performs the standard NimBLE flow:

  1. Initialise the host stack (once per boot).
  2. Register the Raft command/response service plus any standard services selected via stdServices.
  3. Start advertising using the configured advIntervalMs, advertising name, manufacturer data, and filter UUID.
  4. On connection, negotiate MTU and (where the central permits) the preferred connection interval, link-layer packet length and PHY.
  5. Pump inbound writes into the comms channel and dispatch outbound CommsChannelMsg traffic over notifications or indications (sendUseInd, pubUseInd).
  6. Track RSSI and connection state; expose them via named values and getStatusJSON().

Up to maxConns centrals (default 2) can be connected, and one of them - the one that sends commands - is served as the session; see Connections. The advertising name is built from the system's friendly name (or getSystemName() / getSystemUniqueString() if that is unset) — see BLEManager::getAdvertisingInfo().

Connections

A central can connect without wanting a session. The Windows Bluetooth stack, for example, connects every few seconds for about 3.5 s, probably to refresh its cache of the device's services, after a browser has used the device. If the peripheral stopped advertising while connected, as it used to, a browser looking for it would only see it between those visits. So, with maxConns 2 (the default):

  • The device keeps advertising while fewer than two centrals are connected.
  • A connection becomes the session when it sends its first command. The comms channel sends on the session's connection.
  • If the second connection sends a command, it becomes the session and the first is disconnected (the newest central that talks wins). Two clients on different devices that both reconnect when they lose the connection would take it from each other in turn.
  • While two centrals are connected, one that has sent no command for idleConnDropMs (default 8 s) after connecting is disconnected.
  • A second subscription to the response characteristic doesn't change the session's.

All the apps on one computer share its connection to a device (Windows does this, and Chrome between tabs), so a second connection only comes from a second computer or phone. A Windows cache visit and a browser on the same PC share one connection, which becomes the session when the browser sends its first command.

Only a session counts as connected for the conn named value and for SysManager's pauseWiFiforBLE: a central that connects and sends nothing doesn't pause WiFi. maxConns 1 keeps the previous behaviour (no advertising while connected). NimBLE's CONFIG_BT_NIMBLE_MAX_CONNECTIONS must be at least maxConns.

Releasing the link

A client should send the bledisconnect REST command when it finishes - including on errors and when it is interrupted. It disconnects every connected central about 200 ms after replying. Some hosts don't drop the link when the client does:

  • Windows keeps the link up after a client disconnects or its process exits. A client that ends without bledisconnect (e.g. it is killed) leaves the link held until Windows ends it, about 25 s later in tests with RaftSysMods 1.27. Windows doesn't unsubscribe from the response characteristic when the process dies, so a held link can't be told apart from a live, idle one. A new client on the same PC meanwhile reuses the held link (and its bledisconnect releases it).
  • With maxConns 2 the device keeps advertising while a link is held, so another central can still connect and take over the session. The remaining cost is that the session - and with pauseWiFiforBLE the WiFi pause - lasts until the link ends.
  • With maxConns 1 (or firmware before RaftSysMods 1.26.0) the device doesn't advertise while the link is held, and held links lasting more than 3 minutes were seen, so the device can't be found over BLE until it is released (e.g. bledisconnect sent over serial or WiFi) or restarted.
  • Windows may also connect to the device by itself (e.g. to refresh its cache of the device's services). Such a connection sends no command, so it never becomes the session.

GATT service layout

The Raft command-response service has three characteristics:

Characteristic Properties Direction Purpose
uuidCmdRespCommand Write, Write-without-response Central → Device Inbound RICREST frames (commands, file blocks, etc.).
uuidCmdRespResponse Notify or Indicate Device → Central Outbound RICREST frames (responses, publishes). Responses use indications if sendUseInd (default), publishes if pubUseInd (default off, so notifications).
(response, read) Read Central → Device (snapshot) Last response value (rarely used by clients; the notify/indicate path is the primary one).

Subscribing: the device only sends on the response characteristic once the session's connection has subscribed to it. A subscription counts when the central enables notifications, indications or both in the characteristic's CCCD (btleplug on Windows, for example, enables only indications), and only a subscription to the response characteristic counts - one to another characteristic (e.g. Service Changed) doesn't switch replies on or off. Whichever the central enables, the device sends using sendUseInd / pubUseInd. (Before RaftSysMods 1.26.0 only the notify bit counted, so a central that enabled only indications got no replies.)

Packet size: each notification or indication carries up to the current connection's ATT MTU − 12 bytes, read from NimBLE (ble_att_mtu(), via BLEGattServer::getConnMTU()) each time a packet is sized, so it is right for the connection even if the central never exchanges the MTU; maxPktLen is used only when there is no connection MTU. An HDLC-framed RICREST message larger than that is split across several packets (and a command from the central can span several writes). See Communications Stack Overview and RICREST Protocol for the wire format.

sendUseInd (responses and other non-publish messages) and pubUseInd (publishes) choose the transport:

  • Indicate (sendUseInd: 1, the default for responses) — every message is acknowledged at the BLE link layer; only one indication is in flight at a time (500 ms timeout, outMsgsInFlightMs). Reliable but rate-limited by the connection interval.
  • Notify (pubUseInd: 0, the default for publishes) — fire-and-forget; multiple notifications can leave per connection event so achievable throughput is much higher, at the cost of central-side ordering/loss handling. Notifications are spaced at least minMsBetweenSends (default 5 ms) apart.

bleconfig/cmdUseInd=0/pubUseInd=1 (either or both, separated by / rather than &) changes these at runtime.

Standard services

Any subset of the standard Bluetooth services can be enabled via the stdServices array:

Service Use
DeviceInfo Manufacturer / model / firmware-version strings shown by phones and OS Bluetooth UIs.
Battery Battery percentage. Source value is read from another SysMod's named-value via sysMod + namedValue.
HeartRate Heart-rate measurement. Same named-value plumbing as Battery.

Each entry chooses read / notify / indicate properties and an updateIntervalMs at which the underlying named-value is sampled. See BLE Manager Settings → Standard Services for the schema and example.

Central mode (scan)

When central: 1, BLEManager configures NimBLE for scanning according to:

Key Meaning
scanIntervalMs / scanWindowMs NimBLE scan duty cycle.
scanPassive Passive (no scan-response request) vs active.
scanNoDup Deduplicate identical advertisements within a window.
scanLimited Filter to limited-discoverable advertisers only.
scanForSecs 0 to scan forever (typical), or a time-limited scan.
scanBTHome Recognise BTHome packets.

Each advertisement is decoded by BLEAdvertDecoder. Recognised packets (BTHome v1/v2 today) are translated into a synthetic device record and pushed into the Bus named by busConnName via RaftBusDevicesIF, so they become indistinguishable from I²C-discovered devices to the rest of the application — they appear in Device Manager listings, are publishable on devjson / devbin, and are recordable by DataLogger.

The device type of these devices is the device type record named by deviceTypeKey in that bus's configuration (in the Device Manager's bus list); it defaults to BLEBTHome, the record that decodes the BTHome records the decoder produces. If no record has that key, a warning is logged and no BLE devices are reported.

Outbound connections (acting as a client to another peripheral) are not currently supported in Raft. Devices that broadcast their state (BTHome thermometers, hygrometers, motion sensors, etc.) are the intended use case.

Comms channel

BLEManager::addCommsChannels() registers a single channel called BLE with the comms core. The channel uses RICSerial framing by default and is the route by which:

  • Inbound writes (commands, RICREST messages, file blocks) are decoded and dispatched to ProtocolExchange.
  • Outbound responses and publish messages are framed and queued for transmission via notify / indicate.

The channel's block sizes are maxPktLen (default 500), so RICREST file uploads over BLE use blocks of up to 333 bytes (2/3 of it).

Outbound queueing is handled by BLEGattOutbound, which keeps two queues:

  • A command queue of outQSize messages (default 30) for responses and other non-publish messages, sent first.
  • A publish queue of up to pubQSize messages (default 30) and pubQMaxBytes bytes (default 32 KB, about a second of link time with notifications). The channel refuses a publish when either is reached, so publishes are dropped rather than delaying responses, and large messages (e.g. camera frames) don't pile up for many seconds. One message may take the queue over pubQMaxBytes, so a message larger than it is still sent when the queue is empty.
  • A message is moved into its queue (a RICREST message just encoded for sending is not copied at all), and each packet is copied out of the queue as it is sent, so a large message costs little to queue and send.
  • A message partly sent from one queue is finished before anything is sent from the other, so HDLC frames aren't interleaved.
  • An indication not acknowledged within outMsgsInFlightMs (500 ms) is treated as done and the next message is sent.

Named value access

Other SysMods (and the standard BLE services) read live state from BLEManager via the named-value mechanism. The supported names are:

Name Meaning
R / r Latest cached RSSI in dBm (refreshed every 2 s while connected).
C / c 1 if there is a session (a connected central that has sent a command), else 0.

These are convenient for cross-SysMod logic — e.g. a power-management SysMod that switches a sensor to low-rate mode while no BLE central is connected.

REST endpoints

Endpoints are added in BLEManager::addRestAPIEndpoints():

Endpoint Method Purpose
blerestart GET Tear down and re-initialise the BLE stack. Useful after a configuration change without rebooting.
bledisconnect GET Disconnect every connected central, about 200 ms after replying so the reply is sent first. Clients should send it when they finish - see Releasing the link. It can be sent over any channel (e.g. the serial console) to release a link held by another host.
bleconfig GET Get or set whether responses and publishes use indications: bleconfig/cmdUseInd=0/pubUseInd=1 (either or both). Replies {"rslt":"ok","cmdUseInd":1,"pubUseInd":0}.

Request/response details follow the standard REST API conventions.

Status and diagnostics

getStatusJSON() reports advertising name, connection state, RSSI, MTU, the negotiated connection parameters and the queue occupancy; while connected, sess (1 if there is a session) and nConn (centrals connected). getDebugJSON() adds counters from BLEManStats — frames in/out, indication ACK timeouts, queue rejections, etc. Visible at runtime via sysmoddebug/BLEMan if LogManager is enabled.

The BLEMan entry in SysManager's periodic report (reportList) includes rxBPS and txBPS, and when they aren't zero pubDrpOp (times the publish queue filled after the start of the connection - see Why publishes can be dropped) and txNoBuf (packets sent again later as NimBLE had no buffer for them - see Notifications and NimBLE's buffers).

Throughput and tuning

The achievable throughput from a Raft device over BLE is dominated by three knobs: connection interval, indication-vs-notify mode, and link-layer packet length / MTU. The trade-offs are non-obvious — the source notes in RaftSysMods/devdocs/BLE-publish-throughput-analysis.md measure the issue end-to-end with a 104 Hz sample stream; the summary below distils that analysis.

Why publishes can be dropped silently

There are two layers that can refuse a publish:

  1. CommsChannelManager::handleOutboundMessageOnChannel skips a publish if any non-publish message is already queued on the channel. Logged only when DEBUG_OUTBOUND_PUBLISH is defined.
  2. BLEGattOutbound::isReadyToSend rejects a publish when the publish queue is full (pubQSize messages or pubQMaxBytes bytes). Responses have their own queue (outQSize), so publishes don't take their slots. A publisher asks again on each loop while the queue is full, so a drop is counted once each time the queue fills: one more than a short grace period after connecting is counted in the stats as operational (pubDrpOp), and logged (at most once a second, with the number of publishes refused since the last warning) when WARN_ON_PUBLISH_QUEUE_FULL is defined.

Both rejections are intentional — they protect command responses from being delayed behind a backlog of publishes — but they are silent unless those debug flags are enabled.

Indication-mode throughput ceiling

With indications (sendUseInd=1 for responses, the default; pubUseInd=1 for publishes), the BLE spec limits indications to one in flight at a time (a peripheral must wait for the ACK before the next indication can be queued). outMsgsInFlightMax is read from the config but not used — the in-flight count is hard-gated to 1.

Each indication ACK takes 1–2 connection intervals, so:

Connection interval Indication round-trip Max indication rate
15 ms (default) 15–30 ms ~33–66 msg/s
7.5 ms (BLE minimum) 7.5–15 ms ~66–133 msg/s
50 ms (low-power) 50–100 ms ~10–20 msg/s

For a typical devbin payload of ~330 B, that translates to roughly 10–66 KB/s in indication mode at the default interval. Light-sleep cannot run while a 7.5 ms interval is in use, so the minimum interval is unsuitable for battery-powered devices.

Effective changes you can make

Change Throughput effect Trade-off
Lower connIntvPrefMs (e.g. 8) Roughly doubles indication rate More power; central may override; battery life ↓
Publishes by notify (pubUseInd: 0, the default) 3–5× higher publish rate than indications (multiple notifications per connection event) No acknowledgement from the central; notifications are spaced by minMsBetweenSends (5 ms by default)
Request 2 M PHY (where supported on both ends) ~1.5× from faster ACK round-trip Both sides must support BLE 5
Increase pubQSize / I²C bus ring size Buys headroom across stalls Doesn't lift the ceiling — only delays overflow
Compress / decode payload firmware-side Linearly reduces required throughput Bigger code change; affects the firmware/JS decoder boundary

If you are streaming high-rate data (sample buffers, multi-axis IMUs at hundreds of Hz), the practical recipe is: notify mode + 7.5 ms connection interval + DLE on (llPacketLengthPref: 251, default) + 2 M PHY where available. Reserve indication mode for command/response and lower-rate status topics.

Notifications and NimBLE's buffers

Notifications are sent without waiting for the central, so they can use up NimBLE's pool of packet buffers when they are sent faster than the link takes them (a long message, e.g. a camera frame, is many packets). A packet that gets no buffer is sent again later, and counted in the stats (txNoBuf) - it's normal for this to count steadily while a large message is being sent.

In RaftSysMods 1.27.0 and earlier such a packet was lost: NimBLE takes a missing buffer to mean "send the characteristic's value", which is empty, and reports success. The message then failed its HDLC CRC at the central and was discarded. With notifications this lost most messages of 4 KB or more (e.g. every camera frame), which is why responses were kept on indications; with indications only one packet is in flight, so the pool is never used up.

Large messages

Measured with live camera frames (about 7-30 KB each) to the raftjs dashboard on Windows, with a 7.5 ms connection interval and notifications: about 20-30 KB/s, with the main loop held for at most about 7 ms (10 KB frames). A publish is refused while the queue holds pubQMaxBytes, so a frame is at most about a second old when it is sent. Keep frames small (QVGA or smaller) for a usable frame rate.

Default configuration baseline

Parameter Default Source
sendUseInd true (responses) BLEConfig.h
pubUseInd false (publishes) BLEConfig.h
connIntvPrefMs 15 ms BLEConfig.h
mtuSize 512 BLEConfig.h
maxPktLen 500 BLEConfig.h
outQSize 30 (responses) BLEConfig.h
pubQSize 30 (publishes) BLEConfig.h
pubQMaxBytes 32768 (publishes) BLEConfig.h
outMsgsInFlightMs 500 BLEConfig.h
minMsBetweenSends 5 (notify only) BLEConfig.h
llPacketLengthPref 251 (DLE) BLEConfig.h
llPacketTimePref 2500 µs BLEConfig.h

For deeper analysis with measured numbers see RaftSysMods/devdocs/BLE-publish-throughput-analysis.md.

See also

Home

Getting Started

Scaffolding & Building

Architecture

Built-in SysMods

Comms & Protocols

Devices & Buses

ROS 2

Helpers

Reference

Clone this wiki locally