OTAUpdateFlow

OTA Update Flow

End-to-end view of how a Raft device receives a new firmware image, writes it to the inactive OTA partition, verifies the result and reboots into the new app. Covers both delivery paths — HTTP-direct (espFwUpdate) and the framework's file-stream pipeline — and the invariants they share.

The OTA machinery is split across several SysMods so that the same low-level write/verify/restart path can be driven from very different transports (BLE, WebSocket, raw HTTP). The code that actually programs flash lives in ESPOTAUpdate; the routing, framing and back-pressure live in ProtocolExchange and the Comms Channels layer.

At a glance

                  ┌─────────────────────────────────┐
HTTP POST  ─────► │ RaftWebServer                   │
(/espFwUpdate)    │   ↓ chunked body                │ ────────┐
                  └─────────────────────────────────┘         │
                                                              ▼
BLE / WS / serial ─► RICREST  ──► ProtocolExchange    ──► ESPOTAUpdate
(ufStart + FileBlock)             (FileStreamSession)      (file system worker)
                                                              │
                                                              ▼
                                                       esp_ota_begin
                                                       esp_ota_write × N
                                                       esp_ota_end
                                                       esp_ota_set_boot_partition
                                                              │
                                                              ▼
                                                       esp_restart  (after 1 s)

Both paths converge on the same SysMod entry points — fileStreamStart(), fileStreamDataBlock(), fileStreamCancelEnd() — so once a transfer is "open", the rest of the pipeline is identical regardless of how the bytes arrived.

Path A — HTTP-direct (espFwUpdate)

The simplest deployment path: a workstation running the raft CLI, an IDE plugin, or curl POSTs the binary as the request body to /espFwUpdate over WiFi.

POST /espFwUpdate HTTP/1.1
Host: my-device.local
Content-Type: application/octet-stream
Content-Length: 1572864

<raw firmware bytes>

Inside the device:

1. RaftWebServer receives the request and streams the body in chunks.
2. The endpoint espFwUpdate is registered by ESPOTAUpdate::addRestAPIEndpoints() with three callbacks:
   • apiFirmwareMain — final response, sent once the update has been written and esp_ota_end() has returned (see step 6); populated from _otaStatus.
   • apiFirmwarePart — invoked for every chunk; immediately calls fileStreamDataBlock().
   • apiReadyToReceiveData — back-pressure hook; returns false while 16 KB (HTTP_PENDING_MAX_BYTES) or more of received blocks are waiting to be written, so the web server stops reading the body and the client is slowed by TCP flow control.
3. The first chunk's firstBlock flag triggers fileStreamStart(), which checks that OTADirect is enabled.
4. Each FileStreamBlock is deep-copied into an OTAUpdateFileBlock (block bytes go into a SpiramAwareAllocator vector) and queued for the shared file system worker. If there isn't the memory for the copy (plus a 16 KB margin) the block fails with RAFT_INSUFFICIENT_RESOURCE. Neither the main loop nor the web server waits for flash writes.
5. A job on the worker takes the blocks in order: it calls esp_ota_begin() on the first one, esp_ota_write() on every block, accumulates a CRC-16/CCITT and per-update statistics under a status mutex, then calls esp_ota_end() + esp_ota_set_boot_partition() on the final block and sets _restartPending.
6. When the whole body has arrived, apiFirmwareMain is called while the last blocks may still be being written. Its reply is deferred until the worker has finished (up to 15 s, OTA_COMPLETION_WAIT_MAX_MS), so it reports the real result rather than "in progress". If the reply can't be deferred, the handler waits for the worker (also up to 15 s).
7. loop() sends the deferred reply, then notices _restartPending is set and, after TIME_TO_WAIT_BEFORE_RESTART_MS (1000 ms) so the HTTP response can flush, calls esp_restart().

The reply is {"req":...,"rslt":"ok"} on success, or "rslt":"fail" with "error" set to the result (Failed if starting or writing failed, FailedOTAEnd, FailedSetBootPartition or FailedCancelled).

Updates can be disabled per device by setting OTADirect: 0 in the ESPOTAUpdate SysType config. fileStreamStart() then rejects the transfer — on both paths, since Path B starts an update through the same call.

Path B — File-stream over RICREST (BLE / WebSocket / serial)

Used by the raftjs browser library and any RICREST client. The wire protocol is the File Download Protocol's upload counterpart — a ufStart command opens the session, a sequence of binary FileBlock RICREST messages carry the payload, and ufEnd closes it.

The routing inside the device:

1. Comms layer decodes the framed CommsChannelMsg (RICSerial / RICFrame / RICJSON) and hands it to ProtocolExchange::processEndpointMsg.

2. The RICREST command frame ufStart with firmware file type/destination is dispatched to a built-in handler that opens a FileStreamSession of content type firmware on the originating channel.

3. ProtocolExchange looks up its registered FW-update handler via the hook installed at boot by ESPOTAUpdate::setup():

   pSysManager->getProtocolExchange()->setFWUpdateHandler(this);

   That hook tells FileStreamSession to forward every block to ESPOTAUpdate::fileStreamDataBlock() — exactly the same entry point used by Path A.

4. Each RICREST FileBlock element carries (streamID, filePos, payload) where streamID is the non-zero session ID returned by ufStart and filePos is the lower 24 bits of the FILEBLOCK position word. The session accepts blocks in order, applies flow control (batch acks, retries), and pushes blocks at the FW handler.

5. From this point onward the file system worker / esp_ota_* / restart sequence is identical to Path A.

6. Each okto ack is held back until the blocks received so far have been written (the session asks ESPOTAUpdate::fileStreamWriteStatus(), which is RAFT_BUSY while blocks are waiting), so the client is never more than a batch ahead of what is in flash. A failure to write cancels the upload (ufCancel, reason failOTAWrite; failOTAStart if the update can't be started, e.g. OTADirect is off). See uploads.

The notable differences from Path A:

Concern	Path A (HTTP)	Path B (RICREST)
Transport	TCP, single connection	BLE / WebSocket / serial via channels
Framing	Raw octets in HTTP body	RICREST FileBlock elements
Flow control	apiReadyToReceiveData pauses the body while 16 KB is waiting to be written	okto batch acks, held until the blocks are written
Multi-session	One at a time	Up to MAX_SIMULTANEOUS_FILE_STREAM_SESSIONS (3) per ProtocolExchange
Cancel	Connection close	ufCancel RICREST message

For full wire-level details of the file-stream side see File Download Protocol — Uploads.

Shared invariants

Regardless of the path, the OTA write side enforces these guarantees:

• One update at a time. Blocks are written in the order received, one at a time. The first block of a new update starts it afresh, aborting (esp_ota_abort) any update still in progress.
• Off the main loop. The flash writes happen on the shared file system worker (ESPOTAUpdate has no task of its own). Callers (HTTP, BLE, etc.) never block on esp_ota_write, which can take tens of milliseconds. The flash erase and writes still stall all code running from flash unless the app runs from PSRAM — see below.
• Status mutex. _fwUpdateStatusSemaphore protects the FWUpdateStatus struct (bytes written, throughput, CRC, last result string) so that getDebugJSON() and apiFirmwareMain can read consistent snapshots.
• CRC-16/CCITT. Computed incrementally as bytes are written; available in getDebugJSON() and as _otaStatus.totalCRC.
• Deferred restart. The reboot is scheduled by setting _restartPending and is performed from loop() 1 s later, so any pending response (HTTP body, RICREST CmdRespJSON, etc.) has time to leave the device.
• isBusy() reports true while an update is in progress, which lets SysManager and StatePublisher reduce publish rates and keep busy indicators visible.

What the application has to do

ESPOTAUpdate::setup() registers itself with ProtocolExchange, so adding ESPOTAUpdate to your SysType config is enough. The equivalent by hand:

auto* pProtoExch = (ProtocolExchange*)getSysManager()->getSysMod("ProtoExchange");
auto* pOTA      = (ESPOTAUpdate*)getSysManager()->getSysMod("ESPOTAUpdate");
if (pProtoExch && pOTA)
    pProtoExch->setFWUpdateHandler(pOTA);

Without this hook the file-stream path (Path B) has nowhere to deliver firmware blocks; the HTTP-direct path (Path A) still works because it goes straight into ESPOTAUpdate via its REST endpoint.

Partition layout

OTA requires two app partitions plus an otadata partition. A typical Raft partitions.csv:

# Name,   Type, SubType, Offset,   Size
nvs,      data, nvs,     0x009000, 0x015000
otadata,  data, ota,     0x01e000, 0x002000
app0,     app,  ota_0,   0x020000, 0x1b0000
app1,     app,  ota_1,   0x1d0000, 0x1b0000
fs,       data, 0x83,    0x380000, 0x080000

esp_ota_begin() chooses the inactive app0/app1 slot; esp_ota_set_boot_partition() flips otadata so the bootloader picks the new image on reboot. See Partitions and Flash Layout for the broader picture.

Failure handling

Failure	What happens
esp_ota_begin fails	The update is marked failed (lastOTAUpdateResult = "Failed"); later blocks of the update are refused (fileStreamWriteStatus() reports the failure), so a Path B upload is cancelled with failOTAWrite and a Path A reply reports the failure.
esp_ota_write returns non-OK	Same as above, and the update is aborted (esp_ota_abort); the partial image stays on the inactive slot but is never marked bootable.
esp_ota_end or esp_ota_set_boot_partition fails	Result FailedOTAEnd or FailedSetBootPartition; no restart.
Cancel (Path B ufCancel, or session timeout)	fileStreamCancelEnd() flags a cancel and queues a cancel request; the worker aborts the update (completeOTAUpdate(true), result FailedCancelled) and drops blocks still waiting.
Not enough memory to copy a block	The block fails with RAFT_INSUFFICIENT_RESOURCE and the upload fails.
Blocks arriving faster than they are written	Path A: the web server stops reading the body. Path B: acks are held, so the sender waits. See also BLE publish-throughput notes.
New image fails to boot	The bootloader keeps the old image marked as the rollback target — not implemented in ESPOTAUpdate today; the new image is unconditionally marked bootable on a clean esp_ota_end. Applications that want rollback must call esp_ota_mark_app_valid_cancel_rollback() themselves after boot, with CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y in sdkconfig.

Flash stalls during an update

esp_ota_begin() erases the inactive slot before the first write — only as much as the image needs when its length is known (an HTTP Content-Length, or fileLen in ufStart), otherwise the whole partition. On ESP32 chips the flash cache is off during an erase or write, so all code running from flash stops; the erase can take several seconds, during which the task watchdog may print a backtrace (it doesn't affect the update). stMs in the diagnostics below is the time taken. Running the app from PSRAM avoids the stall — see Flash writes stop the whole chip.

Diagnostics

ESPOTAUpdate::getDebugJSON() returns:

{
  "Bps": 41234.5,
  "stMs": 12,
  "bytes": 1572864,
  "wrPS": 65000.0,
  "elapS": 38.2,
  "blk": 4096
}

Field	Meaning
Bps	Average bytes/sec across the whole transfer
stMs	esp_ota_begin duration
bytes	Total bytes written so far
wrPS	Bytes/sec measured against esp_ota_write time only (excludes transport stalls)
elapS	Wall-clock seconds since the transfer started
blk	Last block size accepted

These fields are the most useful primary signal for diagnosing slow OTA: a low wrPS indicates flash contention, a low Bps with a high wrPS indicates a transport bottleneck (typical on BLE — see throughput notes).

See also

• ESPOTAUpdate SysMod — the SysMod itself.
• ProtocolExchange — file/stream session ownership and the FW-update hook.
• File Download Protocol (OKTO) — wire-level batched transfer, including uploads.
• File System — the file system worker that writes the firmware.
• Comms Channels — how RICREST messages reach ProtocolExchange.
• Partitions and Flash Layout — partition table requirements.
• BLE Manager — throughput and tuning — limits when delivering Path B over BLE.