RICRESTProtocol

RICREST Protocol

RICREST is Raft's transport-agnostic message envelope that carries REST-style requests, responses and binary stream blocks over non-HTTP links (BLE, serial, sockets). On the wire it is normally wrapped by one of the framing protocols RICSerial, RICFrame or RICJSON; the RICREST payload itself is the same in all three.

For higher-level flow see Communications Stack Overview and ProtocolExchange.

Frame layers

   ┌────────────────────────────────────────────────────────────────┐
   │  Outer framing : RICSerial (HDLC) | RICFrame (len-prefixed) |  │
   │                  RICJSON (raw JSON)                            │
   └────────────────────────────────────────────────────────────────┘
                                  │
                                  ▼
   ┌────────────────────────────────────────────────────────────────┐
   │  CommsChannelMsg header                                        │
   │   - msg number (1 byte for RICFrame; managed by RICSerial HDLC)│
   │   - protocol code  (RICREST, BridgeRICREST, RawCmdFrame, ...)  │
   │   - type code      (Command, Response, Publish, Report)        │
   └────────────────────────────────────────────────────────────────┘
                                  │
                                  ▼
   ┌────────────────────────────────────────────────────────────────┐
   │  RICREST element  (only when protocol == RICREST)              │
   │   - 1 byte element code                                        │
   │   - element-specific payload                                   │
   └────────────────────────────────────────────────────────────────┘

Outer framings

RICSerial framing

ProtocolRICSerial uses MiniHDLC to frame and deframe a byte stream:

E7 | esc(header + payload) | esc(crcHi) | esc(crcLo) | E7

• 0xE7 is the frame boundary byte that starts and ends each frame.
• 0xD7 is the escape byte. A 0xE7 or 0xD7 in the frame (including in the CRC) is sent as 0xD7 followed by the byte XOR 0x20; no other bytes are escaped.
• The CRC is CRC-16/CCITT-FALSE (polynomial 0x1021, initial value 0xFFFF, crc16("123456789") = 0x29B1) over the unescaped header and payload, sent high byte first.
• The bytes inside the frame are the CommsChannelMsg header followed by its payload (see below).
• The device drops a frame longer than MaxRxMsgLen (5000 bytes, or 200000 with PSRAM) or with a bad CRC. The boundary and escape bytes and the limits can be changed in the SysType (see ProtocolExchange → Configuration).

This is the framing used over UART, BLE GATT characteristics and (by default) WebSockets. A frame can span several BLE writes or notifications.

• Serial: the SerialConsole shares the port with log text and typed commands, so the HDLC bytes are re-encoded by ProtocolOverAscii so that every byte sent has its top bit set. Received bytes >= 0x80 go to the OverAscii decoder and then to HDLC; bytes < 0x80 are text.

  Byte	Sent as
  00-0F	85, (b ^ 0x20) | 0x80
  10-7F	b | 0x80
  80-8F	8E, b ^ 0x20
  90-FF	8F, b

• WebSocket: each HDLC frame is sent as a binary WebSocket message at ws://<host>/<pfix> (pfix default ws), with no OverAscii encoding.

• BLE: frames are written to the command characteristic and responses arrive on the response characteristic (see BLEManager).

Test vectors (a URL request with message number 1, the URL followed by a NUL as raftjs sends it; the device doesn't need the NUL):

Request	Frame
v	e7 01 02 00 76 00 f4 ca e7
filelist	e7 01 02 00 66 69 6c 65 6c 69 73 74 00 d1 dc e7
v, message number 0xE7 (escaped)	e7 d7 c7 02 00 76 00 02 43 e7

RICFrame framing

ProtocolRICFrame is used where the transport already provides framing (TCP socket, WebSocket binary frame). It does not add HDLC byte-stuffing or a CRC. Each frame is just:

byte 0   : msgNumber (0..255)
byte 1   : (msgTypeCode << 6) | (msgProtocolCode & 0x3f)
bytes 2+ : payload (= RICREST element when protocol == RICREST)

msgTypeCode is 0=COMMAND, 1=RESPONSE, 2=PUBLISH, 3=REPORT. msgProtocolCode is the CommsMsgProtocol enum (2=RICREST, 3=BridgeRICREST, 0x3e=RawCmdFrame, 0x3f=None, 0=ROSSerial).

RICJSON framing

ProtocolRICJSON carries pure JSON with no envelope. The JSON document itself encodes whatever metadata it needs and is delivered to the application as a CommsChannelMsg with the JSON in the payload.

CommsChannelMsg header (RICSerial)

When carried inside a RICSerial HDLC frame, the first two bytes are the same as RICFrame's header:

byte 0   : msgNumber
byte 1   : (msgTypeCode << 6) | (msgProtocolCode & 0x3f)
bytes 2+ : payload

So at the API level RICSerial and RICFrame produce identical CommsChannelMsg objects — they differ only in how the bytes are framed for the underlying transport. A RICREST command is therefore [msgNum, 0x02, elemCode, ...] and its response [msgNum, 0x42, 0x01, json..., 0x00].

RICREST element format

When msgProtocolCode == MSG_PROTOCOL_RICREST (2), the payload begins with a one-byte element code:

Code	Name	Inbound role	Outbound role
0	URL	Request as URL string (endpoint/path?query=...).	—
1	CMDRESPJSON	Decoded but not acted on (no response).	Response wrapping the endpoint's JSON output.
2	BODY	POST-style body chunk. Decoded, but POST over RICREST isn't implemented (no response).	—
3	COMMAND_FRAME	JSON { "cmdName": "...", … } flattened into a query string.	—
4	FILEBLOCK	A binary chunk of a file/stream session.	A binary chunk of a file/stream session.

URL (code 0)

byte 0     : 0x00
bytes 1..n : ASCII URL, e.g.  "subscription?action=update&topic=devjson&rateHz=10"

The whole tail of the message is the URL as a UTF-8 string (max length capped by RICRESTMsg::getMaxRestBodySize() — 5000 without PSRAM, 200000 with).

CMDRESPJSON (code 1)

byte 0     : 0x01
bytes 1..n : ASCII JSON object

Outbound: the JSON is the endpoint's response document followed by a NUL byte, e.g.

{"req":"subscription","rslt":"ok"}

The outbound message is marked as a response (msg type code 1) and reuses the original message's msgNumber so the requester can match request to response.

Inbound: a CMDRESPJSON element is decoded (its reqStr field is read as the request name) but ProtocolExchange doesn't act on it and sends no response. Send requests as URL or COMMAND_FRAME elements.

BODY (code 2)

byte 0       : 0x02
bytes 1..4   : bufferPos      (uint32, big-endian)   ← offset of this chunk in the overall body
bytes 5..8   : totalBytes     (uint32, big-endian)   ← total body length
bytes 9..n   : binary payload

Intended for a multi-chunk request body (POST over RICREST). It is decoded, but ProtocolExchange doesn't implement it and sends no response, so POST endpoints (e.g. fileupload, postsettings) can't be used over RICREST; uploads use the file transfer commands instead.

COMMAND_FRAME (code 3)

byte 0     : 0x03
bytes 1..n : ASCII JSON object with at least a "cmdName" field

The JSON is converted into a request string of the form cmdName?<flattened-fields> and dispatched via the same path as a URL element. This is convenient for senders that prefer to encode all parameters as JSON instead of a URL. The file transfer commands (ufStart, dfStart etc. - see File transfers) are command frames handled by the file/stream sessions rather than REST endpoints.

FILEBLOCK (code 4)

byte 0       : 0x04
byte 1       : streamID         (uint8, non-zero session ID)
bytes 2..4   : filePos / offset (24-bit unsigned integer, big-endian)
bytes 5..n   : binary payload

Used by the file/stream session machinery (see File Download Protocol, Real-Time Streams, and the upload/firmware-update flows owned by ProtocolExchange).

The streamID byte identifies the active file/stream session allocated by ufStart or dfStart. It is an 8-bit field, so non-zero session IDs are representable in the range 1..255. streamID value 0 is reserved internally as FILE_STREAM_ID_ANY ("unspecified / match by other criteria") and must not be used for normal FILEBLOCK data belonging to an allocated session.

The stream ID and position are stored as one big-endian 32-bit word in code:

(streamID << 24) | (filePos & 0x00ffffff)

This limits the per-block file/stream position field to 24 bits. Protocol implementations that need larger transfers must handle that at the session/protocol layer rather than placing position bits in the top byte.

Note — the precise byte offsets above match the constants in RICRESTMsg.h (RICREST_BODY_*, RICREST_FILEBLOCK_*). For exact, copy-pasteable values see that header. If you are implementing a non-Raft client, refer directly to the decode() and encode() functions in RICRESTMsg.cpp rather than transcribing the table.

When to use which framing

Transport	Recommended framing
BLE GATT characteristic	RICSerial (HDLC)
UART / RS-232	RICSerial (HDLC)
TCP socket	RICFrame
WebSocket (binary)	RICSerial (the WebServer default pcol, used by raftjs and RaftCLI) or RICFrame
HTTP REST	none — use the URL directly via RestAPIEndpointManager

A RICJSON channel passes each received JSON object ({"cmdName":...}) to the endpoint as a raw command frame, but the response isn't sent back, so it only suits commands whose reply isn't needed.

Message numbering and matching

For request/response correlation:

• The sender numbers its requests 1..255, wrapping back to 1. Message number 0 means unnumbered; the device uses it for messages it sends on its own (e.g. upload timer acks, cancel notices, download blocks).
• The framework's response path calls endpointMsg.setAsResponse(cmdMsg) which copies the channelID and msgNumber from the original request.
• Receivers match responses to requests on (channelID, msgNumber, msgTypeCode == RESPONSE).
• A response may come later than responses to requests sent after it: an endpoint can defer its response to a numbered request. If a request is resent with the same message number while its response is still deferred, the device ignores the resend, so a client can safely retry with the same number.

Asynchronous messages (publishes, reports) use MSG_TYPE_PUBLISH / MSG_TYPE_REPORT and a different msgNumber stream.

File transfers

Files are uploaded and downloaded over RICREST with COMMAND_FRAME commands (cmdName below) and FILEBLOCK elements, handled by the file/stream sessions in ProtocolExchange. The replies are CMDRESPJSON responses.

Upload (ufStart / blocks / ufEnd)

1. ufStart, e.g. {"cmdName":"ufStart","reqStr":"fileupload","fileType":"fs","fileName":"/local/a.txt","fileLen":2048,"batchMsgSize":800,"batchAckSize":10}.
   • fileType: fs or file (or empty) for a file, fw or ricfw for a firmware update (OTA Update Flow), rtstream for a real-time stream.
   • batchMsgSize (block size, default 5000) and batchAckSize (blocks per acknowledgement, default 40) are requests. The device clamps the block size to between 20 bytes and 2/3 of the channel's inboundBlockLen (800 bytes on serial and on WebSocket without PSRAM, 16000 on WebSocket with PSRAM, 333 on BLE), limits block × batch to 50000 bytes, and reduces both if the memory isn't there.
   • The reply gives the values used and the session's stream ID: {"req":"ufStart","batchMsgSize":800,"batchAckSize":10,"streamID":1,"rslt":"ok"}. Always use the values in the reply.
   • An optional CRC16 (e.g. "CRC16":"0xa1f2", the CRC-16/CCITT of the whole file as filehash gives it) has the file checked once written: if it doesn't match, the file is deleted and the upload cancelled (failCRC). See Checking the file. Firmware with RaftCore 1.63.0 or earlier ignores it.
2. Blocks: FILEBLOCK elements carrying the stream ID and the position of the data, e.g. [msgNum, 0x02, 0x04, streamID, pos>>16, pos>>8, pos, data...].
3. Acks: the device replies {"req":"ufBlock","okto":N,"rslt":"ok"} (as the response to a block, or with message number 0 when it is sent later) - N is the number of bytes accepted so far - after the first block, after every batchAckSize blocks, and after the last block. File blocks are written on the file system worker (firmware by the firmware updater), and an ack is only sent once the blocks it covers are written. A block at an unexpected position is dropped (the first one is acked at once so the host can resend from okto). If no blocks arrive the device sends timer acks: after 5 s while fewer than two blocks have arrived, then after 1 s; after 4 timer acks with no block it cancels the upload (failRetries). The host sends the first block, then batches of batchAckSize, waits for okto and resumes from it (go-back-N).
4. ufEnd gets {"req":"ufEnd","rslt":"ok"}, with "crc16":"xxxx" added when the file was checked against the CRC16 given (so the host needn't check it with filehash). The session normally ends by itself after the last block, and a ufEnd that finds no session still gets ok (and the crc16, which is kept after the session ends).
5. ufCancel (with an optional reason) cancels an upload from the host. The device sends a response with "cmdName":"ufCancel","reason":... (message number 0) when it cancels one itself (e.g. failFileWrite, failOTAWrite, failCRC when the file doesn't match its CRC16, or failRetries after the timer acks).

Download (dfStart / blocks / dfEnd)

dfStart opens a download session; the device then sends the file as FILEBLOCK elements (as commands with message number 0), at most one block every 100 ms, and the host acknowledges with dfAck (okto, no reply). dfEnd ends it, and dfCancel cancels it. A dfStart for a file that doesn't exist is refused with "reason":"sessionNotStarted". See File Download Protocol (OKTO) for the details.

Related

• Communications Stack Overview
• Comms Channels
• ProtocolExchange
• Real-Time Streams
• Adding REST API Endpoints
• File Download Protocol (OKTO)