RaftFileManagerSysMod

FileManager SysMod

Built-in SysMod that mounts a local flash file system (LittleFS or SPIFFS), optionally an SD card, and exposes file-system operations as REST endpoints.

Source: RaftSysMods/components/FileManager/

What it provides

• Mount of local (LittleFS or SPIFFS) and optionally sd.
• File listing, read (whole file or a binary-safe section), CRC16 hash, delete, upload (HTTP and chunked-RICREST) and full reformat.
• File system work for these requests done on the file system worker, so it doesn't hold up the main loop.
• Routing of incoming FileBlock chunks to ProtocolExchange when uploads come over BLE/serial.

SysType configuration

{
    "name": "FileManager",
    "LocalFsDefault": "littlefs",
    "LocalFsFormatIfCorrupt": 1,
    "SDEnabled": 0,
    "DefaultSD": 0,
    "CacheFileSysInfo": 0,
    "SDScanUsedAtBoot": 0,
    "WritePacingPct": 30,
    "SDMOSI": "",
    "SDMISO": "",
    "SDCLK": "",
    "SDCS": ""
}

Key	Default	Meaning
LocalFsDefault	"littlefs"	Local FS driver: "littlefs" or "spiffs"
LocalFsFormatIfCorrupt	true	Auto-format if mount fails
SDEnabled	false	Enable SPI SD card mount
DefaultSD	false	When set, default file ops resolve to the SD card if present
CacheFileSysInfo	false	Cache directory listings between calls
SDScanUsedAtBoot	false	Start the SD card used-space scan (which can take several seconds) at boot rather than on first use of the card; either way it runs in a background task
SDMOSI / SDMISO / SDCLK / SDCS	""	SD SPI pins (named pin or numeric string)
WritePacingPct	30, or 0 when flash writes don't stall other code	Maximum percentage of time spent in slow file system writes during an upload (0 = no pacing). See Write pacing. (RaftCore versions after 1.58.1; the default of 0 from 1.63.0)

Write pacing

Writing to flash periodically needs a 4 KB sector to be erased, which takes about 60-70 ms. On ESP32 chips the flash cache is disabled on both cores during an erase, so all code running from flash stalls whichever task is writing. A continuous upload can therefore stall the whole system most of the time (measured at about 76% on a UM FeatherS3).

With write pacing, after a slow write, uploads are held off long enough that slow writes take at most WritePacingPct of the time:

• HTTP uploads (fileupload): the web server stops reading the request, so the client is slowed by TCP flow control.
• RICSerial uploads (e.g. raftjs over WebSocket, BLE or serial): received messages stay queued, so the transfer's own flow control slows the sender.

Uploads still only complete once the data is written. The cost is upload speed: at the default 30% a FeatherS3 uploaded at about 17 kB/s (HTTP) instead of 44 kB/s.

HTTP uploads are also held back while 16 KB (HTTP_UPLOAD_PENDING_MAX_BYTES) of received data is still waiting to be written, or while there isn't the memory to hold another 4 KB chunk. RICREST uploads are held back by their acknowledgements, which wait until the blocks are written (see uploads).

Running the app from PSRAM (CONFIG_SPIRAM_XIP_FROM_PSRAM) or flash chips which support auto-suspend (CONFIG_SPI_FLASH_AUTO_SUSPEND, ESP32-S3 and later with specific flash chips only) avoid the stall altogether, and from RaftCore 1.63.0 the default is then 0 (no pacing). See Flash writes stop the whole chip.

REST endpoints exposed

In all paths the file system name (local or sd) is the first path segment after the verb; folders use ~ instead of /.

Endpoint	Method	Purpose
reformatfs/<fs>	GET	Reformat a file system — destructive (reformatfs/<fs>/force to force)
filelist/<fs>/<folder>	GET	List directory contents as JSON
fileread/<fs>/<path>	GET	Read file contents (the reply is the raw file text)
filesection/<fs>/<path>/<start>/<len>	GET	Read a section of a file, base64 encoded in JSON (binary-safe)
filehash/<fs>/<path>	GET	CRC16 of a file's contents, e.g. to check an upload
filedelete/<fs>/<path>	GET	Delete a file
fileupload	POST	Upload a file over HTTP (multipart). Over BLE, WebSocket or serial use ufStart — see uploads

A request for a file that doesn't exist (fileread, filesection, filehash) gets {"req":...,"rslt":"fail","error":"nofile"}.

filesection

filesection/<fs>/<path>/<start>/<len> reads len bytes from byte start. If len is omitted or 0, or goes past the end of the file, the rest of the file is read; a start past the end gives an empty section.

{"req":"filesection/local/logs~a.csv/0/1024","rslt":"ok","fs":"local","fname":"logs/a.csv","start":0,"len":1024,"total":45678,"data":"<base64>"}

Field	Meaning
start	Position of the section in the file
len	Bytes in this section (before base64 encoding)
total	Length of the whole file
data	The section, base64 encoded (no line breaks)

There's no limit on len: the reply is about 4/3 × len plus about 100 bytes, so choose len to suit the link (e.g. small sections over BLE).

filehash

filehash/<fs>/<path> reads the whole file and returns its CRC-16/CCITT (the same CRC as the download protocol), as four hex digits:

{"req":"filehash/local/logs~a.csv","rslt":"ok","fs":"local","fname":"logs/a.csv","size":45678,"crc16":"a1f2"}

A read that stops early gives "error":"readFailed".

Replies from the file system worker

filelist, fileread, filesection, filehash and filedelete do their file system work on the file system worker and reply when it is done, as a deferred response (with a 60 s limit). Where the transport can't take a deferred reply (e.g. a RICREST message with message number 0), or the worker's queue is full, the work is done at once in the main loop, as before.

The reply to an HTTP fileupload is likewise held until the upload has been written, and is {"rslt":"fail","error":"FileWriteFailed"} if writing it failed.

Related

• File Download Protocol (OKTO) — file streaming over RICREST (downloads and uploads)
• File System — the file system worker, flash write stalls and the C++ API
• ProtocolExchange — owns file/stream sessions
• WebServer SysMod — for HTTP access