Skip to content

File uploads: check the file against the CRC16 given in ufStart - #29

Merged
robdobsn merged 2 commits into
mainfrom
feature/upload-crc-check
Oct 25, 2026
Merged

robdobsn merged 2 commits into
mainfrom
feature/upload-crc-check

Conversation

@robdobsn

@robdobsn robdobsn commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

ufStart has an optional CRC16. It was parsed and passed along with every block, but never checked, so clients verified uploads with a separate filehash request.

  • The check. FileStreamSession keeps a CRC16-CCITT of the data as it writes it, in order, the same CRC filehash reports. For background writes this runs on the file system worker. After the final block it compares the result with the given CRC16.
  • On a mismatch (new RAFT_CRC_MISMATCH):
    • the file is deleted;
    • the client gets a ufCancel with reason failCRC instead of the final ack;
    • or, if blocks are written as they arrive, a ufStatus reply with reason CRCMismatch.
  • On a match, the ufEnd reply says so with "crc16":"xxxx", so a client can skip filehash. The session ends once the final block is written, so ufEnd usually finds no session. ProtocolExchange therefore keeps the checked CRC, by channel and file name, after the session ends. A CRC is reported only once the whole file has been written and has matched.
  • Compatibility. Older firmware gives no crc16, so a client that doesn't see one keeps checking with filehash. Firmware updates aren't checked here, since ESP-IDF validates the image.

RaftCLI sends the CRC and uses the reply: robdobsn/RaftCLI#4.

Testing

  • Linux:
    • test_file_upload_background covers matching, wrong and absent CRCs: no cancel and a checked CRC; failCRC and the file deleted; no check.
    • test_protocol_exchange covers the real sequence: upload, the session ends, then ufEnd, with the CRC in the reply only when it matched.
    • make check passes.
  • On an Axiom, over the WebSocket:
    • right CRC: blocks acked, and ufEnd replies {"crc16":"19ff","rslt":"ok"};
    • wrong CRC: failCRC in place of the final ack, no crc16 in the ufEnd reply, and the file gone.
  • RaftCLI on the same device: uploads over HTTP, WebSocket, serial and BLE all verified and read back identical.

🤖 Generated with Claude Code

robdobsn and others added 2 commits October 25, 2026 16:03
ufStart has an optional CRC16, which was parsed and passed along with every
block but never checked, so clients checked uploads with a separate filehash
request.

FileStreamSession now keeps a CRC16-CCITT (as filehash reports it) of the data
as it is written, in order - on the file system worker for background writes -
and after the final block compares it with the CRC16 given. A mismatch fails
the upload with the new RAFT_CRC_MISMATCH: the file is deleted and the client
gets a ufCancel with reason failCRC instead of the final ack (or a ufStatus
CRCMismatch reply when blocks are written as they arrive). When the CRC was
given and matched, the ufEnd reply includes it ("crc16":"xxxx"), so a client
knows the device checked the file and can skip filehash; older firmware
doesn't, so clients that see no crc16 in the reply still check.

Firmware updates aren't checked here (ESP-IDF validates the image).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…its CRC

The upload session ends once the final block is written, so ufEnd usually
finds no session (and got a plain "ok" from ProtocolExchange). The protocol's
own ufEnd reply also couldn't know the CRC had been compared.

The session now reports the CRC only once the whole file has been written and
matched (getCheckedCRC()). ProtocolExchange keeps it, by channel and file name,
when the session ends, and gives it in the ufEnd reply ("crc16":"xxxx") whether
the session is still there or not. A mismatch gives a ufCancel (failCRC), no
crc16 in the ufEnd reply, and the file deleted - tested through
ProtocolExchange. The write job no longer also logs a CRC mismatch as a failed
write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@robdobsn
robdobsn merged commit b94528f into main Oct 25, 2026
1 check passed
@robdobsn
robdobsn deleted the feature/upload-crc-check branch October 25, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant