Repository navigation
api: Add a /diag endpoint reporting why the device last restarted - #112
Conversation
An unexpected reset currently leaves no trace a release build can report.
system_check_reboot_loop() decodes rst_info->reason at every boot and
writes it only to DEBUG_SYSTEM(), which compiles to nothing unless the
firmware is built with debugging. Worse, the allocators call rtc_reset()
before system_restart(), which clears FULL_BOOT_TEST_ADDR and makes
rtc_get_value() return 0 for everything on the next boot, so even the
unexpected-reset counter is lost.
GET /diag (admin) reports:
{
"last_reset": {"reason": "out-of-memory", "alloc_size": 1536},
"free_mem": 17584,
"boot_count": 42
}
reason is one of unknown, power, external, software, exception,
hardware-watchdog, software-watchdog, deep-sleep-wake or out-of-memory.
alloc_size holds the size of the allocation that failed and is 0 for every
other reason. free_mem is in bytes, unlike the device's mem_usage, which is
a percentage of MAX_AVAILABLE_RAM and hides the real figure.
This lives on its own endpoint rather than among the device attributes,
since it describes the firmware's own health and has nothing to do with the
qToggle API.
Two details worth knowing:
* An allocation failure restarts through system_restart(), which the SDK
reports as a plain software reset or, depending on where it was called
from, as an exception. Both were observed on the bench, so the RTC
record is the only thing that identifies it. The magic is mixed with
the size so both slots have to agree, and the record is cleared on
every boot so it always describes exactly one reset.
* system_record_alloc_failure() allocates nothing. It writes straight to
RTC memory through new raw accessors that bypass rtc_get_value()'s
full-boot gate, which matters because it runs when the heap is already
exhausted and because rtc_reset() clears that gate on the way out.
Tests under test/test-cases/02-api/18-diag/ cover the response shape, the
auth gate, and a real POST /reset followed by asserting the reason is
exactly "software" with alloc_size 0.
There was a problem hiding this comment.
🔵 Needs a closer look
It modifies the reliability-critical allocator-reset path and hardware-specific RTC memory, which can only be built and validated on real ESP8266 hardware and not in this environment.
No open issues 🎉
What changed in this PR
Adds a new admin-only GET /diag endpoint that reports why the device last restarted (including an out-of-memory cause that the SDK otherwise masks), the free heap in bytes, and the boot count. To make out-of-memory restarts observable in release builds, the allocator failure path now records the failed allocation size to RTC memory via new raw accessors that bypass the full-boot gate, and system_check_reboot_loop() decodes that record into a machine-readable reason string instead of only emitting a debug log.
Changes:
- New
GET /diagAPI endpoint (api_get_diag) exposinglast_reset(reason,alloc_size),free_mem, andboot_count, gated to admin access. - Allocator (
my_malloc/my_realloc/my_zalloc) records the failed size to RTC before restarting;system_check_reboot_loop()detects it via a magic-XOR-size record and maps all SDK reset reasons to stable strings. - New
rtc_get_raw_value/rtc_set_raw_valueaccessors that skip the full-boot gate, plus three blackbox test cases covering response shape, the auth gate, and reason after a real reset.
| File | Description |
|---|---|
src/api.c |
Adds /diag dispatch and the api_get_diag handler building the diagnostics response. |
src/api.h |
Declares api_get_diag. |
src/espgoodies/system.c |
Maps reset reasons to strings, decodes the RTC alloc-failure record, and adds system_record_alloc_failure/getters. |
src/espgoodies/system.h |
Declares the new system diagnostics functions with doc comments. |
src/espgoodies/utils.c |
Records the failed allocation size before resetting in all three allocator wrappers. |
src/espgoodies/rtc.c |
Implements raw (un-gated) RTC read/write accessors. |
src/espgoodies/rtc.h |
Declares the raw RTC accessors. |
test/test-cases/02-api/18-diag/01-get-diag.json |
Validates the full /diag response schema. |
test/test-cases/02-api/18-diag/02-diag-after-reset.json |
Asserts reason software/alloc_size 0 after a real POST /reset. |
test/test-cases/02-api/18-diag/03-diag-security.json |
Verifies the 401/403 auth gate. |
Review effort: Balanced
Makes the reboot cause observable in a normal release build, without a debug image.
Why
An unexpected reset currently leaves no trace anyone can read in production:
system_check_reboot_loop()decodesrst_info->reasonat every boot and writes it only toDEBUG_SYSTEM(...), which compiles to nothing in a release build.rtc_reset()beforesystem_restart(). That clearsFULL_BOOT_TEST_ADDR, so the next boot is treated as a full boot andrtc_get_value()returns 0 for everything — even the unexpected-reset counter is lost.The endpoint
reasonis one ofunknown,power,external,software,exception,hardware-watchdog,software-watchdog,deep-sleep-wake,out-of-memory.alloc_sizeis the size of the allocation that failed, and 0 for every other reason.free_memis in bytes — unlike the device'smem_usage, which is a percentage ofMAX_AVAILABLE_RAMand hides the real figure.boot_countcounts boots since the last full boot, so a burst of resets is visible without history.This sits on its own endpoint rather than among the device attributes, since it describes the firmware's own health and has nothing to do with the qToggle API.
Two details worth knowing
system_restart(), which the SDK reports as a plain software reset or, depending on where it was called from, as an exception. Both were observed on the bench, so the RTC record is the only thing that identifies it. The magic word is mixed with the size so that both slots have to agree, and the record is cleared on every boot so it always describes exactly one reset.system_record_alloc_failure()allocates nothing. It writes straight to RTC memory through new raw accessors that bypassrtc_get_value()'s full-boot gate — which matters because it runs when the heap is already exhausted, and becausertc_reset()clears that gate on the way out.Tests
test/test-cases/02-api/18-diag/— response shape, the auth gate (401/403), and a realPOST /resetthat waits for the device to drop off the AP and come back, then asserts the reason is exactlysoftwarewithalloc_size0. That exercises the whole chain end to end on hardware.All three pass on the bench, along with the rest of the API suite.
DEBUG=false(which adds-Werror) andDEBUG=true DEBUG_FLAGS="system rtc api"both build clean.Note
This endpoint is how the reboot in #111 was classified. Worth having on field devices regardless of that fix, so the next unexplained reset reports itself.