A production-grade system spanning Go microservices, a C++ ESP32 SDK, and a React dashboard — handling provisioning, telemetry, command dispatch, OTA firmware updates, digital twins, and local AI inference with cryptographic security at every layer.
What Is ARGUS · Engineering Decisions · Architecture · Features · OTA Security Model · Firmware Versioning · CI/CD · Load Tests · Tech Stack · Quick Start · API Endpoints · Testing
Most IoT projects stop at "sensor sends data to a dashboard." ARGUS goes further — every ESP32 device is a remotely observable, controllable, and versioned compute node with cryptographic identity, fleet-wide OTA deployments, and local AI that detects anomalies without sending a single byte to an external embedding API.
Built as a learning project that grew into a real distributed system. Two Go microservices — one for fleet management and device control, one for high-volume telemetry ingestion and AI analytics — wired together with Kafka, MQTT, gRPC, and PostgreSQL, containerised in Docker with full Kubernetes manifests.
A few design choices that shaped how ARGUS works.
Fleet-wide OTA with per-device identity. Provisioning generates a unique config_{id}.ino sketch for each device — UUID, API key, mTLS certs, and OTA public key written to NVS flash. After that, an identical firmware binary deploys to every device in the fleet. No per-device values compiled in. Identity survives updates.
ESP32 C++ that tests without hardware. A POSIX/OpenSSL HAL layer stubs out WiFiClientSecure, PubSubClient, HTTPClient, Preferences, and Update.h using real TCP sockets. The SDK compiles and runs on Linux x86_64, making actual HTTPS and MQTT connections to a Dockerised backend. libsodium and mbedTLS run natively. Ed25519 OTA verification is real, not mocked — which is what makes CI possible.
Auditable command lifecycle. Commands move through PENDING → DISPATCHED → ACKED / NACKED / TIMEOUT — written to Postgres, published to Kafka, consumed by an MQTT bridge, delivered over Mosquitto, with the device response flowing back through MQTT → Kafka → Postgres. A background goroutine sweeps unacknowledged commands to TIMED_OUT. Every transition is recorded. Core Service handles only low-volume MQTT traffic (state, commands, OTA) while Telemetry Service independently ingests high-volume device telemetry directly from the broker.
Four-layer OTA verification. Before a single byte touches flash: TLS cert pinning prevents MITM, SHA-256 checksum is verified chunk-by-chunk, Ed25519 signature (libsodium) is validated against the provisioned public key, and key ID is checked against what was written at provisioning. Any failure aborts the update and publishes a NACK.
Dual authentication. Each device gets an ECDSA client certificate signed by an internal CA (validated at TLS handshake) and a per-device API key (SHA-256 hashed, prefix-indexed, validated on every HTTP request). Both layers are independent — either alone is sufficient to reject an unauthorised device.
Local AI, no external APIs. The telemetry service runs all-MiniLM-L6-v2 via ONNX Runtime for 384-dim embeddings entirely in-process. Vectors land in pgvector. RAG queries route across device, fleet, historical, and incident handlers before hitting Groq for reasoning. Anomaly detection runs EWMA + Welford's online algorithm on live streams.
┌─────────────────────────────────────────────────────────────┐
│ Frontend (React 18 + TypeScript) │
│ nginx ──► /api proxy ──► core-service │
└────────────────────────────┬────────────────────────────────┘
│ HTTP/REST + WebSocket
┌───────────┴────────────┐
│ Core Service │ ◄──gRPC──► ┌──────────────────────────┐
│ Go :8080 │ │ Telemetry Service │
│ gRPC server :50051 │ │ Go :8081 │
│ │ │ gRPC server :50052 │
│ Device registry │ │ │
│ Fleet management │ │ MQTT subscriber │
│ Command dispatch │ │ (argus/+/telemetry) │
│ OTA lifecycle │ │ Kafka consumer │
│ Shadow state │ │ Anomaly detection │
│ Internal CA │ │ ONNX embeddings │
│ JWT auth │ │ pgvector RAG │
│ WebSocket broadcast │ │ Rules + alerts │
└──┬───┬───┬─────────────┘ └──┬───┬───┬───┬───────────┘
│ │ │ │ │ │ │
PG Redis MQTT PG Redis Kafka MinIO
+vec (state, Mosquitto +vec /RP
commands,
OTA only)
│ │
│ MQTT │ Kafka
┌──────┴──────────────┐ ┌──────────┴──────────┐
│ ESP32 Devices │ │ telemetry.raw │
│ (argus_sdk C++) │ │ (Telemetry → Core │
│ NVS identity │ │ for WS broadcast) │
│ mTLS + API key │ └─────────────────────┘
│ Ed25519 OTA verify │
│ State machine │
└─────────────────────┘
| Channel | Flow | Purpose |
|---|---|---|
| gRPC | Core ↔ Telemetry | AI queries, snapshots, rules, incidents, device resolution |
MQTT argus/+/telemetry |
Device → Telemetry | High-volume telemetry ingestion (direct from broker, QoS 0) |
MQTT argus/+/state |
Device → Core | Device state updates, LWT presence |
MQTT argus/+/results |
Device → Core | Command result payloads |
MQTT argus/+/ota/status |
Device → Core | OTA progress updates |
Kafka telemetry.raw |
Telemetry → Core | Ingested telemetry for WebSocket broadcast to frontend |
Kafka telemetry.incidents |
Telemetry → Core | Anomaly incidents |
| Feature | What It Does |
|---|---|
| Fleet Provisioning | Create N devices from one API call. Each gets a unique config sketch with UUID, API key, mTLS certs, and OTA public key — all written to ESP32 NVS flash. |
| OTA Firmware Deployment | Fleet-wide or per-device. Four-layer security: TLS cert pinning, SHA-256 checksum, Ed25519 signature, key ID validation. Corrupted binaries never reach flash. |
| Command & Control | Full PENDING → DISPATCHED → ACKED / NACKED / TIMEOUT lifecycle. Postgres audit trail, Kafka distribution, MQTT delivery, WebSocket push to dashboard. |
| Digital Twins | Redis-backed desired/reported state per device. Drift detection flags when the two diverge. |
| Local AI Inference | ONNX Runtime runs all-MiniLM-L6-v2 in-process — no external API. pgvector semantic search, event deduplication, RAG across four handler types. |
| Anomaly Detection | EWMA + Welford's online algorithm on live telemetry streams. Real-time Z-score anomalies trigger alerts through the rules engine. |
| RAG Query Engine | Routes queries across device, fleet, historical, and incident knowledge bases. Groq provides reasoning over local vector context. |
| Session Lifecycle | All telemetry scoped to sessions: start → run → stop. On session end: device summaries, metric aggregates, hourly rollups, and incident archives as MinIO artifacts. |
| 16-Page Dashboard | Landing · Login · Workspaces · Fleet Overview · Devices · Device Detail · Sessions · Telemetry · Commands · OTA · Alerts · AI Insights · Observability · Docs |
| Observability Stack | Prometheus metrics, Grafana dashboards, Loki + Promtail log aggregation, pre-configured alerting rules — all in Docker. |
| Multi-Workspace | Workspace isolation with per-workspace JWT scoping, rate limiting, and resource ownership. |
| Dual Authentication | ECDSA client certificates (mTLS) + per-device API keys (SHA-256 hashed). Both layers are independent and sufficient on their own. |
Layer 1 — TLS + Certificate Pinning
ESP32 pins the server cert SHA-256 fingerprint at compile time.
A rogue CA cannot intercept the connection.
Layer 2 — SHA-256 Checksum
Computed server-side on upload. Verified chunk-by-chunk during download.
Layer 3 — Ed25519 Digital Signature (libsodium)
Server signs the SHA-256 hex string with the fleet private key.
Device verifies using the public key written to NVS at provisioning.
Key ID in the manifest is checked against the key ID in firmware.
Layer 4 — Abort on Any Failure
Wrong checksum · bad signature · unknown key ID
→ update aborted, NACK published, existing firmware untouched.
Keypairs generated by cmd/ota-keygen/. Public key lives in NVS. Private key is environment config only, never in the binary.
The version of a device is the version of the binary it is running. There is no UI field and no database override, so the two cannot drift apart.
One source of truth. FleetFirmwareVersion in
services/core-service/internal/firmware/generator.go is the only place the
version is written. Downloading the fleet template renders it as:
#define ARGUS_FIRMWARE_VERSION "1.0.0"argus_version.h seeds ARGUS_FW_VERSION from that constant and embeds an
ARGUSVER: marker in the compiled image. NVS still receives a fw_version at
provisioning time as an "intended version" for the device row, but the running
firmware does not read it — otherwise a device flashed over OTA would keep
reporting the version it was provisioned with.
Upload derives the version from the file. OTA artifacts are compiled .bin
images. The upload endpoint scans the bytes for the marker:
ARGUSVER:\0<major.minor.patch>\0
The first match wins and the value is stored on firmware_artifacts.version,
which the updates panel displays. A binary with no marker, a malformed marker,
or a payload that is not semver is rejected, never defaulted — see
internal/domain/ota/firmware_version.go. Uploading firmware that carries the
wrong version is therefore not possible.
Artifacts uploaded before this change have no machine-derived version. They are left as they are and remain deployable; the marker requirement applies only to new uploads.
fleets.firmware_version is retained on the fleet form as "intended version"
metadata for the device row. It does not influence which binary runs or whether
an OTA is allowed.
Changing the version means editing FleetFirmwareVersion, re-downloading the
fleet template, recompiling, and uploading the new binary.
Three pipelines. All three must be green before master merges.
e2e-ci.yml — full stack, triggers on master
1. Generate TLS certs + Ed25519 OTA keypair
2. Start 10 infrastructure containers via Docker Compose
3. Provision test device via Go bootstrap script (real API calls)
4. Build C++ SDK test runner natively (CMake + libsodium + OpenSSL)
5. Run simulator for 120s — telemetry 5s, heartbeat 30s, OTA poll 60s
✅ Telemetry — asserted via mosquitto_sub
✅ Commands — ping dispatched, "Pong from hardware" asserted on /results
✅ OTA — firmware uploaded and signed, simulator receives manifest,
verifies SHA-256 + Ed25519, flash write and ACK asserted
go-ci.yml — backend + frontend, triggers on master
Both Go services built with ONNX Runtime. Full unit test suite with -race. React TypeScript build validated.
sdk-ci.yml — ESP32 SDK, triggers on feat/sdk-dev
Arduino-cli compile check against ESP32 core. Native C++ integration test (120s) against a live backend over real HTTPS + MQTT.
Full reports: benchmark_10_devices_20min.md · benchmark_100_devices_60min.md
Hardware: ASUS ROG Zephyrus G14 2022 (Ryzen 9 6900HS, 16 GB DDR5) — core-service as a native Go binary, all infra in Docker.
Tested Pipeline: Benchmark Producer → Kafka (telemetry.raw) → Core Service WebSocket → Frontend (isolates Kafka ingestion & websocket fanout throughput; full device → MQTT → Kafka path validated in CI/CD Pipelines).
| 10 devices · 20 min | 100 devices · 60 min | |
|---|---|---|
| Messages Processed | 11,990 / 12,000 | 359,900 / 360,000 |
| Kafka Ingestion Loss | 0 | 0 |
| Kafka Publish Failures | 0 | 0 |
| Effective Rate | 9.99 msg/s | 99.97 msg/s |
| Avg Enqueue Latency | 21.6 µs | 532 ns |
| Primary Telemetry Consumer Lag | 0 throughout | 0 throughout |
| App Avg CPU | 1.88% | 6.25% |
| App Peak RSS | 228 MB | 402 MB |
| Artifact Generated | 0.06 MB | 0.75 MB |
| Result | ✅ PASS | ✅ PASS |
Notes on Architecture & Transport:
- Benchmark Isolation: The capacity benchmark writes directly to Kafka to measure backend ingestion and aggregation limits without Mosquitto socket bottlenecking.
- Telemetry Durability: Device telemetry (
argus/+/telemetry) is published via MQTT QoS 0 (best-effort) into an internal in-memory buffer to protect MQTT client event loops under high load. Durability and zero message loss guarantees apply once telemetry reaches Kafka.- Kafka Consumer Lag: Primary live telemetry consumer lag remained 0 throughout. Secondary consumer groups (e.g. background AI incident workers) process asynchronously.
| Domain | Technologies |
|---|---|
| Backend | Go 1.22+, gRPC + Protobuf, chi router, Swagger/OpenAPI |
| Frontend | React 18, TypeScript 5, Vite, TanStack Query & Table, Recharts, Framer Motion |
| Device SDK | C++17, Arduino Core for ESP32, libsodium, mbedTLS, PubSubClient, ArduinoJson |
| Data | PostgreSQL 16 + pgvector, Redis 7 |
| Messaging | Eclipse Mosquitto (MQTT 3.1.1), Redpanda (Kafka-compatible) |
| Storage | MinIO — firmware artifacts + presigned URLs |
| AI / ML | ONNX Runtime (local embeddings), Groq API (LLM reasoning) |
| Security | Ed25519, SHA-256, ECDSA mTLS, JWT RS256, TLS cert pinning |
| Observability | Prometheus, Grafana, Loki, Promtail |
| Deploy | Docker Compose (10 containers), Kubernetes manifests + Kustomize, nginx |
| CI/CD | GitHub Actions, CMake, arduino-cli, race detector |
argus/
├── services/
│ ├── core-service/ # Go API — HTTP :8080, gRPC :50051
│ │ └── internal/
│ │ ├── domain/ # Device, fleet, OTA, session, cert, policy
│ │ ├── transport/ # HTTP handlers, gRPC, auth + mTLS middleware
│ │ ├── infrastructure/ # Postgres, Redis, Kafka, MQTT, MinIO
│ │ └── firmware/ # Arduino .ino template generation
│ └── telemetry-service/ # AI + analytics — HTTP :8081, gRPC :50052
│ └── internal/
│ ├── ai/ # Anomaly, embeddings, RAG, actions, memory
│ ├── domain/ # Telemetry service, rule engine
│ └── infrastructure/ # MQTT (direct ingestion), Kafka, Redis, Postgres+pgvector, ONNX, Groq
├── shared/ # Shared Go modules + Protobuf definitions
├── frontend/ # React 18 SPA — 16 pages
├── argus_sdk/src/ # C++ ESP32 SDK
│ # argus · argus_nvs · argus_http · argus_mqtt
│ # argus_ota · argus_security · argus_state_machine
│ # argus_rollback · argus_time · argus_diag
├── sdk_tests/
│ ├── hal/ # POSIX/OpenSSL HAL stubs
│ ├── bootstrap/ # Go device provisioning tool
│ └── ci/ # CI compose override + cert generation
├── deployments/
│ ├── compose/ # Docker Compose — 10 containers
│ └── k8s/ # Kubernetes manifests + Kustomize
├── cmd/
│ ├── ota-keygen/ # Ed25519 keypair generator
│ └── gen-certs/ # TLS cert generator
├── benchmark_10_devices_20min.md
└── benchmark_100_devices_60min.md
# 1. Generate OTA signing keys
cd sdk_tests/bootstrap
go run bootstrap.go --generate-ota-keys --ota-keys-output=../../.env
# 2. Start infrastructure
docker compose -f deployments/compose/docker-compose.yml up -d
# 3. Run services
go run ./services/core-service/cmd/api # :8080
go run ./services/telemetry-service/cmd # :8081
# 4. Frontend
cd frontend && npm install && npm run dev
# Swagger UI → http://localhost:8080/docs
# 5. Kubernetes deployment (optional)
cp deployments/k8s/base/secrets.env.example deployments/k8s/base/secrets.env # or: make k8s-secrets
kubectl apply -k deployments/k8s/overlays/dev| Method | Path | Description |
|---|---|---|
POST |
/auth/register /auth/login |
JWT access + refresh tokens |
POST |
/fleets/ |
Create fleet — provisions N devices, returns zip of config sketches |
POST |
/fleets/{id}/ota |
Deploy firmware to every device in fleet |
GET |
/fleets/{id}/firmware |
Download fleet_firmware.ino for customisation |
POST |
/devices/{id}/commands |
Dispatch command |
GET/PUT |
/devices/{id}/shadow |
Read / set device shadow state |
POST |
/devices/{id}/ota |
Single-device OTA deploy |
POST |
/ota/firmware |
Upload firmware — triggers Ed25519 signing |
GET |
/ai/query |
RAG query over device / fleet / incident knowledge base |
GET |
/docs |
Swagger UI |
# Unit tests
go test -v -race ./services/core-service/...
go test -v -race ./services/telemetry-service/...
# SDK — native C++, no hardware needed
cmake sdk_tests/ -B sdk_tests/build && cmake --build sdk_tests/build
./sdk_tests/build/argus_sdk_test
# Full E2E locally
bash sdk_tests/ci/gen_certs.sh
docker compose -f deployments/compose/docker-compose.yml \
-f sdk_tests/ci/docker-compose.ci.yml up -d --build
cd sdk_tests/bootstrap && go run bootstrap.go \
--base-url=https://localhost:8080 \
--ca-cert=../../certs/root-ca.pem --output=device.env
cmake sdk_tests/ -B sdk_tests/build && cmake --build sdk_tests/build
set -a && source sdk_tests/bootstrap/device.env && set +a
./sdk_tests/build/argus_sdk_test- Fork the repository
- Create a feature branch (
git checkout -b feat/my-feature) - Run the full test suite (
go test -race ./...+ E2E) - Submit a pull request against
master
All three CI pipelines must pass before merge.
Built by Vishal Shetagar
Go · React · C++ · gRPC · MQTT · Kafka · Redis · PostgreSQL · ONNX · Ed25519 · ESP32
