Quickstart · Architecture · Benchmarks · Hardware Errata · ISA Docs · Clean-Room Boundary
Clean-room userspace runtime, closed-form VLIW ISA synthesizer, and llama.cpp GGML backend driving Google Pixel Tensor G5 TPU directly over /dev/edgetpu-soc with zero proprietary vendor .so dependencies.
| Layer | Component | Responsibility |
|---|---|---|
| LLM Inference | llama-cpp-edgetpu/ |
EDGETPU GGML ACCEL backend (MUL_MAT Q8_0 / Q4_0 / F32, shared-A QKV & gate/up batching, 1D bias ADD, 64B host storage) |
| Open Runtime | libopenbuenos/ |
C11/C++17 static library: 1,286.4 MHz DVFS + FW WakeLock, FD-capped DMA-BUF pool & O(1) weight cache, 16-credit VII ring, M=1 & M=64 VLIW GEMM emitter |
| Kernel Transport | runtime/ |
Standalone bare-metal runner & UAPI/Litebuf VII headers (0x0008 -> 0x00fe, /dev/edgetpu-soc 0xED) |
| ISA Tooling | tools/ |
Pure-Python DGC0 dissector/patcher, multi-tile (M, K, N) VLIW synthesizer, 0x640 minimax spline codec, 0xa40 chained fusion verifier, INT4 packer |
Build libopenbuenos, verify strict C11/C++17 headers, and run the offline unit test suite (zero device or proprietary .dgc0 files required):
git clone https://github.com/saariuslystoned/freetpu.git
cd freetpu
# 1. Build & test libopenbuenos (CMake + host mock mode)
cmake -S libopenbuenos -B libopenbuenos/build/cmake-host -DCMAKE_BUILD_TYPE=Release
cmake --build libopenbuenos/build/cmake-host -j
ctest --test-dir libopenbuenos/build/cmake-host --output-on-failure
# 2. Run clean-room Python VLIW ISA & 0x640 minimax spline self-tests
python3 tools/emit_custom_gemm.py --self-test
python3 tools/spline_0x640.py --self-test --no-emit
python3 tools/pack_int4.pyTo cross-build libopenbuenos.a, tpu_baremetal_runner, and llama.cpp (llama-cli / llama-bench / test-backend-ops) for Android ARM64 (android-34):
make -C libopenbuenos verify-header android verify-android
make -C runtime verify-baremetalSee llama-cpp-edgetpu/README.md for patching upstream ggml-org/llama.cpp (b12608).
All hardware metrics below were measured on Google Pixel 10 Pro Fold (rango, Tensor G5 buenos_a0, buenos.ko ABI 1.4) with /proc/self/maps confirming zero loaded Google/vendor .so libraries:
| Proof Suite | Scope | Verified Result |
|---|---|---|
| DVFS + FW WakeLock Silicon Unlock | 1x2048x2048 & 64x1536x1536 (PM=5 @ 1,286.4 MHz + Litebuf AcquireWakeLock) |
27.59 GB/s (152 us HW), 983.6 INT8 GOPS @ 177.2 mW = 5.55 TOPS/W (307 us HW, max_abs_err = 0) |
| Closed-Form Multi-Tile & Large-Tile ISA | 92 compiled reference models (81 base + 6 lt_1x* + 5 lt_64x*) |
92/92 bit-exact chunk[0..6], ext[0] (DIVE), ext[2]; 16/16 libopenbuenos host suites PASS |
0x640 Minimax Splines & Chained Fusion |
SiLU/GELU/exp + 4 Mode-A (+0 B) + 3 multi-op 0xa40 probes (MLP, SwiGLU, QKV) |
12,288/12,288 spline bytes + 3/3 chained probes (60.8–72.8 us, 8/8 ext[2] slices bit-exact, max_abs_err = 0) |
Native W8A16 (BF16) FP32-I/O GEMM |
1x64x32 (2,048 B u8 + 128 B per-channel FP32 scales) |
32/32 bit-exact FP32 (max_abs_err = 0.0, 411–426 us HW, zero Google .sos) |
ggml-edgetpu Backend Ops |
Live rango test-backend-ops -b EDGETPU -o MUL_MAT (Q8_0 & Q4_0) |
104/104 q8_0 & 104/104 q4_0 PASS (327/327 host mock), cpu_fallback_slices = 0, DT_NEEDED = [libdl.so, libm.so, libc.so] |
Live LLM Throughput & Energy (llama-bench) |
SmolLM2-135M-Q8_0 & Qwen2.5-1.5B-{Q8_0,Q4_0} (PM=5 @ 1,286.4 MHz, MAPS_GOOGLE=0) |
SmolLM2: pp512 = 563.82 t/s, tg64 = 17.36 t/s; Qwen2.5-1.5B-Q8_0: pp512 = 105.87 t/s (-21.5% system mJ/tok vs CPU), tg64 = 5.85 t/s, 22/22 verbatim CPU token match |
See docs/PHASE6_PERF_REPORT.md for the full 18-case ODPM mJ/tok energy matrix and silicon scaling tables.
DEQUANTIZE(INT4) -> FULLY_CONNECTED(FP32)Compiler Deadlock (LiteRT#12178): Live silicon ablation proves Tensor G5 (buenos_a0) has no native 4-bit nibble-unpack DMA or systolic unit. Descriptor0x980(w8[21] = 1,w8[29:22] = zp) is a1:2hardwareUINT8 -> BFLOAT16expander (BF16(float(u8 - zp)),16/16exact at398 us), so emitting1,024 Bof nibbles delivers2,048 Binstead of4,096 BofBF16to the MAC and stalls execution indefinitely. FreeTPU rule: unpackQ4_0weights once at load time into cacheduint8DMA-BUFs (same1 B/weightTPU DMA bandwidth asQ8_0), eliminating per-token CPU unpack overhead.- Native
W8A16 (BF16)FP32-I/O GEMM Unlock: Unpacking to2,048 Buint8(w + 8) +128 Bper-channelfloat32scales at+0x800executesFP32 in -> BF16 MAC (1:2 u8->BF16 weight dequant) -> per-channel FP32 scale -> FP32 outnatively on silicon in411–426 us(32/32bit-exactFP32,max_abs_err = 0.0). Local LLMs get fullBF16activation precision (zeroINT8clipping loss) atQ8_0DRAM bandwidth. See docs/hardware-errata.md.
freetpu is strictly clean-room source and specification only:
- Zero vendor binaries: No proprietary
.so,.ko,.img,.apk,.gguf,.tflite,.dgc0, or firmware blobs are included in this repository. - Zero vendor userspace dependencies:
libopenbuenos,tpu_baremetal_runner, andggml-edgetputalk directly to the Linux kernel (/dev/edgetpu-socand/dev/dma_heap/system). - Host mock mode: Setting
ob_config_t::mock_mode = 1(orOPENBUENOS_MOCK=1) runs the full buffer pool, graph LRU cache, 16-credit async ring, VLIW emitter, and quantized math suites offline on macOS/Linux without TPU hardware.
Running synthesized .dgc0 microkernels on physical Tensor G5 hardware requires the 638,977 B RV64GCV DIVE VM interpreter image (ext[1]) from the user's own device. Do not redistribute this binary. Extract it locally on an unlocked Pixel using libopenbuenos:
#include "openbuenos.h"
#include <stdlib.h>
size_t interp_size = 0;
ob_gemm_extract_dive_interp_from_dgc0("/path/to/local_compiled.dgc0", NULL, 0, &interp_size);
uint8_t *buf = (uint8_t *)malloc(interp_size);
ob_gemm_extract_dive_interp_from_dgc0("/path/to/local_compiled.dgc0", buf, interp_size, &interp_size);Or point libopenbuenos / ggml-edgetpu directly at your local extracted image (or a local .dgc0 file) at runtime:
export OPENBUENOS_DIVE_INTERP_PATH=/data/local/tmp/tpu_open_stack/dive_interp_rv64.bin- Phase 6 Performance & Energy Report —
1,286.4 MHzDVFS + WakeLock unlock,983.6 GOPS(5.55 TOPS/W), batchedM=64prefill,SmolLM2/Qwen2.5benchmarks & 18-case ODPMmJ/toksweep - Phase 5B Live LLM Spec —
Probe Adynamicparam_slice,O(1)weight cache, outlier peeling & baseline bring-up - DVFS, Power States & WakeLock Spec — Corrected
+0x1c..+0x20Litebuf wire fields,TpuCorePowerState(0x03..0x19),MemoryPowerState(0x01..0x15) & FW WakeLock (union_type=4/5) - Chained Multi-Op Fusion & W8A16(BF16) Spec — Zero-DRAM
0xa40chaining (MLP,SwiGLU0x100MUL,QKVconcat) &W8A16(BF16)scaling - Hardware & Driver Specs — Tensor G5 (
buenos_a0) geometry,buenos.koABI, rounding & SRAM layout - Hardware Errata & W8A16 (BF16) —
DEQUANTIZE(INT4)deadlock root cause,Q4_0caching rule &W8A16 (BF16)FP32 GEMM unlock - Litebuf VII Wire Protocol — Two-layer
0x0008 -> 0x00fecommand/response framing & 16-credit ring - Multi-Tile GEMM ISA — Shadow-CSR bank,
4x4serpentine grid, 2D DMA rewind,DIVE/DWS0/DGC0layout - INT4 (
W4A8) & Kernel Fusion —CSR 0x56, 4-bit nibble packing (50%BW savings), Mode-AGEMM + SiLU/GELU 0x640Minimax Splines —208 BLoadCoefficientTablescodec &SiLU/GELU/expRemez tables- DMA & TTU Bitfields —
0xa40/0x9c0/0x940/0x980DMA descriptors & SRAM ring setup - Vector ALU ISA —
stream125FP32-RNE quantization & activation pipeline - Kernel UAPI Headers — Clean-room
/dev/edgetpu-soc,/dev/gxp, and GCIP IIF headers libopenbuenosGuide &ggml-edgetpuGuide
Licensed under the Apache License 2.0.
