Skip to content

Python scripts with MicroPython (py, tdui, machine, Python Editor) - #11

Merged
schikani merged 14 commits into
mainfrom
feature/micropython
Oct 31, 2026
Merged

schikani merged 14 commits into
mainfrom
feature/micropython

Conversation

@schikani

Copy link
Copy Markdown
Collaborator

Draft: merge after #9 and #10, once main is merged in here and CI and the board tests pass again.

Adds Python to TinyDesk: the shell command py runs MicroPython v1.29.0 scripts, one-liners and the interactive prompt, in the Terminal, over SSH and in the PC programs.

Commits

  • py command: MicroPython embedded (third_party/micropython, vendored and generated by tools/micropython/gen.sh; port in ports/common/python). Scripts use TinyDesk's files and paths; the tinydesk module runs shell commands, publishes MQTT and reads or writes Modbus. Heap only while a script runs: 512 KB PSRAM (ESP32 with PSRAM), 64 KB (C6), the RAM above a 32 KB reserve (4 MB ESP32).
  • asyncio with select, hashlib, os.urandom, .mpy files; @micropython.native / @viper for root on the ESP32 boards and Linux.
  • Editor: Python mode (colours, line numbers, indent, completion), F5 Run with an output pane, error line from the traceback, find and replace.
  • machine (root only, ESP32 boards): Pin with interrupts, ADC, PWM, I2C, SPI, UART, soft buses, neopixel. Board-configuration and link pins are refused; everything a script opened is reset when it ends.
  • tdui: windows on the desktop from Python (labels, buttons, checkboxes, text boxes, progress bars, lists, canvases, timers, dialogs, menus), and window sizes (resize, move, maximize, on_resize).
  • Input: a terminal size reply arriving in the middle of a bracketed paste is taken out of the pasted text.
  • ESP32-C6: 8 KB USB receive buffer (was 1 KB), so long pastes arrive whole.

Tested

  • Host tests (ctest) on Windows and Linux.
  • On the ESP32-C6, the ESP32 with PSRAM and the 4 MB ESP32: Python features 6/6, tdui 14/14, window sizes 8/8, machine 4/4, Python Editor 11/11; five 4.5 KB pastes into the Editor intact on the C6 and the ESP32 with PSRAM.

Before this leaves draft

Docs: tinydesk-project/tinydesk-project.github.io, branch feature/micropython.

py runs a Python script, one line (-c) or the interactive prompt in
TinyDesk Shell, on the ESP32-C6, the ESP32 (with PSRAM and 4 MB) and the
PC programs. MicroPython v1.29.0 is generated into third_party/micropython
by tools/micropython (embed port plus the extmod modules TinyDesk uses);
TinyDesk's port is in ports/common/python: configuration, a file system
that goes through the shell's path rules, and the tinydesk module (shell,
log, MQTT publish, Modbus read/write). The heap is allocated only while a
script runs (512 KB PSRAM, 64 KB on the C6, what is free above a 32 KB
reserve on the 4 MB ESP32).

Also: Ctrl+C now stops long commands (modbus read -i, py) in the PC
programs; CI runs a py smoke test; the desktop PC packages carry
MicroPython's licence.
asyncio (frozen into the firmware from ports/common/python/manifest.py)
with select.poll; the loop's waits sleep the shell task in slices, so
Ctrl+C still stops an idle script. hashlib: sha256 everywhere, sha1 and
md5 through mbedTLS on the boards and in PC builds that have it.
os.urandom from the shell's random source. py FILE.mpy runs a module
compiled by mpy-cross. @micropython.native/viper and native .mpy files
on the classic ESP32 and Linux x86-64, for root only; not on the
ESP32-C6 (RAM stays non-executable) or Windows.

Fixes /lib in sys.path, which was the qstr name "_slash_lib" instead of
the path.
For every file: Enter keeps the indent, Tab / Shift+Tab indent the
selected lines, Ctrl+/ comments them, Ctrl+F / F3 find, Ctrl+R replaces
all, Ctrl+G goes to a line. A .py file also gets colours, line numbers,
Python indent rules and the matching bracket. F5 saves and runs it (or a
.tdsh script) in an output pane that shows the Terminal's session (new
td_terminal_command/draw_view/key/send/set_viewer/pump); F7 types into
it. The line of a traceback is marked and F8 goes there. Ctrl+Space
suggests names (apps/editor_py.c); they can pop up while typing, saved
in the user's settings (td_editor_prefs).

The text buffer is now sized to the file and grows up to TD_EDITOR_MAX,
so a small script leaves RAM to run it on the 4 MB ESP32 (issue #6).
The Files app and the desktop run .py files too (td_is_python,
td_is_runnable), with a py icon.
Pin (with interrupts), ADC, PWM, I2C, SPI, UART, SoftI2C, SoftSPI,
time_pulse_us, bitstream and a frozen neopixel module. Pins and buses
TinyDesk uses are refused, strapping pins warn, and everything a script
opened is released when it ends. The PC programs import machine but
report that there is no hardware.
A script started from the desktop opens windows with labels, buttons,
check boxes, text boxes, progress bars, lists and character canvases,
shows message boxes, input boxes and popup menus, and handles clicks,
keys and timers in Python. The desktop makes the windows on its own loop
(apps/pyui.c); the script talks to it through two queues (_tdui), so a
busy script never slows the desktop. Windows close when the script ends,
and Ctrl+C in one of them stops it.
The desktop asks the terminal for its size once a second. A reply that
arrived in the middle of a bracketed paste (a few KB at 921600 baud takes
50-100 ms) was stored as pasted text, so a pasted script could contain
"[25;80R". The reply is now taken out of the paste and handled as usual,
and the query is not sent while a paste is arriving.
Window.resize/move/maximize/minimize/restore/fullscreen, width and height
kept current, and on_resize(win, w, h) for every change, the user's too
(the desktop compares each script window's client size every tick).
Canvas.resize/move, and lists with a size of 0 or less reach to the
window's edge and follow it.
The IDF 5.3.1 USB Serial/JTAG driver drops 64-byte packets when its
receive ring is full. With 1 KB, a paste of a few KB that arrived while
the desktop was drawing lost text. 8 KB matches the UART boards and
TD_PASTE_MAX; it costs 7 KB of RAM.
ESP32-S3: MicroPython's interpreter state goes to PSRAM (extram.lf, as on
the classic ESP32) and the heap is the usual 512 KB of PSRAM. The banner
names the chip (TD_PY_ESP32S3 from the component). machine knows the S3:
UART0 on GPIO43/44 is the desktop's link unless console.* moves it, UART0
is kept for the console, and GPIO0, 3, 45 and 46 are strapping pins; the
flash and Octal PSRAM pins are refused as the chip's. tests/test_py_pins.c
covers it.

@micropython.native/viper and native .mpy files (root only) now also run
on the ESP32-S3 (Xtensa) and the ESP32-C6 (MICROPY_EMIT_RV32,
-march=rv32imc). Both chips' firmware kept RAM non-executable, so
CONFIG_ESP_SYSTEM_MEMPROT_FEATURE=n on the S3 and
CONFIG_ESP_SYSTEM_PMP_IDRAM_SPLIT=n on the C6, as in MicroPython's own
firmware; the classic ESP32 has no such protection. The C6 has the same
free internal RAM at boot as before (115.8 KB). Regenerating
third_party/micropython with the RISC-V emitter on gives the same files.

Tested on an ESP32-S3 (16 MB flash, 8 MB Octal PSRAM): py features 6/6
(native and viper included), machine 4/4, tdui 14/14, tdui sizes 8/8,
Python Editor 11/11, long pastes 5/5. ESP32-C6: py features 6/6 with a
RISC-V native .mpy. PC unit tests 11/11.
The firmware images carry wolfSSH and wolfSSL (GPLv3) and now MicroPython
(MIT); the notes name them and where the licence texts are.
Conflict in CMakeLists.txt: both added a test (test_py_pins and
test_desktop); both kept.
@schikani
schikani marked this pull request as ready for review October 31, 2026 05:24
@schikani
schikani merged commit 80059d8 into main Oct 31, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant