Skip to content

feat(amx,examples): add minimal AMX support and Qwen prefill runtime - #244

Merged
zhen8838 merged 9 commits into
tile-ai:mainfrom
zhen8838:codex/qwen-amx-prefill-migration
Oct 19, 2026
Merged

zhen8838 merged 9 commits into
tile-ai:mainfrom
zhen8838:codex/qwen-amx-prefill-migration

Conversation

@zhen8838

@zhen8838 zhen8838 commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Keep the Qwen3-1.7B and Qwen3.5-4B AMX prefill experiments executable with shared HIR definitions and a directly callable runtime module.

What

  • Add the four AMX operations used by the example (X/Y loads, FP16 outer product with FP32 accumulation, and Z store) to TileFoundry, together with minimal native instruction wrappers shipped with the package.
  • Admit Copy/Fill on Apple core and AMX executor scopes.
  • Add whole-model HIR, scheduled HIR, explicit Python runtime twins and C++ projection kernels under examples/qwen_prefill-amx. Build/cache native kernels on first use; prepare GGUF weights from the entry point when needed.
  • Keep architecture parameters in Python. No JSON artifacts or standalone build/export/check scripts are included.

Contract

AMX contracts and wrappers belong to TileFoundry; model-specific orchestration and kernels belong to the example. The runtime is hand-written and callable directly after loading resources. CPU host operations use Torch. The example covers batch-one text prefill through logits.

Execution validation on M2 Pro: both full models complete 128-token prefill using existing prepared Q8_0 resources, with finite logits and next-token IDs 1813 / 306. Native kernels build automatically. No tests are added or modified relative to main; no performance measurements were repeated.

Risk

Requires Apple Silicon and Command Line Tools. Execution checks do not establish generation quality. This adds only the AMX instructions needed by the example, without an AMX machine-code generator.

@zhen8838 zhen8838 changed the title feat(examples): migrate Qwen AMX prefill to shared definitions feat(amx,examples): add minimal AMX support and Qwen prefill runtime Oct 19, 2026
Comment thread src/tilefoundry/analysis/footprint.py Outdated
Comment thread examples/qwen_prefill-amx/kernels/prefill.cpp
Comment thread src/tilefoundry/ir/pattern/utils.py Outdated
@zhen8838
zhen8838 merged commit 1457a80 into tile-ai:main Oct 19, 2026
1 check passed
@zhen8838
zhen8838 deleted the codex/qwen-amx-prefill-migration branch October 19, 2026 12:14
zhen8838 added a commit that referenced this pull request Oct 20, 2026
## Why
- TileFoundry 0.0.3 is about to be tagged. `README.md` is the PyPI long
description (`pyproject.toml: readme = "README.md"`), so the news has to
land before the tag for the PyPI page to show it.

## What
- Add three 10/2026 entries to Latest News: the 0.0.3 release, the
Qwen3.8-27B prefill examples (#229, #236), and the Apple AMX prefill
example (#244).
- Move the two 08/2026 entries into the collapsed "Earlier" list so
Latest News keeps the five most recent items.

## Contract
- None.

## Risk
- The 0.0.3 entry reads "is on PyPI" a few minutes before the tag
publishes it, as the 0.0.2 entry did.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant