Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -586,6 +586,12 @@ openpi web /path/to/repo # 指定初始工作区

Web 可以在选择工作区之前预选可用模型。选择仅保留在当前页面,创建会话后确认模型生效再发送第一条消息;模型不可用时会提示并阻止发送,不会自动换成默认模型。打开已有会话时使用该会话的模型。

在 Plan/Setup 原有工具范围内,模型调用 `ask_user` 时,发起任务的 Web 标签页会显示结构化问题卡片:选择选项、添加补充说明或填写自己的答案,复核后才提交给正在等待的工具调用。关闭卡片不会提交草稿;留空的自定义答案表示要求澄清问题。刷新同一标签可恢复尚未过期的提问(未提交草稿不持久化),其他标签不能代答。提问最多等待 15 分钟,停止运行、切换 Session 或关闭 Host 会取消等待;它不替代原生权限审批,也不意味着任意终端自定义界面已支持 Web。

OpenPI 的 `/plan` 调研通过 `plan_ready` 成功提交计划后,Web 会在聊天区展示可收起、可复制的 Markdown 计划卡片,刷新后仍可查看。卡片展示的是工具返回的计划,不代表批准或开始实施;超出 Web 传输上限的结果会标明为预览。输入框上方的开关由现有 Plan 扩展切换当前 Session 的规划状态,不调用模型、不发送命令气泡,也不清空草稿;发送任务后才开始规划。首次发送后,输入框占位提示会说明“本次对话使用 Plan 模式”,刷新后保留,退出后恢复。任务运行期间不能切换模式,退出不代表批准或开始实施。手动 `/plan <目标>` 仍会直接开始规划,`/plan off` 仍可退出;Plan Ready 的实施确认仍需使用 TUI。

本地 #562 后续改动还补充了 `human_handoff` 的人工操作卡片:选择“已完成”或“无法完成”、复核后提交,模型仍需验证完成信号。已适配的 `/openpi-setup`、`/plan`、`/usage`、`/cron` 可从命令菜单选择;Plan 会明确提示实施确认的限制。命令原文和 Plan/Cron 通知、Usage 查询结果显示在聊天区,不额外进入模型上下文。Cron 显示的是执行命令时的任务快照,任务本身仍只保存在原生会话内存中。

Pi 当前只原生分派 `install`、`remove`、`update`、`list`、`config` 和 `auth` 等固定子命令,package 不能注册新的顶层子命令。因此 Web 入口是独立 CLI 的 `openpi web`,不是会被 Pi 当成初始 Prompt 的 `pi open`。Web 进程仍沿用 Pi 的 Provider、模型、凭据、Settings、Trust、Session 格式和 extension 资源加载,不引入第二套 Provider 或 Session 存储。

### 命令速查
Expand Down
2 changes: 2 additions & 0 deletions docs/design/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,7 @@ These records predate [`Decision 0001`](../decisions/0001-documentation-and-evid
- [`WEB_SESSION_CREATION_TARGET.md`](WEB_SESSION_CREATION_TARGET.md) — stable receipt identity and fail-closed target binding for a newly created Web Session's model selection and first prompt
- [`WEB_SLASH_COMMAND_DISCOVERY.md`](WEB_SLASH_COMMAND_DISCOVERY.md) — Pi-owned command discovery, bounded Web projection, availability policy, and Composer completion interaction
- [`COMPLETION_INBOX.md`](COMPLETION_INBOX.md) — shared owner, epoch, consumption, retry, and receipt contract for background completions
- [`WEB_STRUCTURED_QUESTIONS.md`](WEB_STRUCTURED_QUESTIONS.md) — Web `ask_user` ownership, reviewed answers, bounded waits and browser validation
- [`WEB_INTERACTION_UI.md`](WEB_INTERACTION_UI.md) — #562 native command feedback, Plan controls, human handoff and streaming/cancellation corrections

开发与热更新流程见 [`docs/development/OPENPI_WEB_DEVELOPMENT.md`](../development/OPENPI_WEB_DEVELOPMENT.md)。
138 changes: 138 additions & 0 deletions docs/design/WEB_INTERACTION_UI.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Web interaction UI

- Status: draft — implementation under review, not an accepted architectural Decision
- Created / verified: 2026-09-20
- Source boundary: `codex/web-ask-user`, upstream base `45f12a4`, plus this implementation
- Issue: https://github.com/openpi-dev/openpi/issues/562
- PR: https://github.com/openpi-dev/openpi/pull/595
- Related: #348 (Setup), #470 (implementation handoff), #549 (controller identity), #540 (Stop), #561 (Web UI)
- Supersedes: none; complements [Web structured questions](WEB_STRUCTURED_QUESTIONS.md)

## Scope

Expose existing Pi tools and extension commands in Web: structured questions,
human handoff, completed plan cards, a mode-only Plan control and command
feedback. Theme controls and direct preference writes are excluded. Configuration
continues through `/openpi-setup`; this change does not alter that contract.

## Plan ownership and presentation

POST `/api/plan` uses the Host's existing authentication and mutation drain.
The serialized runtime controller validates the selected Session, idle and
unqueued state, and availability of the owned Plan command. A Session-scoped
callback in the Plan extension compares the latest persisted Plan entry ID,
then uses its existing persistence, tool gate and broadcast function. Stale
revisions fail closed. Ready or invalid state requires an explicit exit.
Registration is removed on shutdown and refreshed on branch restoration.

Changing mode sends no prompt, creates no model turn and preserves the draft.
The extension projects its current read-only/ready stance at
`before_agent_start`, including after Session restoration. Permission enforcement
remains in runtime tool gates. The placeholder appears after a real user message
in the current planning episode and is derived from branch entries, not a
second persisted mode flag. `/plan <objective>` retains its immediate-start path.

The plan card displays a successful `plan_ready` result, not streamed arguments.
It does not imply implementation approval. Ready state remains read-only until
an explicit owner action; automatically releasing that gate is not part of this
UI. Implementation review still requires the TUI and belongs to #470.

## Native commands and model-visible context

Registered command identities and owning source paths determine the reviewed
Web menu entries. Unknown extensions remain unadapted; manually submitted input
still reaches Pi with an operator warning. No second command dispatcher exists.

Command input and Plan/Cron/Usage feedback are bounded custom Session entries
used for operator display, not additional model messages. The input records
submission, not success. Native command errors are visible. Usage reuses the
existing provider adapters; Cron output is a historical command result, not a
second scheduler or live task store.

Setup instructions now require explaining invalid values in the user's language
and obtaining an explicit legal alternative before writing. Footer presets and
footer style enum values are distinguished. This guides model interpretation;
the typed writer retains actual input validation. Prompts cannot prove that a
model-selected legal substitute was authorized by the user.

## Human handoff

The existing parent-only `human_handoff` shares the controller-bound question
transport. Done and Unable require explicit review and submission; notes retain
the 8,000-byte limit. Free-form input cannot manufacture completion. Abort,
expiry and missing UI never imply completion. Done remains a user report; the
model must verify the completion signal with available tools.

## Streaming and cancellation corrections

Transcript reconciliation uses native timestamps/tool-call identity and original
part order. Hidden custom entries occupy no blank rows. Uncollapsed tool rows
retain independent keys so regrouping does not discard answer DOM or expanded
state. Collapsed groups retain their existing four-row threshold.

A provider burst regression reproduces disappearing unfinished messages:
`response.write(false)` was treated as a broken SSE connection even though it
only indicates buffered backpressure. Reconnection restored a persisted-only
snapshot, losing unfinished assistant content. Live records and heartbeats now
tolerate native buffering while bounding outstanding bytes plus the next record
at 256 KiB. A separate regression verifies disconnection above that budget.
Genuine disconnect recovery can still temporarily drop unfinished projections.

Stopping a pending question can return its cancelled result before Pi attempts
another model step. Pi's lazy auth setup classified an already-aborted signal as
an ordinary error. Web checks the original signal before delegating to the
existing stream function so Agent's native run lifecycle emits `aborted`.
No dependency file, error-text classifier or provider replacement is introduced.
The guard covers already-aborted steps, not every cancellation timing during auth.

## Validation and limits

Separate tests cover owner state and read-only enforcement, authenticated Host
operations, question ownership/replay/expiry, UI drafts and recovery, native
persisted cancellation evidence and deterministic provider/browser flows.
Question cancellation asserts the receipt and a subsequent successful message,
not just card disappearance. Plan streaming includes both spaced deltas and a
90-delta burst followed by a pause before completion.

Manual acceptance reports confirm that continuous flicker stopped, pending
questions restore on refresh, Plan generation can be stopped, the mode switch
preserves drafts, and completed plans restore on refresh. The final cancellation
classification change has automated coverage; fresh live-model acceptance is
not claimed. No private Session, recording, credential or raw diagnostic export
is included. Final check commands and results belong in the associated PR.

At the earlier `9d8ce99` source boundary, `bun run check` passes; the separate Vitest run passes
245 cases across 13 files; the standard browser suite passes 34 cases and the
native-provider browser suite passes 10. The full Node suite reports 1,683
passes, two failures and one platform skip. Both failures are the existing
PlanMode Markdown tests (`a rich plan renders without throwing and keeps its
content`, `expanded survives edge-content plans at extreme widths`). An archive
of unmodified upstream `9d8ce99`, using the same installed dependencies,
reproduces both failures (22 passes, two failures in that file). No full-suite
green result is claimed. These results use local Node 24.16.0 and synthetic
browser data, not private provider requests.

At that earlier boundary, open PRs #549, #540 and #561 modified shared Web
surfaces and had not been integrated. The 2026-09-20 semantic rebase now includes
#561 at `45f12a4`, preserving its image transport, workbar, settings dialog,
transcript processes and Setup filtering. The Web echo of a Setup command is
hidden with its native episode so hidden replies cannot leave a permanently
waiting command bubble. Settings-hosted questions and Stop remain accessible
inside the active dialog. Exclusive Web Locks prevent inherited controller IDs
from being used concurrently; see [the controller contract](WEB_STRUCTURED_QUESTIONS.md).
#549 and #540 remain separate work. #578 is a separate TUI answer-editor
fix, not this Web questionnaire. This implementation does not close all of #343,
#348, #470 or the remaining proposals in #562.

Integrated validation on 2026-09-20 against `45f12a4`: `bun run check`
passed; the Node suite reported 1,712 passes, two failures and one platform
skip. An unmodified archive of this upstream revision reproduced the same
two Plan rendering failures (22 passes / two failures). The separate complete
Vitest suite passed 318 tests across 22 files with `--maxWorkers=2`; default
high parallelism had produced timeout failures in existing settings/sidebar
tests. No timeout or assertion was relaxed. The standard browser suite passed
50 tests and the native-provider suite passed 12, including copied-storage
controller isolation and settings-dialog answer/cancellation flows. After the
final CSS ordering adjustment, all seven question/handoff browser tests passed
again. These are local automated results, not upstream acceptance or a fresh
live-provider manual smoke.
Loading
Loading