CoR: Migrate to "Copilot" harness (update MCP; fix autopilot, model & custom agent carryover)

docs/copilot-create-project.md

@@ -134,9 +134,10 @@ session** running the next agent. Between hand‑offs, agents open **webviews**
 ## Prerequisites
 
 - **VS Code** with **GitHub Copilot** enabled and signed in.
-- A Copilot plan with access to at least one supported model. The model picker lists the Opus, Sonnet,
-  GPT Sol, and GPT Terra models currently available through GitHub Copilot, so newly available
-  versions appear without an extension update. The lowest-version available Opus model is selected by default.
+- The **Copilot CLI** chat target. Copilot on Rails manages Auto through the workspace setting
+  `github.copilot.chat.cli.autoModel.enabled`, enabling it only when **Auto** is selected in the picker.
+- A Copilot plan. If no models load in time, the view shows **Models did not finish loading**.
+  You can still press **Plan** using **VS Code default** and change the model in Chat.
 - **A clean project folder.** The flow needs an empty workspace root to build in. If the open folder already
   contains files, choose **Create in New Subfolder…** to create the project under the current folder, or
   **Choose Empty Folder…** to build elsewhere. Either choice opens the project in a separate window without
@@ -173,9 +174,9 @@ description, optionally pick a **Model**, and press **Plan** (or `Ctrl+Enter`).
   <img src="images/copilot-create-project/03-create-project-prompt.png" alt="Create with Copilot prompt view" />
 </p>
 
-Pressing **Plan** starts the **`azure-project-plan`** agent in a new Copilot chat session. Every new-project
-prompt goes through this flow, including frontend-only apps with no backend, database, or Azure services —
-those simply plan a single `frontend` service with **No datastore required**.
+Pressing **Plan** opens Copilot Chat if needed, then starts the **`azure-project-plan`** agent in a new chat
+session. Every new-project prompt goes through this flow, including frontend-only apps with no backend,
+database, or Azure services. Those simply plan a single `frontend` service with **No datastore required**.
 
 ## Stage 2 — Review requirements
 
@@ -456,7 +457,7 @@ silently so a stale copy can't make an agent follow outdated steps.
 ## The MCP tools
 
 The extension exposes these tools to Copilot through the `vscode-azureresourcegroups.mcp` server
-("Copilot Azure Resources Extension Tools"). Agents call them to open views and trigger the next stage.
+("Copilot Azure Resources Extension Tools").
 
 | Tool | Effect |
 | --- | --- |
@@ -494,6 +495,7 @@ Everything the flow produces lives in the workspace, so it's inspectable and rev
 | `.copilot-azure/sessions/{id}/context.json` | deploy agent | Current phase and completed phases. Drives the Deployment progress view. |
 | `.azure/deploy-result.json` *or* `.copilot-azure/sessions/{id}/deploy-result.json` | deploy agent | In-progress and final deployment status, target, endpoints, resources, and recovery attempts. Drives Deployment progress and backs Deployment results. A workspace can hold several; the active session's result is used. |
 | `.github/agents/**` (+ `.version`) | extension | Copied agent instruction files and the version stamp. |
+| `.vscode/settings.json` | extension | Workspace overrides that enable the Copilot Harness for CoR sessions. |
 
 Session/diagnostics state is kept in VS Code **workspaceState** (not files): `copilotOnRails.prompt`,
 `copilotOnRails.createdAt`, and `copilotOnRails.diagnosticEvents` (see below).
@@ -554,13 +556,37 @@ The diagnostics object has four fields:
 
 | Field | Value |
 | --- | --- |
-| `prompt` | The project description the user typed. |
-| `createdAt` | ISO‑8601 timestamp of when the project was first prompted. |
-| `systemInfo` | The operating system, CPU, Node.js, and VS Code versions captured when the project started. |
+| `prompt` | The submitted project description. |
+| `createdAt` | ISO-8601 timestamp of when Plan was pressed. |
+| `systemInfo` | The operating system, CPU, Node.js, and VS Code versions captured on prompt submission. |
 | `diagnosticEvents` | Up to the **75 most recent** events, each: `timestamp`, `name` (command/tool), `type` (`extensionAction` \| `mcpTool` \| `webviewAction`), `status` (`start` \| `success` \| `error`), and a `properties` bag. Error messages are **masked** before being recorded. |
 
 `report_agent_launch` contributes at most one diagnostic lifecycle for each agent in a chat session.
 
+Pressing **Plan** resets cached CoR project state and records the submitted prompt, timestamp,
+system info, and telemetry-only `corProjectId`.
+
+An empty catalog gets one retry after one second; the retry budget resets on extension-host reload.
+Discovery adds these properties to the `createProjectWithCopilot` event and CoR telemetry:
+
+| Property | Meaning |
+| --- | --- |
+| `modelPickerVendor` | The queried vendor, `copilotcli`. |
+| `modelPickerAvailableModelCount` | Number of entries in `modelPickerAvailableModels`, before filtering. |
+| `modelPickerFilteredModelCount` | Number of entries in `modelPickerFilteredModels`, after filtering and deduplication. |
+| `modelPickerRetried` | Whether this invocation retried an empty catalog. |
+| `modelPickerOutcome` | `querying` before discovery completes, `emptyCatalog` for no returned models, `noSupportedModels` when none pass the filter, `queryFailed` when the API throws, or `ready` when supported models are available. |
+
+The diagnostic-only lists `modelPickerAvailableModels` and `modelPickerFilteredModels` contain returned
+names and deduplicated supported names, respectively. Auto is excluded from the filtered list/count
+but remains in the available list/count if returned by the API. The `default` sentinel is never counted.
+Discovery errors record a masked `modelPickerError`.
+
+The fallback records `copilotModel: default`, `modelSelectedInView: false`, and
+`chatModelSelectionSource: default`, and omits `modelSelector`. Explicit selections record
+`chatModelSelectorId` and `chatModelSelectorVendor` in telemetry and diagnostics.
+`chatModelResolved` confirms selector construction, not VS Code's final Chat selection.
+
 Privacy guarantees, by design:
 
 - Diagnostics are **workspace‑cached only** (VS Code `workspaceState`).
@@ -579,6 +605,8 @@ before submitting.
 | --- | --- | --- |
 | *"Choose where to create your project."* | The open folder isn't empty. | Choose **Create in New Subfolder…** to build under the current folder, or **Choose Empty Folder…** to build elsewhere. Both open the project in a separate window. |
 | The new project window opens in Restricted Mode and the Azure Project view is unavailable. | Workspace Trust must be granted explicitly; extensions cannot trust a folder on your behalf. | Select **Trust** from the Restricted Mode banner or Workspace Trust editor. The pending create flow resumes automatically after the extension activates. |
+| The create view shows a model-unavailable message instead of the picker. | The CLI model catalog is empty, has no supported models, or discovery failed. | Submit the plan normally using VS Code's default, or wait a moment and reopen the page to rediscover models. Inspect the discovery properties described above to distinguish the cases, including `modelPickerError` when the API failed and `modelPickerRetried` for the one-time retry. |
+| Chat shows Auto after selecting a named model. | Wrong model pool or a selection change in VS Code. | Compare `copilotModel` with `chatModelSelectorId`; `chatModelSelectorVendor` should be `agent-host-copilotcli`. |
 | An agent says it needs its instruction files, or behaves oddly / follows outdated steps. | `.github/agents/` is missing or stale. | Accept the download prompt, or run **Download Azure Agent Instructions**. The version stamp auto‑refreshes stale copies. |
 | Chat opens with the wrong agent or generic Agent mode. | VS Code did not honor the requested custom mode, the custom instructions were not loaded, or the MCP tool was unavailable. | Inspect `diagnosticEvents` for a successful `report_agent_launch` event. Its `agentName` property identifies the agent that reported. If the event is missing, the startup report never reached the CoR MCP server. |
 | Frontend preview stuck on *"Starting…"*; **Approve UI** never enables (but the app loads in a normal browser). | A second dev server is contending for the preview port. | Stop **all** manually‑started dev servers, free the port, ensure the frontend's `vite.config` is the clean minimal version, then reopen the preview and let it own the server. Don't verify by starting your own server. |

evals/agent-assets.lock.json

@@ -1,9 +1,9 @@
 {
-    "agentAssetsHash": "d89408d687ef9d5863e1ff1ac173f6c0440a273f449484afb90e096a2df7f710",
+    "agentAssetsHash": "1f75802a0c28b60ebb7b5d7f22a485353991869aea6d58e2e939fbcb0471d3ca",
     "scope": "azure-debug-generate.agent.md, azure-debug-generate/**, azure-debug-plan.agent.md, azure-debug-plan/**, azure-deploy.agent.md, azure-deploy/**, azure-project-integrate.agent.md, azure-project-integrate/**, azure-project-plan.agent.md, azure-project-plan/**, azure-project-scaffold.agent.md, azure-project-scaffold/**, shared-references/**",
-    "updatedAt": "2026-09-22T17:17:03.781Z",
+    "updatedAt": "2026-10-05T18:03:28.672Z",
     "files": {
-        "azure-debug-generate.agent.md": "c7f90ce78e00a2900bcac459393db8a2e76029dd0de77da5d8c0a44cd744b799",
+        "azure-debug-generate.agent.md": "f0d7ab870df4619ded4252380198d197eaf03aca8d003694cf473f7a8b580d76",
         "azure-debug-generate/.metadata.json": "c0ccce18678415f23db0073d48cc646f2de446e7821c99c3130553548d958e0c",
         "azure-debug-generate/instructions.md": "2dbf252f05dc7f523d726500941d4862a64a5579ff530b21400fc8a5294fed29",
         "azure-debug-generate/references/api-test-collections.md": "9ef4777f4b682b471b5386c97bb02cb08037545eb520d4534f69b784f5cd798b",
@@ -25,7 +25,7 @@
         "azure-debug-generate/references/runtimes/dotnet.md": "191ce6cb3eee84d4277d746c4ad094750a5dde0003b40ee7a64cbc621af0522f",
         "azure-debug-generate/references/runtimes/node.md": "7e037c0fdcf48ac440c046df35aa3112a9c356ea404655c3ecbbf862237b30a9",
         "azure-debug-generate/references/validation.md": "0daa62d9da8fc7d2a7e6a29ced7378720a2fb962cd67e032ce7e74074a6381cd",
-        "azure-debug-plan.agent.md": "e04cd53adee43b573ccc42eb423c5b4a29b7c28870599ffdf786ea51dbbef219",
+        "azure-debug-plan.agent.md": "150dd1c1a1ca4e9d0bae38b802050c9fbcf5624f0340ed70da361f7196113239",
         "azure-debug-plan/.metadata.json": "108c105ec0eaf5d7f0c2268ab2e0b04eb037649dc6668272f5ce42aa6da51de9",
         "azure-debug-plan/instructions.md": "0e15215047a1703dbbb868a6cd2a58ee1e2dc2bc231fc3f2e81e8056cd3dd9ea",
         "azure-debug-plan/references/classify.md": "1673b52fb25ff89f0437263ce7082f690c208a0783f140f432bb94a463376efa",
@@ -35,7 +35,7 @@
         "azure-debug-plan/references/plan-template.md": "fa08df53649226bd749371f08a9f096cd22ac83d1497e9ae90638993c464af0f",
         "azure-debug-plan/references/project-types.md": "4eca6fbec5cd2220b18ac5bad0ee7034870aa3739c78fb48d4338e2701bde4c6",
         "azure-debug-plan/references/runtimes.md": "97c9903a22e0580c4c0ff4d27a1e31c8a16e7ce07056f573701bf816bdfe45c3",
-        "azure-deploy.agent.md": "082f70b8e524d68ddef0faec69e514a4353d9cbc5540fbbc4d2cd7255a26c7d5",
+        "azure-deploy.agent.md": "a1da0183a9b4c07c5d7e1182cd435a02e19088d0f75b9dd1d06a9f5b3821849a",
         "azure-deploy/.metadata.json": "be0709e660b5da6f558d73e464bed2cd39a3382619bc9076796078843d43cb56",
         "azure-deploy/cor-references/migration-access.md": "2dce754d450740ade47ab1079e1d3046850c54da7d5acc19209825af8f9fdfb0",
         "azure-deploy/deploy/instructions.md": "30f877417f9d1da592abf903a12dc961e36c4b5063c48bab561ac1c152055abe",
@@ -124,22 +124,22 @@
         "azure-deploy/scaffold/references/waf-checklist.md": "d0b50ac5df46569034dd16431aa9ad9faa447f4330a4f4e41447042b160a3f0a",
         "azure-deploy/scaffold/scripts/scaffold-conformance.ps1": "85398b7fcd06e8207e536f558cca68d24ecf6186e54bf5680d7872aa38913ea0",
         "azure-deploy/scaffold/scripts/scaffold-conformance.sh": "c9961efccdece4c6d2a1c3a832389b359fb21ca0706fd7c33215d4e8bf267664",
-        "azure-project-integrate.agent.md": "8b955cc97e2d8e6ef406ef958980b1f0f7d360ab4ab6a602e86f3379e285d72b",
+        "azure-project-integrate.agent.md": "4ace6f03550113856273d82d7624b32272247431fc363bb0e12c1376238e8697",
         "azure-project-integrate/.metadata.json": "ee73bb3c96f66ed4c6f9d37db0716600e5deb3a23f55adcefcf34d6ed8d556c0",
         "azure-project-integrate/instructions.md": "0937e021c41429e71e8977b9a23bc5586e11de80d22ca0f85312b35643ea2ec4",
         "azure-project-integrate/references/end-to-end.md": "11bc63351517d530c7abc34a33842a89e25f9bad8951a7df2035b8a3c3a872aa",
         "azure-project-integrate/references/migrations.md": "4aa872a898ad09d4b22841e609c56350529a894a1e4f06030b0cd89ead0f2b13",
         "azure-project-integrate/references/smoke-test.md": "7c76b1734d5e82efaa23b6b61865b5d6c51d8cce8295b82046e2ac623ac45fa4",
         "azure-project-integrate/references/wire-live-data.md": "723e0b8ff0be74577d1619af774e9693ba2d97e0e7507eb95540cbacafa14643",
-        "azure-project-plan.agent.md": "5cc322c956a9f1061060674b884cca45d5ee869eaa7f3e5f977533e6a73188d5",
+        "azure-project-plan.agent.md": "aec64bb9d77ad872c5e44ff78400cee537db4e30990d3e52e43c412ab8ff74a4",
         "azure-project-plan/.metadata.json": "cec03694d63e45c8cb96983bd586790878b5813289885ae94d6cefc42243f37e",
         "azure-project-plan/instructions.md": "0d68b18de65bbe4fe375329d008373a8c915c6740f1aba5ecfce51980022e0c3",
         "azure-project-plan/plan.md": "a56f162fca888e1c81ea3f0c419c5f03712df1ebfb039bbcd665d5391572b64d",
         "azure-project-plan/references/html-preview.md": "2db272c10c4535743366483610dc46fb885d24795298220d0585880671665369",
         "azure-project-plan/requirements.md": "d53f610177edb81869165abc500cfd19960a887415e12cb77aa4cc0ace4a9b3d",
-        "azure-project-scaffold.agent.md": "fae02fae75b856b3a57d2bc5133ba9fe42d40257c9e862b1864606228b3d58ab",
+        "azure-project-scaffold.agent.md": "1fd514686d4c9c30e85947acfefc77c9e4dc7537cd1a5ebd4535c77e8a0126ab",
         "azure-project-scaffold/.metadata.json": "c6e76bb8c81c61ad58ce659e57d9ac7526bb680ad3923913db3a14e1333cd1e0",
-        "azure-project-scaffold/instructions.md": "99ffd0dfc6afd4f9a21a2610ab1b6eb92b63f5b4e8252b962192495bdeef7143",
+        "azure-project-scaffold/instructions.md": "e59d0266cc8c226f8f9b9d7c0985320daed59e33c3a376fe1046a325125f74f6",
         "azure-project-scaffold/references/frontend-preview-steps.md": "dc60b0dd5a997bd1b38410b8c1e68d6453261a3e23b93de00654c864b49539cf",
         "azure-project-scaffold/references/frontend-quality-bar.md": "3893f2cc21b7d3334cd3c464d065e7d50c3fc68f2d1cc680c85f78c781f45f52",
         "azure-project-scaffold/references/sub-agent-strategy.md": "225ae17f9c89722f31bd40e6470ffd8a1c693a4cbc00c8024127f070fc5d9b8e",

package.json

@@ -1045,7 +1045,7 @@
         "@types/vscode-webview": "^1.57.5",
         "@types/ws": "^8.5.10",
         "@vscode/test-cli": "*",
-        "@vscode/test-electron": "*",
+        "@vscode/test-electron": "^3.1.0",
         "@vscode/vsce": "*",
         "esbuild": "*",
         "esbuild-plugin-copy": "*",
@@ -1061,7 +1061,7 @@
         "@microsoft/vscode-azext-azureutils": "^4.3.0",
         "@microsoft/vscode-azext-utils": "^4.1.0",
         "@microsoft/vscode-azext-webview": "^1.0.2",
-        "@microsoft/vscode-inproc-mcp": "^1.0.0",
+        "@microsoft/vscode-inproc-mcp": "^1.1.0",
         "form-data": "^4.0.6",
         "fs-extra": "^11.3.0",
         "jsonc-parser": "^2.2.1",

resources/agents/azure-debug-generate.agent.md

@@ -1,7 +1,7 @@
 ---
 name: azure-debug-generate
 description: Generate artifacts for an approved VS Code debugging plan. Responsible for generating docker-compose, VS Code launch/tasks, and emulator orchestration for streamlined project debugging.
-tools: [vscode, copilot-azure-resources-extension-tools/*, tool_search, execute, read, agent, browser, edit, search, web, azure-mcp/search, todo]
+tools: [vscode, copilot-azure-resources-extension-tools/*, Copilot Azure Resources Extension Tools/*, tool_search, toolSearch, execute, read, agent, browser, edit, search, web, azure-mcp/search, Azure MCP/*, todo]
 target: vscode
 ---
 

resources/agents/azure-debug-plan.agent.md

@@ -1,7 +1,7 @@
 ---
 name: azure-debug-plan
 description: Scan an Azure-centric workspace project. Classify its services and dependencies, and produce a local debugging plan covering automated emulator startup, VS Code launch/task configs, and API tests.
-tools: [vscode, copilot-azure-resources-extension-tools/*, tool_search, execute, read, browser, edit, search, web, todo]
+tools: [vscode, copilot-azure-resources-extension-tools/*, Copilot Azure Resources Extension Tools/*, tool_search, toolSearch, execute, read, browser, edit, search, web, todo]
 target: vscode
 ---
 

resources/agents/azure-deploy.agent.md

@@ -1,7 +1,7 @@
 ---
 name: azure-deploy
 description: "Onboard and deploy an Azure-centric project end-to-end using a guided, self-contained onboarding pipeline. Analyzes deployment readiness, selects Azure services and SKUs, estimates cost, validates quota, generates secure Bicep/Terraform, provisions resources, deploys application code, and verifies health. Run after local development is set up. WHEN: deploy to Azure, ship to Azure, host on Azure, create infrastructure, generate IaC, provision resources, go live."
-tools: [vscode, copilot-azure-resources-extension-tools/*, tool_search, execute, read, agent, browser, edit, search, web, azure-mcp/search, todo]
+tools: [vscode, copilot-azure-resources-extension-tools/*, Copilot Azure Resources Extension Tools/*, tool_search, toolSearch, execute, read, agent, browser, edit, search, web, azure-mcp/search, Azure MCP/*, todo]
 ---
 
 <!-- azure-cor-disclaimer -->