From fc4a6a9155fc73f1c8f8dacf1b8c3c682610f847 Mon Sep 17 00:00:00 2001 From: TTTT Date: Sun, 30 Aug 2026 08:36:03 +0800 Subject: [PATCH] feat(skills): support MCP-first execution and clarify CLI fallback hierarchy - Add MCP tool mappings and execution hierarchy across core skill definitions - Remove restrictive allowed-tools bash frontmatter to enable direct MCP invocation - Clarify that missing local CLI does not indicate Firecrawl unavailability - Forbid unauthorized automatic npx package installation in headless agent runs --- skills/firecrawl-agent/SKILL.md | 11 ++++++--- skills/firecrawl-crawl/SKILL.md | 11 ++++++--- skills/firecrawl-download/SKILL.md | 9 ++++--- skills/firecrawl-interact/SKILL.md | 11 ++++++--- skills/firecrawl-map/SKILL.md | 10 +++++--- skills/firecrawl-monitor/SKILL.md | 11 ++++++--- skills/firecrawl-parse/SKILL.md | 10 +++++--- skills/firecrawl-scrape/SKILL.md | 10 +++++--- skills/firecrawl-search/SKILL.md | 11 ++++++--- skills/firecrawl/SKILL.md | 39 ++++++++++++++++++------------ skills/firecrawl/rules/install.md | 14 ++++++++--- 11 files changed, 100 insertions(+), 47 deletions(-) diff --git a/skills/firecrawl-agent/SKILL.md b/skills/firecrawl-agent/SKILL.md index 63944396cc..d9a95ae66f 100644 --- a/skills/firecrawl-agent/SKILL.md +++ b/skills/firecrawl-agent/SKILL.md @@ -2,15 +2,20 @@ name: firecrawl-agent description: | Autonomous multi-page extraction into structured JSON. Use when the user wants website data matching a schema — pricing tiers, product listings — beyond a single-page scrape. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl agent AI-powered autonomous extraction. The agent navigates sites and extracts structured data (takes 2-5 minutes). +## Tool Choice & Quick Start + +- MCP: **`firecrawl_agent(prompt, urls?, schema?)`** + Status polling: **`firecrawl_agent_status(id)`** + Call directly when Firecrawl MCP tools are exposed. +- CLI: **`firecrawl agent [--urls ] [--schema ] [--wait] [-o ]`** + Fallback when running in a CLI environment. + ## Quick start ```bash diff --git a/skills/firecrawl-crawl/SKILL.md b/skills/firecrawl-crawl/SKILL.md index e2286572f6..59bd2966bc 100644 --- a/skills/firecrawl-crawl/SKILL.md +++ b/skills/firecrawl-crawl/SKILL.md @@ -2,9 +2,6 @@ name: firecrawl-crawl description: | Bulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl crawl @@ -13,6 +10,14 @@ Bulk extract content from a website. Crawls pages following links up to a depth/ **Prerequisite:** `crawl` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login. +## Tool Choice & Quick Start + +- MCP: **`firecrawl_crawl(url, includePaths?, excludePaths?, limit?, maxDiscoveryDepth?, ...)`** + Check status: **`firecrawl_check_crawl_status(id)`** + Call directly when Firecrawl MCP tools are exposed. +- CLI: **`firecrawl crawl [--include-paths ] [--limit ] [--wait] [-o ]`** + Fallback when running in a CLI environment. + ## Quick start ```bash diff --git a/skills/firecrawl-download/SKILL.md b/skills/firecrawl-download/SKILL.md index 7947e2b9a0..d14ab01c65 100644 --- a/skills/firecrawl-download/SKILL.md +++ b/skills/firecrawl-download/SKILL.md @@ -2,9 +2,6 @@ name: firecrawl-download description: | Save a site or section as local files (markdown, screenshots). Use for "download the site", offline docs, or a local copy for reference. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl download (invoked as `firecrawl x download`) @@ -13,6 +10,12 @@ allowed-tools: **Prerequisite:** `download` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login. +## Tool Choice & Quick Start + +- MCP: Use **`firecrawl_map(url, ...)`** to discover pages and **`firecrawl_scrape(url, ...)`** to extract and save page contents. +- CLI: **`firecrawl x download [--include-paths ] [--limit ] -y`** + Command line batch download directly to `.firecrawl/`. + Maps the site origin first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. Use `--include-paths` to scope a non-root URL to one section. Automated runs always pass `-y` — without it the command opens an interactive wizard that blocks on a prompt. ## Quick start diff --git a/skills/firecrawl-interact/SKILL.md b/skills/firecrawl-interact/SKILL.md index 0250e793d3..0da89decf5 100644 --- a/skills/firecrawl-interact/SKILL.md +++ b/skills/firecrawl-interact/SKILL.md @@ -2,15 +2,20 @@ name: firecrawl-interact description: | Drive a live browser on a scraped page: click, fill forms, log in, paginate, infinite-scroll. Use when content requires interaction or a scrape failed or returned incomplete content. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl interact Interact with scraped pages in a live browser session. Scrape a page first, then use natural language prompts or code to click, fill forms, navigate, and extract data. For web searches, use `search` — interact is for acting on a specific page. +## Tool Choice & Quick Start + +- MCP: **`firecrawl_interact(url?, scrapeId?, prompt?, code?, ...)`** + Stop session: **`firecrawl_interact_stop(scrapeId)`** + Call directly when Firecrawl MCP tools are exposed. +- CLI: **`firecrawl interact [scrapeId] [--code ]`** + Fallback when running in a CLI environment. + ## Quick start ```bash diff --git a/skills/firecrawl-map/SKILL.md b/skills/firecrawl-map/SKILL.md index 4742ccad26..b005256d95 100644 --- a/skills/firecrawl-map/SKILL.md +++ b/skills/firecrawl-map/SKILL.md @@ -2,9 +2,6 @@ name: firecrawl-map description: | Discover and list a site's URLs, with search filtering. Use for "map the site" or "find the URL for" requests — when the user knows the site but not the exact page, or wants site structure. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl map @@ -13,6 +10,13 @@ Discover URLs on a site. Use `--search` to find a specific page within a large s **Prerequisite:** `map` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login. +## Tool Choice & Quick Start + +- MCP: **`firecrawl_map(url, search?, limit?, ...)`** + Call directly when Firecrawl MCP tools are exposed. +- CLI: **`firecrawl map [--search ] [--limit ] [-o ]`** + Fallback when running in a CLI environment. + ## Quick start ```bash diff --git a/skills/firecrawl-monitor/SKILL.md b/skills/firecrawl-monitor/SKILL.md index a81eba7e21..1dbd1f30d0 100644 --- a/skills/firecrawl-monitor/SKILL.md +++ b/skills/firecrawl-monitor/SKILL.md @@ -2,15 +2,20 @@ name: firecrawl-monitor description: | Alert by webhook/email on web changes — use for "monitor/watch/track/alert me when": recurring checks on known URLs (prefer over repeated one-off scrapes) or web-wide watches for new results (queries + goal). -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl monitor Detect when content on a website changes and get notified by webhook or email. Firecrawl handles fetching, diffing, judging, and notifying server-side. Each page in a check is labeled `same`, `new`, `changed`, `removed`, or `error`. +## Tool Choice & Quick Start + +- MCP: **`firecrawl_monitor_create(goal?, page?, pages?, queries?, scheduleText?, ...)`** + Manage & check: **`firecrawl_monitor_get`**, **`firecrawl_monitor_run`**, **`firecrawl_monitor_check`**, **`firecrawl_monitor_checks`**, **`firecrawl_monitor_update`**, **`firecrawl_monitor_delete`**, **`firecrawl_monitor_list`** + Call directly when Firecrawl MCP tools are exposed. +- CLI: **`firecrawl monitor `** + Fallback when running in a CLI environment. + **Pick a target mode** by what you're watching: | Mode | Flags | Watches | diff --git a/skills/firecrawl-parse/SKILL.md b/skills/firecrawl-parse/SKILL.md index 849448c1b8..1081f10d89 100644 --- a/skills/firecrawl-parse/SKILL.md +++ b/skills/firecrawl-parse/SKILL.md @@ -2,15 +2,19 @@ name: firecrawl-parse description: | Convert a local file (PDF, DOCX, XLSX, HTML, …) to markdown, or answer questions about its content. Use whenever the input is a file path, not a URL. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl parse Turn a local document into clean markdown on disk. Supports **PDF, DOCX, DOC, ODT, RTF, XLSX, XLS, HTML/HTM**. +## Tool Choice & Quick Start + +- MCP: **`firecrawl_parse(filePath?, formats?, ...)`** + Call directly when Firecrawl MCP tools are exposed. +- CLI: **`firecrawl parse [-o ]`** + Fallback when running in a CLI environment. + ## Quick start Always save to `.firecrawl/` with `-o` — parsed docs can be hundreds of KB and blow up context if streamed to stdout. Add `.firecrawl/` to `.gitignore`. diff --git a/skills/firecrawl-scrape/SKILL.md b/skills/firecrawl-scrape/SKILL.md index 7211eebb1b..b2f7f61130 100644 --- a/skills/firecrawl-scrape/SKILL.md +++ b/skills/firecrawl-scrape/SKILL.md @@ -2,15 +2,19 @@ name: firecrawl-scrape description: | Extract a URL's content as clean markdown, including JS-rendered pages. Use whenever the user provides a URL and wants its content; prefer over WebFetch. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl scrape Scrape one or more URLs. Returns clean, LLM-optimized markdown. Multiple URLs are scraped concurrently. +## Tool Choice & Quick Start + +- MCP: **`firecrawl_scrape(url, formats?, onlyMainContent?, waitFor?, ...)`** + Call directly when Firecrawl MCP tools are exposed. +- CLI: **`firecrawl scrape [--only-main-content] [--wait-for ] [-o ]`** + Fallback when running in a CLI environment. + ## Quick start ```bash diff --git a/skills/firecrawl-search/SKILL.md b/skills/firecrawl-search/SKILL.md index c3b4334878..4f589f8288 100644 --- a/skills/firecrawl-search/SKILL.md +++ b/skills/firecrawl-search/SKILL.md @@ -2,15 +2,20 @@ name: firecrawl-search description: | Web search with full page content. Use when no URL is known: finding sources, articles, or news. For papers use firecrawl-research-index; for library, API, error, or bug questions use firecrawl-developer-index. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # firecrawl search Web search with optional content scraping. Returns search results as JSON, optionally with full page content. +## Tool Choice & Quick Start + +- MCP: **`firecrawl_search(query, scrapeOptions?, categories?, limit?, sources?, ...)`** + Call directly when Firecrawl MCP tools are exposed. + Feedback: **`firecrawl_search_feedback(searchId, rating, ...)`** +- CLI: **`firecrawl search [--scrape] [--json] [-o ]`** + Fallback when running in a CLI environment. + ## Quick start ```bash diff --git a/skills/firecrawl/SKILL.md b/skills/firecrawl/SKILL.md index e38876f267..e96664dd46 100644 --- a/skills/firecrawl/SKILL.md +++ b/skills/firecrawl/SKILL.md @@ -2,13 +2,20 @@ name: firecrawl description: | Any live-web task via the Firecrawl CLI — including ordinary web research: searching the web, reading or extracting pages, gathering sources, discovering site URLs, bulk extraction, downloading a site, change alerts, or pages needing clicks/login — web only; local files route to firecrawl-parse. For papers use firecrawl-research-index; for library, API, error, or bug questions use firecrawl-developer-index. -allowed-tools: - - Bash(firecrawl *) - - Bash(npx firecrawl-cli *) --- # Firecrawl CLI +## Tool Selection Hierarchy + +Follow this precedence order based on your current execution environment: + +1. **MCP First (Recommended)**: If Firecrawl MCP tools (`firecrawl_search`, `firecrawl_scrape`, `firecrawl_map`, `firecrawl_crawl`, `firecrawl_agent`, `firecrawl_interact`, `firecrawl_parse`, `firecrawl_monitor_*`, `firecrawl_developer_search`, `firecrawl_research_*`) are available in the tool catalog, **call the MCP tools directly**. +2. **CLI Fallback**: If MCP tools are not exposed, check if the `firecrawl` CLI is installed on `PATH` and run CLI commands. +3. **Missing Local CLI != Unavailable**: If `firecrawl` command is not found on `PATH`, do **not** declare Firecrawl unavailable without checking for exposed MCP tools first. **Do not run unauthorized `npx` commands or install packages silently**. + +--- + Search, scrape, and interact with the web. Returns clean markdown optimized for LLM context windows. Run `firecrawl --help` or `firecrawl --help` for full option details. For app integration or outcome workflows (research briefs, SEO audits, etc.), route to the `firecrawl-build` / `firecrawl-workflows` skills — see [When to Load References](#when-to-load-references). @@ -30,19 +37,19 @@ Follow this escalation pattern: 5. **Monitor** - Need recurring checks or ongoing alerts. Prefer setting a monitor with `--page` plus `--goal` instead of doing repeated one-off scrapes. 6. **Interact** - Scrape first, then interact with the page (pagination, modals, form submissions, multi-step navigation). -| Need | Command | When | -| --------------------------- | --------------------- | --------------------------------------------------------------- | -| Find pages on a topic | `search` | No specific URL yet | -| Find research papers | `research` | Biomedical/clinical/scientific literature — use the paper index | -| Answer a coding question | `developer` | Issues, merged PRs, READMEs, and docs — not a general web page | -| Get a page's content | `scrape` | Have a URL, page is static or JS-rendered | -| Find URLs within a site | `map` | Need to locate a specific subpage | -| Bulk extract a site section | `crawl` | Need many pages (e.g., all /docs/) | -| AI-powered data extraction | `agent` | Need structured data from complex sites | -| Interact with a page | `scrape` + `interact` | Content requires clicks, form fills, pagination, or login | -| Download a site to files | `x download` | Save an entire site as local files | -| Parse a local file | `parse` | File on disk (PDF, DOCX, XLSX, etc.) — not a URL | -| Watch pages for changes | `monitor` | Schedule recurring scrapes/crawls, diff against snapshots | +| Need | MCP Tool | CLI Command | When | +| --------------------------- | ------------------------------------ | --------------------- | --------------------------------------------------------------- | +| Find pages on a topic | `firecrawl_search` | `search` | No specific URL yet | +| Find research papers | `firecrawl_research_search_papers` | `research` | Biomedical/clinical/scientific literature — use the paper index | +| Answer a coding question | `firecrawl_developer_search` | `developer` | Issues, merged PRs, READMEs, and docs — not a general web page | +| Get a page's content | `firecrawl_scrape` | `scrape` | Have a URL, page is static or JS-rendered | +| Find URLs within a site | `firecrawl_map` | `map` | Need to locate a specific subpage | +| Bulk extract a site section | `firecrawl_crawl` | `crawl` | Need many pages (e.g., all /docs/) | +| AI-powered data extraction | `firecrawl_agent` | `agent` | Need structured data from complex sites | +| Interact with a page | `firecrawl_interact` | `scrape` + `interact` | Content requires clicks, form fills, pagination, or login | +| Download a site to files | `firecrawl_map` + `firecrawl_scrape` | `x download` | Save an entire site as local files | +| Parse a local file | `firecrawl_parse` | `parse` | File on disk (PDF, DOCX, XLSX, etc.) — not a URL | +| Watch pages for changes | `firecrawl_monitor_create` | `monitor` | Schedule recurring scrapes/crawls, diff against snapshots | For detailed command reference, run `firecrawl --help`. diff --git a/skills/firecrawl/rules/install.md b/skills/firecrawl/rules/install.md index 5142afb949..d5d88d7579 100644 --- a/skills/firecrawl/rules/install.md +++ b/skills/firecrawl/rules/install.md @@ -9,6 +9,12 @@ description: | # Firecrawl CLI Installation +## Important: Tool Availability & Execution Hierarchy + +1. **Check for Native MCP Tools First**: Before attempting to install or check the CLI, verify whether Firecrawl MCP tools (e.g. `firecrawl_search`, `firecrawl_scrape`, `firecrawl_crawl`, `firecrawl_map`, etc.) are exposed in your current agent environment. If MCP tools are available, **use them directly** without installing the CLI. +2. **Missing Local CLI != Firecrawl Unavailable**: If `firecrawl` command is not found on `PATH`, this only means the CLI executable is absent locally; it does not mean Firecrawl functionality is unavailable. Check for MCP tools before reporting failure. +3. **No Unauthorized Package Installation**: Agents must **not** execute `npx firecrawl-cli init` or install npm packages unless explicitly requested by the user. + ## Quick Setup (Recommended) ```bash @@ -84,8 +90,8 @@ If you cannot obtain a key and the user cannot sign up, search, scrape, and inte ### Command not found -If `firecrawl` is not found after installation: +If `firecrawl` is not found: -1. Ensure npm global bin is in PATH -2. Try: `npx firecrawl-cli@latest --version` -3. Reinstall: `npm install -g firecrawl-cli@latest` +1. Check if Firecrawl MCP tools are exposed in the agent environment — if so, use them directly instead of troubleshooting CLI. +2. If CLI is explicitly needed, ensure npm global bin directory is included in `PATH`. +3. If CLI installation was requested by the user, install via `npm install -g firecrawl-cli@latest`.