firecrawl-scrape

作者: firecrawl

从任何URL提取干净的Markdown内容,包括JavaScript渲染的单页应用。当用户提供URL并希望获取其内容,或提到“scrape”时,使用此技能。

npx skills add https://github.com/firecrawl/firecrawl-codex-plugin --skill firecrawl-scrape

firecrawl scrape

Scrape one or more URLs. Returns clean, LLM-optimized markdown. Multiple URLs are scraped concurrently.

When to use

  • You have a specific URL and want its content
  • The page is static or JS-rendered (SPA)
  • Step 2 in the workflow escalation pattern: search → scrape → map → crawl → interact

Quick start

# Basic markdown extraction
firecrawl scrape "<url>" -o .firecrawl/page.md

# Main content only, no nav/footer
firecrawl scrape "<url>" --only-main-content -o .firecrawl/page.md

# Wait for JS to render, then scrape
firecrawl scrape "<url>" --wait-for 3000 -o .firecrawl/page.md

# Multiple URLs (each saved to .firecrawl/)
firecrawl scrape https://example.com https://example.com/blog https://example.com/docs

# Get markdown and links together
firecrawl scrape "<url>" --format markdown,links -o .firecrawl/page.json

# Ask a question about the page
firecrawl scrape "https://example.com/pricing" --query "What is the enterprise plan price?"

Options

OptionDescription
-f, --format <formats>Output formats: markdown, html, rawHtml, links, screenshot, json
-Q, --query <prompt>Ask a question about the page content (5 credits)
-HInclude HTTP headers in output
--only-main-contentStrip nav, footer, sidebar — main content only
--wait-for <ms>Wait for JS rendering before scraping
--include-tags <tags>Only include these HTML tags
--exclude-tags <tags>Exclude these HTML tags
-o, --output <path>Output file path

Tips

  • Prefer plain scrape over --query. Scrape to a file, then use grep, head, or read the markdown directly — you can search and reason over the full content yourself. Use --query only when you want a single targeted answer without saving the page (costs 5 extra credits).
  • Try scrape before interact. Scrape handles static pages and JS-rendered SPAs. Only escalate to interact when you need interaction (clicks, form fills, pagination).
  • Multiple URLs are scraped concurrently — check firecrawl --status for your concurrency limit.
  • Single format outputs raw content. Multiple formats (e.g., --format markdown,links) output JSON.
  • Always quote URLs — shell interprets ? and & as special characters.
  • Naming convention: .firecrawl/{site}-{path}.md

See also

来自 firecrawl 的更多技能

firecrawl-research-index
firecrawl
使用 Firecrawl Research 查找回答研究查询的论文,采用语义搜索、语义与结构扩展以及正文内验证。对于任何文献查找/论文检索任务——无论是单篇论文查找还是完整的多篇论文集合——始终使用此技能。
data-analysisresearchweb-scraping
oracle
firecrawl
使用oracle CLI的最佳实践(提示词与文件打包、引擎、会话及文件附件模式)。
pinecone
firecrawl
面向生产级AI应用的托管向量数据库。全托管、自动扩缩容,支持混合搜索(稠密+稀疏)、元数据过滤和命名空间。
wpds
firecrawl
在构建利用WordPress设计系统(WPDS)及其组件、令牌、模式等的用户界面时使用。
audiocraft-audio-generation
firecrawl
用于音频生成的PyTorch库,包括文本生成音乐(MusicGen)和文本生成声音(AudioGen)。当需要从文本生成音乐时使用…
skypilot-multi-cloud-orchestration
firecrawl
跨多云编排机器学习工作负载,自动优化成本。当您需要在多个云上运行训练或批处理作业时使用,利用…
firecrawl-seo-audit
firecrawl
使用 Firecrawl 对网站进行 SEO 审计。适用于用户要求进行 SEO 审计、元数据和标题审查、站点地图/网站结构分析、关键词机会、竞争对手 SERP 对比,或优先搜索优化建议的场景。
data-analysisresearchweb-scraping
gh-issues
firecrawl
获取GitHub议题,生成子代理来实施修复并开启拉取请求,然后监控并处理PR审查评论。用法:/gh-issues [owner/repo] [--label…]