scanner

by google-gemini

Scans a website deeply, converting HTML pages to markdown, respecting robots.txt, and updating the snapshots log.

npx skills add https://github.com/google-gemini/gemini-managed-agents-templates --skill scanner

Download ZIP GitHub

Scanner Skill

Use this skill to scan and analyze all relevant pages under a target website domain to build or refresh a local customer support corpus.

Embedded Script

python skills/scanner/scripts/scan.py <URL> [--force]

Arguments

Argument	Description
`<URL>`	The start/seed URL of the website to analyze (e.g. `https://example.com`)
`--force`	Force scanning and bypass the 24-hour cache check

Features

Robots.txt Compliance: Checks robots.txt before parsing. If restricted, aborts scanning. Make sure to perform scanning only on sites that are allowed.
Domain-Locked Recursive Scanning: Only analyzes links within the same domain/subdomain to avoid leaking to other websites.
HTML to Markdown: Converts HTML structure into clean, readable Markdown text suitable for LLM document matching.
Caching & Snapshot Maintenance: Creates or updates .agents/workspace/snapshots.json with mapping and timestamp.
Corpus Directory Index: Automatically generates .agents/workspace/pages/index.md listing and explaining all files in a structured, clickable table format so agents can locate matching topics immediately.

More skills from google-gemini

A friendly greeter skill

Automated code review for local changes and remote pull requests with structured analysis across correctness, maintainability, and security. Supports both local file system changes (staged and unstaged) and remote PRs (by number or URL) with automatic GitHub CLI checkout Analyzes code across seven dimensions: correctness, maintainability, readability, efficiency, security, edge case handling, and test coverage Runs optional preflight verification suites (e.g., npm run preflight ) to catch...

review-duplication

Use this skill during code reviews to proactively investigate the codebase for duplicated functionality, reinvented wheels, or failure to reuse existing…

Reconcile loaded expenses against the pre-parsed invoice database, flagging discrepancies like amount mismatches, missing invoices, and merchant mismatches…

Guide for using the Gemini API CLI tool. Use when you need to interact with the Gemini API via the command line, manage agents, or generate media (images,…

Main Agents: Do NOT use this skill directly. If you need to test the TUI, invoke the `tui_tester` subagent. Drive terminal UI (TUI) applications…

async-pr-review

Trigger this skill when the user wants to start an asynchronous PR review, run background checks on a PR, or check the status of a previously started async PR…

behavioral-evals

Guidance for creating, running, fixing, and promoting behavioral evaluations. Use when verifying agent decision logic, debugging failures, debugging prompt…