agent-browser

作者: 101-skills

Browser automation for AI agents via inference.sh. Navigate web pages, interact with elements using @e refs, take screenshots, record video. Capabilities: web scraping, form filling, clicking, typing, drag-drop, file upload, JavaScript execution. Use for: web automation, data extraction, testing, agent browsing, research. Triggers: browser, web automation, scrape, navigate, click, fill form, screenshot, browse web, playwright, headless browser, web agent, surf internet, record video

npx skills add https://github.com/101-skills/superpowers --skill agent-browser

Install the belt CLI skill: npx skills add belt-sh/cli

Agentic Browser

Browser automation for AI agents via inference.sh. Uses Playwright under the hood with a simple @e ref system for element interaction.

Agentic Browser

Quick Start

Requires inference.sh CLI (belt). Install instructions

belt login

# Open a page and get interactive elements
belt app run agent-browser --function open --input '{"url": "https://example.com"}' --session new

Core Workflow

Every browser automation follows this pattern:

  1. Open - Navigate to URL, get @e refs for elements
  2. Interact - Use refs to click, fill, drag, etc.
  3. Re-snapshot - After navigation/changes, get fresh refs
  4. Close - End session (returns video if recording)
# 1. Start session
RESULT=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/login"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')
# Elements: @e1 [input] "Email", @e2 [input] "Password", @e3 [button] "Sign In"

# 2. Fill and submit
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e1", "text": "user@example.com"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e2", "text": "password123"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click", "ref": "@e3"
}'

# 3. Re-snapshot after navigation
belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}'

# 4. Close when done
belt app run agent-browser --function close --session $SESSION_ID --input '{}'

Functions

FunctionDescription
openNavigate to URL, configure browser (viewport, proxy, video recording)
snapshotRe-fetch page state with @e refs after DOM changes
interactPerform actions using @e refs (click, fill, drag, upload, etc.)
screenshotTake page screenshot (viewport or full page)
executeRun JavaScript code on the page
closeClose session, returns video if recording was enabled

Interact Actions

ActionDescriptionRequired Fields
clickClick elementref
dblclickDouble-click elementref
fillClear and type textref, text
typeType text (no clear)text
pressPress key (Enter, Tab, etc.)text
selectSelect dropdown optionref, text
hoverHover over elementref
checkCheck checkboxref
uncheckUncheck checkboxref
dragDrag and dropref, target_ref
uploadUpload file(s)ref, file_paths
scrollScroll pagedirection (up/down/left/right), scroll_amount
backGo back in history-
waitWait millisecondswait_ms
gotoNavigate to URLurl

Element Refs

Elements are returned with @e refs:

@e1 [a] "Home" href="/"
@e2 [input type="text"] placeholder="Search"
@e3 [button] "Submit"
@e4 [select] "Choose option"
@e5 [input type="checkbox"] name="agree"

Important: Refs are invalidated after navigation. Always re-snapshot after:

  • Clicking links/buttons that navigate
  • Form submissions
  • Dynamic content loading

Features

Video Recording

Record browser sessions for debugging or documentation:

# Start with recording enabled (optionally show cursor indicator)
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true,
  "show_cursor": true
}' | jq -r '.session_id')

# ... perform actions ...

# Close to get the video file
belt app run agent-browser --function close --session $SESSION --input '{}'
# Returns: {"success": true, "video": <File>}

Cursor Indicator

Show a visible cursor in screenshots and video (useful for demos):

belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "show_cursor": true,
  "record_video": true
}'

The cursor appears as a red dot that follows mouse movements and shows click feedback.

Proxy Support

Route traffic through a proxy server:

belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "user",
  "proxy_password": "pass"
}'

File Upload

Upload files to file inputs:

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "upload",
  "ref": "@e5",
  "file_paths": ["/path/to/file.pdf"]
}'

Drag and Drop

Drag elements to targets:

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "drag",
  "ref": "@e1",
  "target_ref": "@e2"
}'

JavaScript Execution

Run custom JavaScript:

belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.querySelectorAll(\"h2\").length"
}'
# Returns: {"result": "5", "screenshot": <File>}

Deep-Dive Documentation

ReferenceDescription
references/commands.mdFull function reference with all options
references/snapshot-refs.mdRef lifecycle, invalidation rules, troubleshooting
references/session-management.mdSession persistence, parallel sessions
references/authentication.mdLogin flows, OAuth, 2FA handling
references/video-recording.mdRecording workflows for debugging
references/proxy-support.mdProxy configuration, geo-testing

Ready-to-Use Templates

TemplateDescription
templates/form-automation.shForm filling with validation
templates/authenticated-session.shLogin once, reuse session
templates/capture-workflow.shContent extraction with screenshots

Examples

Form Submission

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/contact"
}' | jq -r '.session_id')

# Get elements: @e1 [input] "Name", @e2 [input] "Email", @e3 [textarea], @e4 [button] "Send"

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "John Doe"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e2", "text": "john@example.com"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e3", "text": "Hello!"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "click", "ref": "@e4"}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'

Search and Extract

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://google.com"
}' | jq -r '.session_id')

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "weather today"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "press", "text": "Enter"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "wait", "wait_ms": 2000}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'

Screenshot with Video

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true
}' | jq -r '.session_id')

# Take full page screenshot
belt app run agent-browser --function screenshot --session $SESSION --input '{
  "full_page": true
}'

# Close and get video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
echo $RESULT | jq '.video'

Sessions

Browser state persists within a session. Always:

  1. Start with --session new on first call
  2. Use returned session_id for subsequent calls
  3. Close session when done

Related Skills

# Web search (for research + browse)
npx skills add inference-sh/skills@web-search

# LLM models (analyze extracted content)
npx skills add inference-sh/skills@llm-models

Documentation

來自 101-skills 的更多技能

agent-browser
101-skills
透過 inference.sh 為 AI 代理提供瀏覽器自動化功能。可導覽網頁、使用 @e 參考與元素互動、擷取螢幕畫面、錄製影片。功能包括:網頁抓取、表單填寫、點擊、打字、拖放、檔案上傳、執行 JavaScript。適用於:網頁自動化、資料擷取、測試、代理瀏覽、研究。觸發詞:瀏覽器、網頁自動化、抓取、導覽、點擊、填寫表單、螢幕截圖、瀏覽網頁、playwright、無頭瀏覽器、網頁代理、上網、錄製影片
browser-automationweb-scrapingtesting
agent-tools
101-skills
透過 inference.sh CLI 執行 AI 應用程式 - 圖片生成、影片創作、大型語言模型、搜尋、3D、Twitter 自動化。模型:FLUX、Veo、Gemini、Grok、Claude、Seedance、OmniHuman、Tavily、Exa、OpenRouter 等。適用於執行 AI 應用程式、生成圖片/影片、呼叫大型語言模型、網路搜尋或自動化 Twitter 時使用。觸發詞:inference.sh、infsh、ai model、run ai、serverless ai、ai api、flux、veo、claude api、image generation、video generation、openrouter、tavily、exa search、twitter api、grok
agent-tools
101-skills
透過 inference.sh CLI 執行 AI 應用程式——圖像生成、影片創作、LLM、搜尋、3D、Twitter 自動化。模型:FLUX、Veo、Gemini、Grok、Claude、Seedance、OmniHuman、Tavily、Exa 等眾多模型。適用於執行 AI 應用程式、生成圖像/影片、呼叫 LLM、網頁搜尋或自動化 Twitter。觸發詞:inference.sh、infsh、AI 模型、執行 AI、無伺服器 AI、AI API、flux、veo、claude API、圖像生成、影片生成、tavily、exa 搜尋、twitter API、grok
apiimagevideo
ai-avatar-video
101-skills
透過 inference.sh CLI 建立 AI 虛擬角色與說話頭像影片。推薦:P-Video-Avatar(最快、最便宜、內建 TTS)。另可選:OmniHuman、Fabric、PixVerse。音訊:Inworld TTS-2(100 多種語言、角色情感引導)、ElevenLabs、Kokoro。功能:音訊驅動虛擬角色、文字轉虛擬角色、唇形同步影片、說話頭像生成、虛擬主持人、UGC 內容。適用於:AI 主持人、解說影片、虛擬網紅、配音、行銷影片、UGC 廣告、遊戲虛擬角色等。
creativevideomedia
ai-avatar-video
101-skills
Create AI avatar and talking head videos via inference.sh CLI. Recommended: P-Video-Avatar (fastest, cheapest, built-in TTS). Also: OmniHuman, Fabric, PixVerse. Audio: Inworld TTS-2 (100+ languages, emotion steering for characters), ElevenLabs, Kokoro. Capabilities: audio-driven avatars, text-to-avatar, lipsync videos, talking head generation, virtual presenters, UGC content. Use for: AI presenters, explainer videos, virtual influencers, dubbing, marketing videos, UGC ads, gaming avatars,...
videomediamarketing
ai-image-generation
101-skills
透過 inference.sh CLI 使用 GPT-Image-2、FLUX、Gemini、Grok、Seedream、Reve 及 50 多種模型生成 AI 圖像。模型包括:GPT-Image-2、FLUX Dev LoRA、FLUX.2 Klein LoRA、Gemini 3 Pro Image、Grok Imagine、Seedream 4.5、Reve、ImagineArt。功能涵蓋:文字轉圖像、圖像轉圖像、修補、LoRA、圖像編輯、放大、文字渲染。適用於:AI 藝術、產品模型、概念藝術、社交媒體圖形、行銷視覺、插圖。觸發詞:flux、圖像生成、AI 圖像、文字轉...
creativemediaimage
ai-image-generation
101-skills
Generate AI images with GPT-Image-2.6, FLUX, Gemini, Grok, Seedream, Reve and 50+ models via inference.sh CLI. Models: GPT-Image-2.6 Flare, GPT-Image-2.6 Sunburst, FLUX Dev LoRA, FLUX.2 Klein LoRA, Nano Banana 2.1, Gemini 3 Pro Image, Grok Imagine 2.0, Seedream 5.5, Reve, ImagineArt. Capabilities: text-to-image, image-to-image, inpainting, LoRA, image editing, upscaling, text rendering. Use for: AI art, product mockups, concept art, social media graphics, marketing visuals, illustrations....
imagecreativemedia
ai-video-generation
101-skills
透過 inference.sh CLI 使用 Google Veo、Seedance 2.0、HappyHorse、Wan、Grok 及 40 多種模型生成 AI 影片。模型包括:Veo 3.1、Veo 3、Seedance 2.0、HappyHorse 1.0、Wan 2.5、Grok Imagine Video、OmniHuman、Fabric、HunyuanVideo。功能涵蓋:文字轉影片、圖片轉影片、參考素材轉影片、影片編輯、唇形同步、虛擬角色動畫、影片畫質提升、擬音音效。適用於:社群媒體影片、行銷內容、解說影片、產品展示、AI 虛擬角色。觸發條件:影片生成、AI 影片...
creativevideomedia