agent-browser

作者: halt-catch-fire

透過 inference.sh 為 AI 代理提供瀏覽器自動化功能。可導覽網頁、使用 @e 參考與元素互動、擷取螢幕截圖、錄製影片。功能包括:網頁抓取、表單填寫、點擊、打字、拖放、檔案上傳、執行 JavaScript。適用於:網頁自動化、資料擷取、測試、代理瀏覽、研究。觸發詞:瀏覽器、網頁自動化、抓取、導覽、點擊、填寫表單、螢幕截圖、瀏覽網頁、playwright、無頭瀏覽器、網頁代理、上網、錄製影片

npx skills add https://github.com/halt-catch-fire/skills --skill agent-browser

Install the belt CLI skill: npx skills add belt-sh/cli

Agentic Browser

Browser automation for AI agents via inference.sh. Uses Playwright under the hood with a simple @e ref system for element interaction.

Agentic Browser

Quick Start

Requires inference.sh CLI (belt). Install instructions

belt login

# Open a page and get interactive elements
belt app run agent-browser --function open --input '{"url": "https://example.com"}' --session new

Core Workflow

Every browser automation follows this pattern:

  1. Open - Navigate to URL, get @e refs for elements
  2. Interact - Use refs to click, fill, drag, etc.
  3. Re-snapshot - After navigation/changes, get fresh refs
  4. Close - End session (returns video if recording)
# 1. Start session
RESULT=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/login"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')
# Elements: @e1 [input] "Email", @e2 [input] "Password", @e3 [button] "Sign In"

# 2. Fill and submit
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e1", "text": "user@example.com"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e2", "text": "password123"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click", "ref": "@e3"
}'

# 3. Re-snapshot after navigation
belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}'

# 4. Close when done
belt app run agent-browser --function close --session $SESSION_ID --input '{}'

Functions

FunctionDescription
openNavigate to URL, configure browser (viewport, proxy, video recording)
snapshotRe-fetch page state with @e refs after DOM changes
interactPerform actions using @e refs (click, fill, drag, upload, etc.)
screenshotTake page screenshot (viewport or full page)
executeRun JavaScript code on the page
closeClose session, returns video if recording was enabled

Interact Actions

ActionDescriptionRequired Fields
clickClick elementref
dblclickDouble-click elementref
fillClear and type textref, text
typeType text (no clear)text
pressPress key (Enter, Tab, etc.)text
selectSelect dropdown optionref, text
hoverHover over elementref
checkCheck checkboxref
uncheckUncheck checkboxref
dragDrag and dropref, target_ref
uploadUpload file(s)ref, file_paths
scrollScroll pagedirection (up/down/left/right), scroll_amount
backGo back in history-
waitWait millisecondswait_ms
gotoNavigate to URLurl

Element Refs

Elements are returned with @e refs:

@e1 [a] "Home" href="/"
@e2 [input type="text"] placeholder="Search"
@e3 [button] "Submit"
@e4 [select] "Choose option"
@e5 [input type="checkbox"] name="agree"

Important: Refs are invalidated after navigation. Always re-snapshot after:

  • Clicking links/buttons that navigate
  • Form submissions
  • Dynamic content loading

Features

Video Recording

Record browser sessions for debugging or documentation:

# Start with recording enabled (optionally show cursor indicator)
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true,
  "show_cursor": true
}' | jq -r '.session_id')

# ... perform actions ...

# Close to get the video file
belt app run agent-browser --function close --session $SESSION --input '{}'
# Returns: {"success": true, "video": <File>}

Cursor Indicator

Show a visible cursor in screenshots and video (useful for demos):

belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "show_cursor": true,
  "record_video": true
}'

The cursor appears as a red dot that follows mouse movements and shows click feedback.

Proxy Support

Route traffic through a proxy server:

belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "user",
  "proxy_password": "pass"
}'

File Upload

Upload files to file inputs:

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "upload",
  "ref": "@e5",
  "file_paths": ["/path/to/file.pdf"]
}'

Drag and Drop

Drag elements to targets:

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "drag",
  "ref": "@e1",
  "target_ref": "@e2"
}'

JavaScript Execution

Run custom JavaScript:

belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.querySelectorAll(\"h2\").length"
}'
# Returns: {"result": "5", "screenshot": <File>}

Deep-Dive Documentation

ReferenceDescription
references/commands.mdFull function reference with all options
references/snapshot-refs.mdRef lifecycle, invalidation rules, troubleshooting
references/session-management.mdSession persistence, parallel sessions
references/authentication.mdLogin flows, OAuth, 2FA handling
references/video-recording.mdRecording workflows for debugging
references/proxy-support.mdProxy configuration, geo-testing

Ready-to-Use Templates

TemplateDescription
templates/form-automation.shForm filling with validation
templates/authenticated-session.shLogin once, reuse session
templates/capture-workflow.shContent extraction with screenshots

Examples

Form Submission

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/contact"
}' | jq -r '.session_id')

# Get elements: @e1 [input] "Name", @e2 [input] "Email", @e3 [textarea], @e4 [button] "Send"

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "John Doe"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e2", "text": "john@example.com"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e3", "text": "Hello!"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "click", "ref": "@e4"}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'

Search and Extract

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://google.com"
}' | jq -r '.session_id')

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "weather today"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "press", "text": "Enter"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "wait", "wait_ms": 2000}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'

Screenshot with Video

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true
}' | jq -r '.session_id')

# Take full page screenshot
belt app run agent-browser --function screenshot --session $SESSION --input '{
  "full_page": true
}'

# Close and get video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
echo $RESULT | jq '.video'

Sessions

Browser state persists within a session. Always:

  1. Start with --session new on first call
  2. Use returned session_id for subsequent calls
  3. Close session when done

Related Skills

# Web search (for research + browse)
npx skills add inference-sh/skills@web-search

# LLM models (analyze extracted content)
npx skills add inference-sh/skills@llm-models

Documentation

來自 halt-catch-fire 的更多技能

agent-tools
halt-catch-fire
透過 inference.sh CLI 執行 250 多個 AI 應用程式,包括圖片生成、影片創作、大型語言模型、搜尋、3D、Twitter 自動化。模型:FLUX、Veo、Gemini、Grok、Claude、Seedance、OmniHuman、Tavily、Exa、OpenRouter 等。適用於執行 AI 應用程式、生成圖片/影片、呼叫大型語言模型、網路搜尋或自動化 Twitter。觸發詞:inference.sh、infsh、ai model、run ai、serverless ai、ai api、flux、veo、claude api、image generation、video generation、openrouter、tavily、exa search、twitter api、grok
developmentapicreative
ai-avatar-video
halt-catch-fire
透過 inference.sh CLI 建立 AI 虛擬角色與說話頭像影片。推薦:P-Video-Avatar(最快、最便宜、內建 TTS)。另可選:OmniHuman、Fabric、PixVerse。音訊:Inworld TTS-2(支援 100 多種語言、角色情感引導)、ElevenLabs、Kokoro。功能:音訊驅動虛擬角色、文字轉虛擬角色、唇形同步影片、說話頭像生成、虛擬主持人、UGC 內容。適用於:AI 主持人、解說影片、虛擬網紅、配音、行銷影片、UGC 廣告、遊戲虛擬角色等。
videocreativemedia
ai-image-generation
halt-catch-fire
透過 inference.sh CLI 使用 GPT-Image-2、FLUX、Gemini、Grok、Seedream、Reve 及 50 多種模型生成 AI 圖像。模型包括:GPT-Image-2、FLUX Dev LoRA、FLUX.2 Klein LoRA、Gemini 3 Pro Image、Grok Imagine、Seedream 4.5、Reve、ImagineArt。功能:文字轉圖像、圖像轉圖像、修補、LoRA、圖像編輯、放大、文字渲染。適用於:AI 藝術、產品模型、概念藝術、社交媒體圖形、行銷視覺、插圖。觸發詞:flux、圖像生成、AI 圖像、文字轉...
creativemediaimage
ai-video-generation
halt-catch-fire
We need to translate the given text from English to Traditional Chinese. The instruction says to preserve product names, protocol names, URLs, numbers, and technical terms. So names like Google Veo, Seedance 2.0, HappyHorse, Wan, Grok, inference.sh CLI, Veo 3.1, etc. should remain as is. Also preserve terms like text-to-video, image-to-video, etc. but translate the surrounding Chinese text. The text inside <text> is a description of an agent skill. We need to output only the translated text, no extra labels. Let's translate paragraph by paragraph: "Generate AI videos with Google Veo, Seedance 2.0, HappyHorse, Wan, Grok and 40+ models via inference.sh CLI." -> "通過 inference.sh CLI 使用 Google Veo、Seedance 2.0、HappyHorse、Wan、Grok 及 40 多個模型生成 AI 影片。" "Models: Veo 3.1, Veo 3,
creativevideomedia
app-store-screenshots
halt-catch-fire
依據精確的平台規格製作App Store與Google Play截圖,涵蓋iOS/Android尺寸、圖庫排序、裝置模擬機及預覽影片。適用於:App Store優化、ASO、應用程式截圖、應用程式預覽、Play Store上架。觸發詞:App Store截圖、ASO、App Store優化、Play Store截圖、應用程式預覽、應用程式上架、iOS截圖、Android截圖、App Store圖片、應用程式模擬機、裝置模擬機、應用程式圖庫、商店上架
designmarketingcreative
character-design-sheet
halt-catch-fire
透過參考圖與LoRA技術,確保AI生成角色圖像的一致性。涵蓋轉面視圖、表情圖譜、色票與風格一致性技巧。適用於:角色設計、遊戲美術、插畫、動畫、漫畫、視覺小說。觸發詞:角色設計、角色圖譜、角色一致性、角色參考、轉面圖、表情圖譜、角色美術、一致角色、角色概念、參考圖、角色創作、原創角色設計...
creativedesignmedia
competitor-teardown
halt-catch-fire
結構化競爭分析,包含功能矩陣、SWOT、定位圖與UX審查。涵蓋研究框架、定價比較、評論挖掘與視覺化交付成果。適用於:市場研究、競爭情報、投資人簡報、產品策略、銷售賦能。觸發詞:競爭者分析、競爭分析、競爭者拆解、市場研究、競爭情報、SWOT分析、競爭者比較、市場格局、競爭者審查、競爭...
researchdata-analysismarketing
image-to-video
halt-catch-fire
靜態轉影片轉換指南:模型選擇、動態提示與鏡頭移動。涵蓋Wan 2.5 i2v、Seedance、Fabric、Grok Video及其適用時機。用途:為圖片添加動畫、從靜態圖像製作影片、加入動態效果、產品動畫。觸發詞:image to video、i2v、animate image、still to video、add motion to image、image animation、photo to video、animate still、wan i2v、image2video、bring image to life、animate photo、motion from image