agent-browser

作者: halt-catch-fire

通过inference.sh为AI代理提供浏览器自动化功能。可导航网页、使用@e引用与元素交互、截图、录制视频。能力包括:网页抓取、表单填写、点击、输入、拖放、文件上传、JavaScript执行。适用于:网页自动化、数据提取、测试、代理浏览、研究。触发词:浏览器、网页自动化、抓取、导航、点击、填写表单、截图、浏览网页、playwright、无头浏览器、网页代理、上网、录制视频

npx skills add https://github.com/halt-catch-fire/skills --skill agent-browser

Install the belt CLI skill: npx skills add belt-sh/cli

Agentic Browser

Browser automation for AI agents via inference.sh. Uses Playwright under the hood with a simple @e ref system for element interaction.

Agentic Browser

Quick Start

Requires inference.sh CLI (belt). Install instructions

belt login

# Open a page and get interactive elements
belt app run agent-browser --function open --input '{"url": "https://example.com"}' --session new

Core Workflow

Every browser automation follows this pattern:

  1. Open - Navigate to URL, get @e refs for elements
  2. Interact - Use refs to click, fill, drag, etc.
  3. Re-snapshot - After navigation/changes, get fresh refs
  4. Close - End session (returns video if recording)
# 1. Start session
RESULT=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/login"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')
# Elements: @e1 [input] "Email", @e2 [input] "Password", @e3 [button] "Sign In"

# 2. Fill and submit
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e1", "text": "user@example.com"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e2", "text": "password123"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click", "ref": "@e3"
}'

# 3. Re-snapshot after navigation
belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}'

# 4. Close when done
belt app run agent-browser --function close --session $SESSION_ID --input '{}'

Functions

FunctionDescription
openNavigate to URL, configure browser (viewport, proxy, video recording)
snapshotRe-fetch page state with @e refs after DOM changes
interactPerform actions using @e refs (click, fill, drag, upload, etc.)
screenshotTake page screenshot (viewport or full page)
executeRun JavaScript code on the page
closeClose session, returns video if recording was enabled

Interact Actions

ActionDescriptionRequired Fields
clickClick elementref
dblclickDouble-click elementref
fillClear and type textref, text
typeType text (no clear)text
pressPress key (Enter, Tab, etc.)text
selectSelect dropdown optionref, text
hoverHover over elementref
checkCheck checkboxref
uncheckUncheck checkboxref
dragDrag and dropref, target_ref
uploadUpload file(s)ref, file_paths
scrollScroll pagedirection (up/down/left/right), scroll_amount
backGo back in history-
waitWait millisecondswait_ms
gotoNavigate to URLurl

Element Refs

Elements are returned with @e refs:

@e1 [a] "Home" href="/"
@e2 [input type="text"] placeholder="Search"
@e3 [button] "Submit"
@e4 [select] "Choose option"
@e5 [input type="checkbox"] name="agree"

Important: Refs are invalidated after navigation. Always re-snapshot after:

  • Clicking links/buttons that navigate
  • Form submissions
  • Dynamic content loading

Features

Video Recording

Record browser sessions for debugging or documentation:

# Start with recording enabled (optionally show cursor indicator)
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true,
  "show_cursor": true
}' | jq -r '.session_id')

# ... perform actions ...

# Close to get the video file
belt app run agent-browser --function close --session $SESSION --input '{}'
# Returns: {"success": true, "video": <File>}

Cursor Indicator

Show a visible cursor in screenshots and video (useful for demos):

belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "show_cursor": true,
  "record_video": true
}'

The cursor appears as a red dot that follows mouse movements and shows click feedback.

Proxy Support

Route traffic through a proxy server:

belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "user",
  "proxy_password": "pass"
}'

File Upload

Upload files to file inputs:

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "upload",
  "ref": "@e5",
  "file_paths": ["/path/to/file.pdf"]
}'

Drag and Drop

Drag elements to targets:

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "drag",
  "ref": "@e1",
  "target_ref": "@e2"
}'

JavaScript Execution

Run custom JavaScript:

belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.querySelectorAll(\"h2\").length"
}'
# Returns: {"result": "5", "screenshot": <File>}

Deep-Dive Documentation

ReferenceDescription
references/commands.mdFull function reference with all options
references/snapshot-refs.mdRef lifecycle, invalidation rules, troubleshooting
references/session-management.mdSession persistence, parallel sessions
references/authentication.mdLogin flows, OAuth, 2FA handling
references/video-recording.mdRecording workflows for debugging
references/proxy-support.mdProxy configuration, geo-testing

Ready-to-Use Templates

TemplateDescription
templates/form-automation.shForm filling with validation
templates/authenticated-session.shLogin once, reuse session
templates/capture-workflow.shContent extraction with screenshots

Examples

Form Submission

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/contact"
}' | jq -r '.session_id')

# Get elements: @e1 [input] "Name", @e2 [input] "Email", @e3 [textarea], @e4 [button] "Send"

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "John Doe"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e2", "text": "john@example.com"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e3", "text": "Hello!"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "click", "ref": "@e4"}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'

Search and Extract

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://google.com"
}' | jq -r '.session_id')

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "weather today"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "press", "text": "Enter"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "wait", "wait_ms": 2000}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'

Screenshot with Video

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true
}' | jq -r '.session_id')

# Take full page screenshot
belt app run agent-browser --function screenshot --session $SESSION --input '{
  "full_page": true
}'

# Close and get video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
echo $RESULT | jq '.video'

Sessions

Browser state persists within a session. Always:

  1. Start with --session new on first call
  2. Use returned session_id for subsequent calls
  3. Close session when done

Related Skills

# Web search (for research + browse)
npx skills add inference-sh/skills@web-search

# LLM models (analyze extracted content)
npx skills add inference-sh/skills@llm-models

Documentation

来自 halt-catch-fire 的更多技能

agent-tools
halt-catch-fire
We need to translate the given text from English to Simplified Chinese. The instruction says to preserve product names, protocol names, URLs, numbers, and technical terms. Also, do not include the name "agent-tools" unless it appears in the source text. The source text does not contain "agent-tools" explicitly, so we don't include it. We just translate the text inside <text>. The text is a description of an agent skill. We need to output only the translation, no extra commentary. Let's translate: "Run 250+ AI apps via inference.sh CLI - image generation, video creation, LLMs, search, 3D, Twitter automation. Models: FLUX, Veo, Gemini, Grok, Claude, Seedance, OmniHuman, Tavily, Exa, OpenRouter, and many more. Use when running AI apps, generating images/videos, calling LLMs, web search, or automating Twitter. Triggers: inference.sh, infsh, ai model, run ai, serverless ai, ai api, flux, veo
developmentapicreative
ai-avatar-video
halt-catch-fire
通过inference.sh CLI创建AI虚拟形象和说话头像视频。推荐:P-Video-Avatar(最快、最便宜、内置TTS)。其他选项:OmniHuman、Fabric、PixVerse。音频:Inworld TTS-2(支持100+语言、角色情感控制)、ElevenLabs、Kokoro。功能:音频驱动虚拟形象、文本转虚拟形象、唇形同步视频、说话头像生成、虚拟主持人、UGC内容。用途:AI主持人、解说视频、虚拟网红、配音、营销视频、UGC广告、游戏虚拟形象……
videocreativemedia
ai-image-generation
halt-catch-fire
通过 inference.sh CLI 使用 GPT-Image-2、FLUX、Gemini、Grok、Seedream、Reve 及 50 多个模型生成 AI 图像。模型包括:GPT-Image-2、FLUX Dev LoRA、FLUX.2 Klein LoRA、Gemini 3 Pro Image、Grok Imagine、Seedream 4.5、Reve、ImagineArt。功能:文本转图像、图像转图像、图像修复、LoRA、图像编辑、图像放大、文本渲染。用途:AI 艺术、产品模型、概念艺术、社交媒体图形、营销视觉、插图。触发词:flux、图像生成、AI 图像、文本转...
creativemediaimage
ai-video-generation
halt-catch-fire
通过 inference.sh CLI,使用 Google Veo、Seedance 2.0、HappyHorse、Wan、Grok 及 40 多个模型生成 AI 视频。模型包括:Veo 3.1、Veo 3、Seedance 2.0、HappyHorse 1.0、Wan 2.5、Grok Imagine Video、OmniHuman、Fabric、HunyuanVideo。功能涵盖:文生视频、图生视频、参考视频生成、视频编辑、唇形同步、虚拟人动画、视频增强、拟音音效。适用于:社交媒体视频、营销内容、解说视频、产品演示、AI 虚拟人。触发词:视频生成、AI 视频……
creativevideomedia
app-store-screenshots
halt-catch-fire
根据精确的平台规格创建App Store和Google Play截图,涵盖iOS/Android尺寸、图库排序、设备样机和预览视频。用途:应用商店优化、ASO、应用截图、应用预览、Play Store列表。触发词:应用商店截图、ASO、应用商店优化、Play Store截图、应用预览、应用列表、iOS截图、Android截图、应用商店图片、应用样机、设备样机、应用图库、商店列表
designmarketingcreative
character-design-sheet
halt-catch-fire
利用参考图和LoRA技术实现AI生成图像中角色的一致性。涵盖转面视图、表情表、配色方案及风格一致性技巧。适用于:角色设计、游戏美术、插画、动画、漫画、视觉小说。触发词:角色设计、角色表、角色一致性、角色参考、转面图、表情表、角色美术、一致角色、角色概念、参考图、角色创建、OC设计……
creativedesignmedia
competitor-teardown
halt-catch-fire
结构化竞争分析,包含功能矩阵、SWOT分析、定位图谱及用户体验评估。涵盖研究框架、价格对比、评论挖掘与可视化交付物。适用于:市场调研、竞争情报、投资者演示、产品策略、销售赋能。触发词:竞争对手分析、竞争分析、竞品拆解、市场调研、竞争情报、SWOT分析、竞品对比、市场格局、竞品评估、竞争性...
researchdata-analysismarketing
image-to-video
halt-catch-fire
We need to translate the given text from English to Simplified Chinese. The text is a description of an agent skill for image-to-video conversion. We must preserve product names, protocol names, URLs, numbers, and technical terms. The name "image-to-video" is to be preserved but not included unless it appears in the source text. The source text does not contain "image-to-video" as a standalone name; it appears as part of triggers like "image to video". So we should not add it. We translate the entire text inside <text> tags. The text includes: "Still-to-video conversion guide: model selection, motion prompting, and camera movement. Covers Wan 2.5 i2v, Seedance, Fabric, Grok Video with when to use each. Use for: animating images, creating video from stills, adding motion, product animations. Triggers: image to video, i2v, animate image, still to video, add motion to image, image animation, photo to video, animate still, wan i2v,