Extentos MCP

MCP server

MCP server

The Extentos MCP server (`@extentos/mcp-server`) is an npm package an AI agent (Claude Code, Cursor, Windsurf, Cline) installs once and then uses to add Meta Ray-Ban smart-glasses capabilities to a native iOS or Android app. It exposes a tight set of deterministic tools across 10 categories — discovery, generation, agent configuration, credentials, analytics, guidance, validation, simulation, production-readiness, and documentation — plus a CLI for account linking, telemetry consent, and update checks. This is the agent's operating manual.

The MCP server is how an AI agent — Claude Code, Cursor, Windsurf, Cline, or any Model Context Protocol-compatible host — operates Extentos. The agent invokes deterministic tools; the server primes the agent on what capabilities the glasses expose, returns canonical SDK code patterns in Kotlin and Swift, scaffolds the project, brokers simulator sessions, and queries debugging traces. The server itself has no planning tool — agents are better planners than regex bundles. Tools are typed primitives the agent composes in sequence.

This page is the section landing — what the server is, the tools at a glance, the canonical agent-driven flow, configuration knobs, the CLI, and how the auth model works. Sub-pages cover each in depth.

Install

claude mcp add extentos -- npx -y @extentos/mcp-server@latest

For non-Claude-Code hosts, see the agent prompt or manual JSON install paths. Full install reference at /docs/mcp-server/install.

The tools, by category

The server exposes a deterministic tool surface (verified in mcp-server/src/tools/definitions.ts), organized into 10 categories. The categories are the agent's mental map; an agent that understands which category a tool lives in can decide when to call it. The always-current full catalog is the generated tools reference.

1. Discovery and SDK reference (3 tools)

The first calls in any new task. Cheap, all local, no side effects.

Tool	What it does
getPlatformInfo	Returns the static platform catalog — library version, the list of SDK capabilities the glasses expose, per-vendor tiers. Always the right first call.
getCapabilityGuide	Per-feature minimal usage in Kotlin + Swift — call shape, config args, gotchas. Pairs with getPlatformInfo (which names the features) to tell the agent how to invoke each.
getCodeExample	Full canonical compositions in both languages. Start with assistant_agent_loop (the Phase-4 canonical voice-assistant flow) and agent_driven_e2e_full_loop (the agent-driven E2E test). Also covers voice_qa_assistant, barge_in_speak, photo_describe_voice, live_transcription_ui, voice_notes, connection_page_setup, byok_anthropic, display_browse_detail, display_media_gallery, and video_frames_ml. Peel from these when writing handler code. The full enumerated list is generated at /docs/reference/mcp-tools.

2. Setup and Generation

Tool	What it does
generateConnectionModule	One-shot scaffold — bootstrap module, Gradle/SPM wiring, dependencies, permissions, manifest. Two-call flow: first call without placement returns a question asking where ExtentosConnectionPage should live; second call with the chosen placement returns the full file set.
getConnectionPageConfig / setConnectionPageConfig	Read / write the per-project connection-page config (theming tokens + section visibility) the dashboard/server holds.
regenerateConnectionPageFile / adoptConnectionPageFile	Sync the committed extentos.connection-page.json with the server config — regenerate (server→file) or adopt (file→server).

After scaffolding, the agent writes its own handler classes against the SDK primitives surfaced by getCapabilityGuide / getCodeExample. Handler code is the customer's authoring surface — there's no initSpec or DSL-population step.

3. Agent configuration & usage (3 tools)

Account-scoped — require a linked account and are gated per-project by the MCP access grant (default Read+Write).

Tool	What it does
getAssistantConfig / setAssistantConfig	Read or change a project's dashboard-managed assistant settings — the OpenAI Realtime model, the voice, the memory (compaction) model, and the within-session memory mode. set is a partial update, validates each value against the catalog, and echoes the cost impact of a model change.
getGatewayUsage	Read the project's managed-gateway usage + exact cost over a recent window — token counts and list-price USD from the billing ledger, broken down by model. Metadata only, never transcripts or content.

4. Credentials (2 tools)

Account-scoped + per-project gated. Write-without-knowing — the secret never passes through the agent.

Tool	What it does
getCredentialStatus	Read whether a project's OpenAI API key (BYOK) and Meta DAT build identity are set — a masked hint + updated-at only, never the value.
setCredential	Start write-without-knowing entry of a credential — returns a dashboard link where the signed-in owner pastes the secret straight into the encrypted vault. Takes no secret argument by design.

5. Analytics (1 tool)

Tool	What it does
getProjectAnalytics	Read a project's production analytics — aggregate telemetry from shipped App Store / Play Store installs (events, active installs, by event / day / vendor / platform). Metadata only, account-scoped, ownership-checked, gated by the Analytics grant. Empty until the app ships and sends prod-attested events (use getEventLog for the live dev/sim stream).

6. Implementation Guidance (2 tools)

Side-quest tools the agent calls during composition.

Tool	What it does
getVoiceCommandGuidance	Analyze proposed wake / command phrases for UX issues (collisions, ambiguity, hard-to-recognize words, Meta wake-word conflicts) before wiring them into a glasses.audio.transcriptions() consumer.
getPermissions	Derive exact platform permissions, Meta DAT requirements, and foreground-service needs from the capability list. Run when adding or removing a primitive from your handler.

7. Validation (2 tools)

Correctness gates. Run after structural changes (new capability declared, dependency bumped, manifest edited).

Tool	What it does
inspectIntegration	Read-only project snapshot — manifest, generated-file hashes, dependency list, connection-page config. Run before manual edits to understand current state.
validateIntegration	Whole-project correctness check — manifest, generated files, dependency declared, permissions cover declared capabilities, bootstrap calls ExtentosGlasses.create(...), toolchain versions, foreground-service hints for continuous-capture flows. The pre-test gate.

8. Simulation

Provision and operate browser-based simulator sessions, plus the agent-driven testing tools that close the end-to-end loop without a human.

Tool	What it does
createSimulatorSession	Get-or-create a browser-mode session at extentos.com/s. Returns the saved sim for this project if one exists (status: "resumed"), or mints a new one (status: "active"). Auto-attaches the running app via the local bridge when reachable; otherwise emits a BuildConfig.EXTENTOS_SESSION_URL snippet (Android) or extentos.session.plist payload (iOS). Pass resetFresh: true to rotate the sessionId for a clean slate.
ensureSimulatorBrowser	Open + confirm a connected simulator browser tab — the precondition for camera and inject flows.
completeAuthLink	After createSimulatorSession returns status: "auth_required" (anonymous install needs to link to mint sessions), polls the backend until the user finishes signup, then persists the bearer token to ~/.extentos/auth.json.
getEventLog	Fetch structured event traces from a session. Filter values: all (no filter) plus the seven chips errors, voice, camera, display, ai, lifecycle, custom — one chip per event, with errors absorbing severity≥warn regardless of modality. Plus cursor, follow, limit for trace-level scope. The primary debugging tool.
getSimulatorStatus	Read a live session's current state — phase, hardware-ready, attached roles, active capability streams, current toggle values.
injectTranscript / injectAssistantUtterance / assertToolCalled	Drive a wake phrase or an assistant turn, then assert which tool the model called — the agent-driven E2E loop, no human needed.
setSimVideo / setSimDevice	Pipe a test video into the simulated camera; switch the simulated device model (e.g. rayban_display to exercise the display path).
getDisplayState / injectInput	Read the display tree currently rendered + drive display input (select / navigate / back).

9. Production (2 tools)

Pre-ship checks.

Tool	What it does
getProductionChecklist	Personalized production-readiness checklist based on declared capabilities + handler names — credential wiring, permission audit, foreground-service requirements (when continuous capture is used), simulator-URL removal from release builds, store-listing readiness.
getCredentialGuide	Step-by-step credential setup for production AI providers — anthropic, openai, google_cloud_vision, google_translate, google_gemini, deepl, azure_cognitive, aws_bedrock, huggingface, or custom — plus Meta DAT registration.

10. Documentation & search (1 tool)

Tool

What it does

searchDocs

Search Extentos documentation by topic or keyword. For voice assistants read assistant_runtime first; conversation_runtime is the deprecated Phase-3 runtime (migration reference only). Other aligned topics: voice_integration, agent_e2e_testing, managed_gateway, conversation_memory, display, plus the stable conceptual set — getting_started, custom_handlers (the canonical SDK-composition doc), simulator_browser_mode, event_log_schema, toggles, library_api, permissions, multi_platform_projects. Topic IDs are stable; the live tool input is authoritative.

Full per-tool reference with input schemas, response shapes, and worked examples: /docs/mcp-server/tools.

The canonical agent-driven flow

In a fresh project, the agent invokes the tools in this order:

1. getPlatformInfo({ sections: ["version", "capabilities"], glasses: "meta_rayban" })
2. getCodeExample({ pattern: "assistant_agent_loop" })  // Phase-4 voice assistant; or whatever pattern fits
3. getCapabilityGuide({ feature: "<each primitive the handler will use>" })
4. generateConnectionModule({ platform, glasses, appPackage })
   → returns "needs_placement" question
5. generateConnectionModule({ ... placement: "<chosen>" })
   → writes scaffold files (ExtentosBootstrap, manifest, etc.)
6. <agent writes handler class(es)> against the SDK primitives
   <agent updates extentos.manifest.json's `capabilities` array>
7. validateIntegration()
   → ✓ all good (or returns structured errors to fix)
8. createSimulatorSession({ glasses })
   → returns sessionId; auto-opens browser at extentos.com/s/<id>
   → if running app is reachable via local bridge, it auto-attaches
9. <developer interacts with the simulator; capability events flow into the backend>
10. getEventLog({ sessionId, filter: "errors" })  → debug
    getSimulatorStatus({ sessionId })             → status

For iteration: edit handler code → rebuild + reinstall → the app auto-attaches to the same simulator session (no remint, the URL is stable). Before shipping: getProductionChecklist and getCredentialGuide.

Configuration

The MCP server reads these environment variables (verified in mcp-server/src/):

Variable	Default	What it does
EXTENTOS_BACKEND_URL	Production backend	Override the backend URL (tools/util/backendClient.ts). For local development of Extentos itself.
EXTENTOS_CONFIG_DIR	~/.extentos	Override the config/auth directory (telemetry/consent.ts).
EXTENTOS_TELEMETRY	unset (consent-default)	Set to 0 to decline telemetry without running the CLI consent command.
EXTENTOS_NO_AUTO_OPEN	unset	Set to 1 to disable browser auto-open on simulator-session creation (useful in headless environments).

Full configuration reference: /docs/mcp-server/configuration.

CLI subcommands

Running npx @extentos/mcp-server@latest with no arguments starts the MCP server over stdio (the path the agent uses). With a subcommand, it acts as a developer CLI:

Subcommand	What it does
login	Link this install to an Extentos account via device-code flow (proactive — useful before the first simulator session, or after logout to re-link).
logout	Clear ~/.extentos/auth.json. Install returns to the anonymous tier; the next simulator session call will re-trigger the device-code flow.
whoami	Not yet implemented (Phase-0 stub). Will print installId, accountId (if linked), tier, auth expiry.
setup	Preflight the local build environment — checks the GitHub Packages PAT (read:packages) the transitive Meta DAT artifacts need, plus other dependency prerequisites.
accept-privacy	Record privacy consent (enables telemetry upload).
decline-privacy	Record privacy decline (disables telemetry upload).
status	Print consent state, install ID, linked account, MCP/library versions.
update	Check for MCP server updates (no-op on npx @latest installs).

Full CLI reference: /docs/mcp-server/auth.

Auth model

The MCP server is anonymous-first. Discovery, capability guides, code examples, validation, docs search, on-device simulation, and real-hardware testing all work without ever signing in. Three things link a free account: minting browser-simulator sessions (createSimulatorSession, HTTP 402), the generateConnectionModule scaffold step (it mints your account-bound project key — same 402 device-code flow; the informational first call is anonymous), and the account-scoped project tools (assistant config, credentials, connection-page writes, analytics — HTTP 401).

The device-code flow: the first gated call returns status: "auth_required" with a verification URL. The agent calls completeAuthLink to poll the backend; the developer signs up with a free email-only account at the URL (Google or email + password, no payment); the backend issues a token; the original tool call retries automatically. After linking, simulator sessions are unlimited.

Extentos's tools, codegen, validation, SDK, and the browser simulator are free — there's no per-seat or subscription fee to build and ship. The one surface that meters usage is the managed AI gateway behind the Phase-4 voice assistant. Full auth model: /docs/mcp-server/auth; pricing: /docs/resources/pricing.

Privacy and telemetry

On first run, the MCP server injects a one-time privacy notice into the response. Telemetry is anonymous (tagged with installId, no source code or personal data) and is dismissed-by-continuing by default — same pattern as Vercel CLI, Astro, Vite. Decline at any time:

npx @extentos/mcp-server@latest decline-privacy
# or
EXTENTOS_TELEMETRY=0 (env var, persistent for the shell)

Privacy notice content is in mcp-server/src/index.ts (PRIVACY_NOTICE constant). Notice is shown once per install via claimFirstPrivacyNotice — never repeats.

Compatible MCP hosts

Verified to work with:

• Claude Code — primary target. One-line install via claude mcp add.
• Cursor — JSON config in ~/.cursor/mcp.json.
• Windsurf — JSON config in ~/.codeium/windsurf/mcp_config.json.
• Cline — JSON config in Cline's MCP settings.
• Any MCP-compatible host — drop the standard mcpServers.extentos JSON block in.

The MCP server speaks the standard MCP protocol over stdio (@modelcontextprotocol/sdk); no host-specific code paths exist on the server side. Per-host install steps: /docs/mcp-server/agents.

The local bridge — auto-bind dev loop

When the server starts, it opens a 127.0.0.1:31337/whoami HTTP listener (mcp-server/src/localBridge.ts). The Extentos library on the developer's app probes this endpoint at runtime to learn its host MCP's installId. The result: every createSimulatorSession call from the agent auto-attaches the running app to the new session — no rebuild, no URL paste.

Reach paths:

• Android emulator: http://10.0.2.2:31337/whoami (host-loopback NAT alias)
• iOS Simulator: http://localhost:31337/whoami (shares host network namespace)
• Physical Android phone via USB: adb reverse tcp:31337 tcp:31337 once, then localhost:31337 from the device
• Cellular phone or cloud-hosted agent: probe times out. The agent uses the URL-bake path instead — createSimulatorSession returns a BuildConfig.EXTENTOS_SESSION_URL snippet (Android) or extentos.session.plist payload (iOS) the developer pastes in, then rebuilds the app once. Less elegant than auto-bind but works on any topology.

Bound to 127.0.0.1 only. The installId is not a secret — it's the same value the MCP sends to api.extentos.com on every tool call. No auth needed at this layer.

If port 31337 is in use (rare; another MCP instance already running), startup logs a warning and continues. Auto-bind silently fails for that session; the developer uses the URL-bake path until the port frees up.

Status

• Package: @extentos/mcp-server on npm (MIT license)
• Engines: Node.js 20+
• Pre-1.0 — APIs may shift between minor versions until the hardware test loop closes. Pin to an exact version if you need cross-session reproducibility.

Related

• Quickstart with an AI agent — install the server and walk through a real dev loop
• Tools reference — full per-tool API
• Configuration — environment variables, config files, install-time settings
• Auth — device-code flow, account linking, CLI auth commands
• Supported agents — per-host install instructions
• Architecture — how the MCP server fits into the broader Extentos system
• Transport vs app simulation — what the simulator the MCP brokers actually does

Related

Quickstart with an AI agent

Install the Extentos MCP server and let your AI agent scaffold Meta Ray-Ban smart-glasses capabilities into a native iOS or Android app. Free to start.

Architecture

How Extentos fits together — AI agent, MCP server, native Kotlin/Swift SDK, three transports (Meta DAT, browser sim, local in-memory sim), and the backend.

Transport vs app simulation

Meta's Mock Device Kit simulates the transport layer; Extentos simulates the app layer — voice, photo capture, and the wearing experience. Both matter.

Install the MCP server

How to install the Extentos MCP server (@extentos/mcp-server) in any MCP-compatible AI coding agent — Claude Code, Cursor, Windsurf, Cline, and others. Per-host install commands, config file locations, copy-pasteable JSON snippets, restart and verify steps, version pinning, updating, troubleshooting common errors, and uninstall instructions. Verified install paths for each supported host.

Tools reference

The Extentos MCP server exposes a tight set of deterministic tools across deterministic categories — discovery + SDK reference, generation, guidance, validation, simulation, production, agent configuration, and search. Per-tool reference covering input parameters, response shapes, when to call each tool, and worked examples. Verified from the actual tool definitions and handler implementations in @extentos/mcp-server.

Configuration

Configure @extentos/mcp-server — environment variables, the ~/.extentos state files (install id, auth, consent), and the default vendor.

Auth

How Extentos auth works. Almost everything runs with no account; only the browser simulator needs sign-in, via the OAuth 2.0 device-code flow. No payment.

Supported agents

Per-host differences for AI coding agents that run Extentos — Claude Code, Cursor, Windsurf, Cline, IDE built-ins. Scope, cross-platform reach, MCP support.

Apple smart glasses

Apple smart glasses for third-party developers — SDK access, app model, distribution, capabilities & AI, and where it sits in the 2026 smart-glasses landscape.

Install the MCP server