ywza
Let an AI model see and drive your Windows or Linux PC through the accessibility tree: numbered element lists instead of screenshots, terminals, and pairing with phones or other PCs peer to peer.
Documentation
ywza
Let an AI model use your PC, from anywhere.
Demo •
Install •
Quick start •
Pairing •
MCP •
Linux •
Roadmap
ywza lets an AI model use your computer the way you do. It can look at your windows, click and type in any app, and work in the terminals you already have open. You can talk to it from the same machine or from another one, and nothing has to be exposed to the internet to make that work.
It runs as a small background agent on the PC you want to control. Other devices connect to it directly, peer to peer, after you pair them once with a 6-digit code. There is no account and no server to host. You bring your own model: anything that speaks the OpenAI chat API works, including local ones like Ollama and LM Studio.
Instead of sending the model a screenshot at every step, ywza describes each window as a numbered list of elements. That keeps the token count low and the model's reasoning focused. It can still ask for a screenshot when it needs to see something.
ywza works perfectly on Windows. Linux works on X11.
[!IMPORTANT] Help Wanted: This tool does not currently support macOS or Apple devices. Pull requests to add support are very welcome!
Demo
|
Opening Notepad, typing Hello World and moving the window to monitor 0 |
Opening Reddit in Chrome |
|
Typing Hello World into an empty terminal, then starting Claude in it |
Alt + ` opens the command box, then asking it to open the repo on GitHub in Brave |
Install
Windows
In PowerShell:
irm https://raw.githubusercontent.com/alielmorsy/ywza/main/scripts/install.ps1 | iex
This downloads the latest installer and runs it. It installs the desktop app, the ywza-agent service and the
ywza CLI, and puts them on your PATH.
Releases
Windows, Linux and Android builds are on the releases page.
Build from source
You need a recent Rust toolchain. On Windows, the MSVC build tools that rustup asks for are enough.
git clone https://github.com/alielmorsy/ywza
cd ywza
cargo build --release
This produces two programs in target/release:
| Program | What it is |
|---|---|
ywza-agent | The background service. Runs on the machine you want to control. |
ywza | The command line client. Runs wherever you give the instructions. |
Put both on your PATH, or install them with cargo:
cargo install --path crates/ywza-agent
cargo install --path crates/ywza-cli
Desktop and phone app
The app needs Node.js and Yarn on top of Rust, plus the Tauri prerequisites for your platform.
yarn install
yarn build # desktop app; on Windows, the installer
yarn android:build --apk --target aarch64 # Android APK, needs the Android SDK and NDK
OCR
Some apps draw their own text and expose nothing to Windows accessibility. For those, build the agent with an OCR engine. You can compile several in and pick one in the config.
| Feature | Engine | Platform |
|---|---|---|
ocr-windows | Built-in Windows OCR | Windows |
ocr-paddle | PaddleOCR through a bundled ONNX Runtime | Windows, Linux |
ocr-paddle-dynamic | PaddleOCR, ONNX Runtime loaded at startup | Linux, older distros |
cargo build --release -p ywza-agent --features ocr-windows
PaddleOCR models download on first use. ocr-paddle-dynamic replaces ocr-paddle, so don't enable both. It needs
libonnxruntime.so next to the agent binary, or at the path in ORT_DYLIB_PATH.
Quick start
Start the agent:
ywza-agent run
The first launch creates your config folder and the agent's identity. Point it at a model:
ywza config set-llm --base-url https://api.openai.com/v1 --model <model> --api-key <key>
Then give it something to do:
ywza prompt "Open Notepad and write hello"
Control it from another device
On the machine you want to control, ask the running agent for a pairing code:
ywza-agent pair
On the device you want to control it from:
ywza pair 123456 --name desktop
ywza -d desktop prompt "Take a screenshot and tell me what's open"
ywza-agent pair also prints a QR code, and the phone app can scan it instead of typing the digits.
The code works once and expires after 10 minutes. On Windows, if the connection times out, run
dist/windows-firewall.bat as administrator with the path to ywza-agent.exe.
Secrets
Keep passwords and tokens away from the model with the vault on the machine that types them:
ywza-agent vault set github # reads the value from stdin
ywza-agent vault list
The model types {{vault:github}}. The agent fills in the value as it types. Tool results show the placeholder
instead of the value, and screenshots blur the value and password fields. Secrets live in vault.toml next to
config.toml.
Token receipt
Each agent run ends with a receipt in the app and the CLI. It shows the tokens used and an estimate of what a
screenshot-based agent would have used for the same steps. To show cost as well, set price_in and price_out
under [llm] in config.toml, in USD per million tokens.
Use it from an MCP client
The MCP server gives any MCP client control of this PC and of every device you have paired, through one server. The model picks the device per call, so a single chat can work on your desktop and a remote machine side by side.
Paired devices don't need to be on the same network. They connect through public relays, so a laptop on a coffee shop wifi can drive a PC at home. Own relay support is planned, so you will be able to run your own instead of the public ones.
Put ywza on your PATH, then install the plugin from the plugin folder. It carries the skill and the MCP server for
Claude Code, Codex and Antigravity. See plugin/README.md. Any other MCP client can run python plugin/server/run.py.
Linux
Only X11 sessions are supported. dist/ywza-agent.service runs the agent on the login screen and in every user
session. Edit the ExecStart path, then follow the comments at the top of the file.