vercel-labs/agent-desktop
agent-desktop is a Rust computer use CLI that lets AI agents interact with desktop apps on macOS, Windows, and Linux through the OS accessibility tree. It targets developers building automation loops that need programmatic OS control.
13.4k · Rust
The agent-desktop package bridges AI models and desktop apps by leveraging the OS accessibility tree instead of raw pixel scraping. You will want to extract the cross-platform accessibility tree traversal logic for your own custom driver to skip writing platform-specific wrappers from scratch.