computer-use

# Computer Use (Mac via Autonomous Buddy) ## Quick Start Autonomous Buddy is a small macOS app the user installs on their Mac. Once paired with the device via the Buddy card on the Monitor web UI, it lets the device launch apps, open URLs, type text, fire keyboard shortcuts, and click UI elements **on the user's actual Mac**. This skill emits inline markers that the device fires asynchronously while TTS speaks the confirmation: ``` [HW:/buddy/exec/<action>:<params-json>] ``` The marker hits `/api/buddy/exec/<action>` on the device, which dispatches over the buddy's persistent WebSocket. Fire-and-forget — no response is awaited (TTS continues immediately). ## Workflow 1. Determine the user's intent and pick one or more actions from the table below. 2. Build the marker(s) — flat params JSON only (no nested objects). 3. Place markers at the **start of the reply**, then add a short confirmation that TTS will speak. 4. If no Mac is paired, say so and tell the user to set it up via the device's web UI Buddy card. ### When to load `reference/vision.md` instead The marker pattern below covers ~90% of computer-use requests: launching apps, opening URLs, typing into the focused field, keyboard shortcuts, named-button clicks. It is fire-and-forget — fast, but cannot return data. Load `reference/vision.md` and follow its synchronous bash/curl loop **only** when the task requires actually seeing the screen: - "Click the blue button in the toolbar" / "click the X on that dialog"...