|
| 1 | +# iphonebase |
| 2 | + |
| 3 | +Swift CLI to control iPhone via macOS iPhone Mirroring. Built for AI agents (OpenClaw, Claude Code, MCP) and standalone terminal use. |
| 4 | + |
| 5 | +## Tech Stack |
| 6 | +- Swift 5.9+, macOS 14+ (requires Sequoia 15.0+ at runtime) |
| 7 | +- ArgumentParser 1.3.0 |
| 8 | +- Frameworks: ScreenCaptureKit, Vision, CoreGraphics, AppKit |
| 9 | +- Karabiner-Elements required (DriverKit virtual HID for input injection) |
| 10 | + |
| 11 | +## Build & Run |
| 12 | +swift build # debug |
| 13 | +swift build -c release # release |
| 14 | +sudo cp .build/release/iphonebase /usr/local/bin/ # install |
| 15 | + |
| 16 | +## Project Structure |
| 17 | +- Sources/IPhoneBaseCore/ — library: WindowManager, ScreenCapture, OCREngine, InputInjector, HIDKeyMap |
| 18 | +- Sources/iphonebase/ — CLI entry point + Commands/ |
| 19 | +- skills/iphonebase/ — OpenClaw skill definition (SKILL.md) |
| 20 | + |
| 21 | +## Adding a New Command |
| 22 | +1. Create Sources/iphonebase/Commands/XxxCommand.swift |
| 23 | +2. Implement AsyncParsableCommand (or ParsableCommand for sync-only) |
| 24 | +3. Add XxxCommand.self to subcommands array in IPhoneBase.swift |
| 25 | +4. Include --json flag for structured output |
| 26 | + |
| 27 | +## Code Conventions |
| 28 | +- Import order: ArgumentParser, IPhoneBaseCore, Foundation |
| 29 | +- All commands support --json (use JSONSerialization with .prettyPrinted) |
| 30 | +- Results to stdout, debug/verbose to stderr |
| 31 | +- InputInjector pattern: connect() then defer { disconnect() } |
| 32 | +- Call wm.bringToFront() before any input injection |
| 33 | +- Errors: typed enums with CustomStringConvertible; throw ExitCode.failure for user errors |
| 34 | + |
| 35 | +## Coordinate System (critical) |
| 36 | +- ScreenCapture captures at 2x retina resolution |
| 37 | +- OCR (Vision) returns normalized coords with bottom-left origin — must invert Y |
| 38 | +- tap --text handles conversion automatically; raw tap x y is relative to window (screen points) |
| 39 | +- All InputInjector operations use absolute screen coordinates (window.bounds.origin + offset) |
| 40 | + |
| 41 | +## Input Injection Gotchas |
| 42 | +- iPhone Mirroring blocks CGEvent clicks — only Karabiner virtual HID works |
| 43 | +- Tap sequence: CGWarp cursor → nudge-sync virtual pointer (3x 1px/-1px) → click via HID |
| 44 | +- Timing delays (usleep) throughout InputInjector are tuned values, not arbitrary |
| 45 | +- Karabiner daemon must be running (not just installed) |
| 46 | + |
| 47 | +## OpenClaw Skill |
| 48 | +- Skill at skills/iphonebase/SKILL.md follows AgentSkills spec (YAML frontmatter + markdown) |
| 49 | +- Requires bins: ["iphonebase"], os: ["darwin"] |
| 50 | +- Install to ~/.openclaw/skills/ for agent discovery |
| 51 | + |
| 52 | +## Commits |
| 53 | +- Imperative mood ("Add feature" not "Added feature") |
| 54 | +- First line under 72 characters |
0 commit comments