feat(screen-mirror): implement cursor-follow interaction using absolute pointer mapping#344
Open
Arbaaz123676 wants to merge 2 commits into
Open
feat(screen-mirror): implement cursor-follow interaction using absolute pointer mapping#344Arbaaz123676 wants to merge 2 commits into
Arbaaz123676 wants to merge 2 commits into
Conversation
Contributor
WalkthroughScreenMirror inlines pointer handling and sends absolute/scroll/click input messages via useRemoteConnection; server accepts an "absolute" InputMessage (x/y), routes to moveRelative (with absolute flag) and falls back to direct positioning; ydotool moveRelative gains an optional absolute mode. ChangesScreenMirror → Absolute input flow
Sequence DiagramsequenceDiagram
participant Client as ScreenMirror
participant Remote as RemoteConnection
participant Server as InputHandler
participant Ydotool as ydotool
Client->>Remote: send("absolute", {x, y})
Remote->>Server: receive InputMessage(type:"absolute", x, y)
Server->>Server: clamp/validate x,y
Server->>Ydotool: moveRelative(x, y, absolute=true)
alt moveRelative succeeds
Ydotool-->>Server: success
else moveRelative fails
Server->>Ydotool: set mouse position (rounded x,y)
Ydotool-->>Server: position set
end
Server-->>Remote: ack / processed
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. 👉 Get your free trial and get 200 agent minutes per Slack user (a $50 value). Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/components/Trackpad/ScreenMirror.tsx`:
- Around line 97-109: handlePointerUp calls
e.currentTarget.releasePointerCapture unconditionally which can throw if capture
was never set (see handlePointerUp, lastPos). Before calling
releasePointerCapture, guard it by checking
e.currentTarget.hasPointerCapture(e.pointerId) (or wrap releasePointerCapture in
a try/catch) and only call releasePointerCapture when true; keep the rest of the
logic (pointerType branch, send calls, and lastPos.current = null) unchanged.
In `@src/server/InputHandler.ts`:
- Around line 161-182: The new "absolute" event handler issues high-frequency OS
calls but isn't passing through the existing throttling used for
"move"/"scroll"; update the input dispatch so "absolute" events go through the
same throttled path (or add "absolute" to the throttle switch) instead of
executing moveRelative/ mouse.setPosition directly. Concretely, route the
"absolute" branch to call the same throttled handler (or wrap its logic in the
existing throttle function), preserving the numeric validation and the try/catch
around moveRelative and mouse.setPosition (functions referenced: moveRelative,
mouse.setPosition, Point) so the absolute positioning respects the throttle
interval and falls back correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3c6c3116-631b-405c-8d90-653a1be15115
📒 Files selected for processing (5)
src/components/Trackpad/ScreenMirror.tsxsrc/routes/trackpad.tsxsrc/server/InputHandler.tssrc/server/websocket.tssrc/server/ydotool.ts
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/components/Trackpad/ScreenMirror.tsx`:
- Around line 61-67: The initial absolute cursor send in handlePointerDown (and
the similar send in the pointer-move handler) should be skipped for touch/pen
scroll gestures; update the code around the send({ type: "absolute", x: ..., y:
... }) calls inside handlePointerDown and the pointer-move handler to only call
send when e.pointerType === "mouse" || !scrollMode, so touch/pen events in
scrollMode do not teleport the remote cursor.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 41b1681a-b001-4f39-9eef-8df8d71186a9
📒 Files selected for processing (1)
src/components/Trackpad/ScreenMirror.tsx
…rg#343) perf: throttle pointer move events and guard pointer capture
Arbaaz123676
force-pushed
the
fix/issue-343-cursor-follow
branch
from
0898958 to
38c8e6a
Compare
Arbaaz123676
force-pushed
the
fix/issue-343-cursor-follow
branch
from
38c8e6a to
4a9c122
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements cursor-follow interaction for Screen Mirror with accurate absolute pointer mapping and improved input handling.
Changes
onPointerDown,onPointerMove,onPointerUp)releasePointerCaptureto avoid DOMExceptionTesting
Checklist