Skip to content

fix(native): make remote gxserver tunnel survive SSH multiplexing#41

Draft
gsxdsm wants to merge 2 commits into
maddada:mainfrom
gsxdsm:fix/remote-gxserver-tunnel-multiplexing
Draft

fix(native): make remote gxserver tunnel survive SSH multiplexing#41
gsxdsm wants to merge 2 commits into
maddada:mainfrom
gsxdsm:fix/remote-gxserver-tunnel-multiplexing

Conversation

@gsxdsm

@gsxdsm gsxdsm commented Jun 13, 2026

Copy link
Copy Markdown

Problem

Remote machine connections failed at the tunnel stage on every attempt. The debug trail showed ssh -N -L … exiting status 0 with empty stderr within ~0.3s, repeatedly:

attempt localPort=47785 argv=/usr/bin/ssh -N … -L 47785:127.0.0.1:58744 eclipxe@host
code1 tunnel exited early status=0 stderr=<empty>

A manual ssh -N -L 50000:127.0.0.1:58744 eclipxe@host + curl …/api/health/server returned HTTP 200, proving the server, token, protocol, and forwarding were all fine.

Root cause

SSH connection multiplexing. When ~/.ssh/config enables ControlMaster/ControlPath (ssh -O check"Master running"), an earlier gxserver SSH call (token read) opens a persistent master; the tunnel ssh -N -L then attaches as a slave, hands its forward to the master, and exits 0 immediately — taking its local -L listener down with it. The health probe then can't connect → tunnelFailed, deterministically.

Fix

  • Pin the tunnel to a dedicated, non-multiplexed connection (ControlMaster=no, ControlPath=none) so -N stays foreground and owns its -L forward. (the actual blocker)
  • Keepalive (ServerAliveInterval=15, ServerAliveCountMax=3, TCPKeepAlive=yes) so an established tunnel isn't silently dropped by idle NAT/firewall timeouts.
  • Keychain: write the remote token + saved SSH password to the file/login keychain (kSecUseDataProtectionKeychain=false). Ad-hoc-signed Debug builds have no keychain-access-groups entitlement, so the data-protection keychain rejected SecItemAdd with errSecMissingEntitlement. Also surface the real OSStatus in the keychainFailed toast.
  • Diagnostics: richer tunnelFailed detail (ssh stderr / last health HTTP / probe error) and a ~/.ghostex/logs/remote-gxserver-tunnel-debug.log recording the ssh argv and per-attempt outcome.

Testing

  • Reproduced the multiplexing exit-0 behavior via the new debug log.
  • ./build-ghostex-host.sh succeeds (Debug, arm64).
  • Manual tunnel + /api/health/server probe returns 200 through the SSH -L forward.

Notes / follow-ups

  • The keychain change unblocks dev builds; for release, a real signing identity with a keychain-access-groups entitlement is still the proper path.
  • Scope is intentionally limited to RemoteGxserverClient.swift.

🤖 Generated with Claude Code


Update — added commit: resilient/non-fatal token Keychain store. Ad-hoc dev builds get a new signature each rebuild, so a token item written by a prior build stays ACL-bound to it; SecItemDelete can't remove it and SecItemAdd then returns errSecDuplicateItem, which aborted connect with keychainFailed. Now: on duplicate, SecItemUpdate in place; and since the token is read fresh over SSH every connect and never read back from Keychain, any remaining store error is logged (OSStatus) and the connection proceeds instead of failing.

gsxdsm and others added 2 commits June 13, 2026 16:36
Remote connections failed at the tunnel stage on every attempt: `ssh -N -L`
attached to a persistent ControlMaster opened by an earlier gxserver SSH call,
handed off its forward, and exited 0 immediately — tearing down the local
listener so the health probe could never connect.

- Pin the tunnel to a dedicated connection (ControlMaster=no, ControlPath=none)
  so `-N` stays in the foreground holding its own -L forward.
- Add ServerAliveInterval/ServerAliveCountMax/TCPKeepAlive so an established
  tunnel is not silently dropped by idle NAT/firewall timeouts.
- Store the remote token and saved SSH password in the file/login keychain
  (kSecUseDataProtectionKeychain=false) so ad-hoc-signed dev builds without a
  keychain-access-groups entitlement stop failing with errSecMissingEntitlement;
  surface the real OSStatus in the keychainFailed toast.
- Add tunnel diagnostics: richer tunnelFailed detail (ssh stderr / last health
  HTTP / probe error) and a remote-gxserver-tunnel-debug.log recording the ssh
  argv and per-attempt outcome.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…non-fatal

Ad-hoc-signed dev builds get a new code signature each rebuild, so a token
item written by a previous build stays ACL-bound to that build. SecItemDelete
then cannot remove it and SecItemAdd reports errSecDuplicateItem, which aborted
the whole connect with keychainFailed even though the connection was otherwise
healthy.

- On errSecDuplicateItem, SecItemUpdate the existing item's value in place.
- The token is read fresh over SSH on every connect and is never read back from
  Keychain, so persisting it is best-effort: on any remaining Keychain error,
  log the OSStatus and continue to the tunnel instead of returning keychainFailed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant