Summary
Privilege changes made outside the running server — by the invoke-usermod / invoke-userdel CLI tools — do not reach live Socket.IO connections. A user who is deactivated or deleted from the command line keeps their open sockets, stays in the rooms they were placed in at connect time, and continues receiving events (including admin-room events if they were an admin) until they reload the page.
Background
PR #9360 established the principle that the JWT proves identity and the database decides authorization. REST requests, queue dequeue, and node boundaries all re-read the user record, so they pick up an out-of-band change immediately.
Socket.IO is the one surface that does not. Room membership is decided once, during connect, from the user record at that moment. It is revised only when the server-internal user_access_changed event fires — which the API routes emit (invokeai/app/api/routers/auth.py), and the CLI does not.
invokeai/app/util/user_management.py constructs UserService(db) directly against the SQLite file:
db = SqliteDatabase(config.db_path, InvokeAILogger.get_logger())
user_service = UserService(db)
It is a separate process with no access to the running server's event bus, so there is no straightforward way for it to emit into the server.
Reproduction
- Run the server in multiuser mode.
- Sign in as user
U in a browser and leave the tab open, so the socket is connected.
- From a shell on the server:
invoke-userdel u@example.com (or invoke-usermod to deactivate).
- Observe:
U's next REST request correctly fails — the DB-derived checks catch it.U's socket stays connected and keeps receiving that user's events.- If
U was an admin, they remain in the admin room and keep receiving admin-scoped events.
- Reloading the page finally disconnects them, because connect re-reads the database.
Deactivating a user mid-generation has a related, smaller gap: the running queue item is not cancelled immediately (no event to trigger it), though the node-boundary owner check does stop it at the next node.
Why the obvious fixes do not work
- Emit from the CLI — separate process, no event bus.
- Refuse to run the CLI while a server is live — breaks the recovery use case the CLI exists for.
- Re-check on every socket emit — a database read per emitted event is too expensive on the progress-event path.
Suggested direction
Stop treating room membership as authoritative and start treating it as a cache: add a periodic reconciliation sweep that re-derives room membership from the database for currently connected sockets, disconnecting users whose record is gone or inactive and adjusting the admin room on role change.
The cost is bounded (one pass over connected sockets on an interval, not per event), and it catches any out-of-band mutation — direct sqlite3 edits, a future admin tool, a restored backup — rather than just the two CLI entry points. The existing user_access_changed event stays as the fast path so API-driven changes remain instant; the sweep is the backstop.
An alternative worth weighing: give the sweep something cheap to poll, e.g. reconcile only when a monotonically increasing counter on the users table has changed since the last pass, so the steady-state cost is a single query.
Related
Summary
Privilege changes made outside the running server — by the
invoke-usermod/invoke-userdelCLI tools — do not reach live Socket.IO connections. A user who is deactivated or deleted from the command line keeps their open sockets, stays in the rooms they were placed in at connect time, and continues receiving events (including admin-room events if they were an admin) until they reload the page.Background
PR #9360 established the principle that the JWT proves identity and the database decides authorization. REST requests, queue dequeue, and node boundaries all re-read the user record, so they pick up an out-of-band change immediately.
Socket.IO is the one surface that does not. Room membership is decided once, during connect, from the user record at that moment. It is revised only when the server-internal
user_access_changedevent fires — which the API routes emit (invokeai/app/api/routers/auth.py), and the CLI does not.invokeai/app/util/user_management.pyconstructsUserService(db)directly against the SQLite file:It is a separate process with no access to the running server's event bus, so there is no straightforward way for it to emit into the server.
Reproduction
Uin a browser and leave the tab open, so the socket is connected.invoke-userdel u@example.com(orinvoke-usermodto deactivate).U's next REST request correctly fails — the DB-derived checks catch it.U's socket stays connected and keeps receiving that user's events.Uwas an admin, they remain in the admin room and keep receiving admin-scoped events.Deactivating a user mid-generation has a related, smaller gap: the running queue item is not cancelled immediately (no event to trigger it), though the node-boundary owner check does stop it at the next node.
Why the obvious fixes do not work
Suggested direction
Stop treating room membership as authoritative and start treating it as a cache: add a periodic reconciliation sweep that re-derives room membership from the database for currently connected sockets, disconnecting users whose record is gone or inactive and adjusting the admin room on role change.
The cost is bounded (one pass over connected sockets on an interval, not per event), and it catches any out-of-band mutation — direct
sqlite3edits, a future admin tool, a restored backup — rather than just the two CLI entry points. The existinguser_access_changedevent stays as the fast path so API-driven changes remain instant; the sweep is the backstop.An alternative worth weighing: give the sweep something cheap to poll, e.g. reconcile only when a monotonically increasing counter on the
userstable has changed since the last pass, so the steady-state cost is a single query.Related
user_access_changedevent.