Skip to content

feat(factories): add the Factory model and automation link - #751

Merged
jpshackelford merged 10 commits into
mainfrom
feat/factories
Nov 18, 2026
Merged

jpshackelford merged 10 commits into
mainfrom
feat/factories

Conversation

@jpshackelford

@jpshackelford jpshackelford commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

HUMAN:

A factory is a set of automations (and eventually other content) which together implements an SDLC for one or more closely related projects. This first PR provides the foundational factory model and is designed to support a factory UI in Agent Canvas which is initially a named group of automations. Other PRs will build on this.

Manual tests were performed by an agent and evidence is in comments below.

AGENT:

Why

An automation is a standalone object today, and nothing says which goal it serves. tags are many-to-many labels with no owner and nothing the platform can attach data to. A factory gives a group of automations one identity that other features can hang off. Worklogs (OpenHands/enterprise#1297) and the run token from #726 build on it.

Summary

This PR has no dependency on another PR; it is based on main. #726 depends on this one.

  • Data model (migration 040, the single head after 039): a factories table (id, org_id, name, created_at, updated_at) and a nullable, indexed automations.factory_id with ON DELETE SET NULL. Existing rows keep NULL, with no backfill. There is no default factory and a factory has no other attribute. The migration uses batch mode so the foreign key also works on SQLite.
  • Factory API under /api/automation/v1/factories: POST, GET, GET /{id}, PATCH /{id} (rename), DELETE /{id}. Responses carry id, name, automation_count (live automations only), created_at and updated_at. Names need not be unique.
  • factory_id on create: POST /v1, /v1/plugin-spec, /v1/preset/prompt and /v1/preset/plugin accept it, through the same check as PATCH (manage_automations else 403; unknown or other-org factory 404). A draft stores it, but its hidden test run does not carry it, and /v1/validate and draft validation report factory_id_403 / factory_id_404 for a factory the caller cannot use. Re-enabling an installed template (template on a create endpoint) returns the existing automation as before, but 409 if the request names a factory it is not in. factories is listed in /v1/capabilities features.
  • Assignment: PATCH /{automation_id} accepts factory_id (null removes it; omitting it leaves it unchanged). The target must be in the automation's org, otherwise 404. Changing it needs manage_automations; the creator exception in _assert_can_manage does not apply, so a creator without that permission gets 403. Resending the stored value, or null for an automation with no factory, is not a change. Any PATCH of an automation that is in a factory also needs manage_automations, so a creator cannot change what runs once an admin has put it in a factory. Every automation response includes factory_id. The existing change audit records the change in changed_fields with no extra code.
  • List filter: GET /v1?factory_id= alongside tag, created_by and run_as_user_id.
  • Delete: 409 with the live-automation count while any live automation belongs to the factory. Otherwise one transaction detaches soft-deleted automations through the ORM (so each detach is audited) and removes the row with a single DELETE ... WHERE NOT EXISTS (live automation). On PostgreSQL the delete locks the factory row FOR UPDATE and an assignment locks it FOR KEY SHARE just before its write, so an assignment cannot land between the check and the delete and a rename is not blocked by a slow create. On SQLite an assignment takes the write lock with a no-op update of the factory row before its write.
  • Authorization: reads need view_automations; create, rename and delete need manage_automations. test_route_permissions.py lists the three new manage-only routes.
  • Git sync neither writes nor reads factory_id (covered by tests).
  • Docs: AGENTS.md gets a Factories section, and the verify-automation feature map gets family F19 (its coverage test requires every route to have an owner), and F03's default features check lists factories.

Decisions to confirm: manage_automations is required to change factory_id (create with a factory, or PATCH to a different value), not to resend it, and to PATCH any automation that is in a factory. Delete, dispatch and cancel are not covered by the second rule (they do not change what runs). A draft's hidden test-run automation cannot be put in a factory (PATCH returns 409). Git sync, plugin refs and external tarball URLs can still change what an automation runs; they are trusted as the source of its code and not guarded here (see AGENTS.md). GET /factories pages with limit/offset (default 50, max 100) and returns total, like the webhook list.

Issue Number

Fixes #725

How to Test

What I ran, in the repository root (uv sync --frozen --group dev first, Docker available for the PostgreSQL fixture):

  • uv run ruff format --check, uv run ruff check and uv run pycodestyle --max-line-length=88 --ignore=E203,E501,W503,E704 on the changed Python files: passed.
  • uv run pyright on the changed Python files: passed after I fixed one type error in tests/test_factories_migration.py.
  • uv run python -m pytest tests/ --cov=openhands/automation --cov-fail-under=76: 3053 passed, 7 skipped, coverage 79.9%. It also had 1 failure and 15 errors. The failure was test_committed_feature_map_owns_and_drives_every_route (the new routes had no owner), fixed by adding F19. The 15 errors were fixture 'caplog' not found, caused by a -p no:logging flag I had added to the command, not by the code; I re-ran those 15 tests by ID without the flag and they passed.
  • After the F19 fix I re-ran tests/test_verify_automation_skill.py: 59 passed.
  • The new tests (tests/test_factories.py, tests/test_factories_migration.py) run on PostgreSQL and SQLite, and passed in the full run above. They cover the API by id, assignment, the list filter, deletion, authorization, the change audit and the git-sync boundary. The migration test upgrades a SQLite database that already holds an automation and a run, checks they survive and that slug is gone, and checks the downgrade.
  • Against a real PostgreSQL 15 container, after renumbering to 039: alembic upgrade head reported 039 (head); \d automations showed the factory_id column, its index and the ON DELETE SET NULL foreign key; downgrade 038 left no factories table; upgrade head restored 039 (head); alembic heads shows one head.

Re-run after the review changes (slug removed, factory_id on create), on Docker with PostgreSQL:

  • tests/test_factories.py, tests/test_factories_migration.py, tests/test_route_permissions.py, tests/test_capabilities_router.py and tests/test_draft_execution_regressions.py: 172 passed on PostgreSQL and SQLite.
  • tests/test_preset_router.py, tests/test_router.py, tests/test_plugin_specs.py, tests/test_draft_router.py and tests/test_startup_migrations.py: 695 passed.
  • The verify-automation structure check passes (59 passed).
  • The affected manual-map families, each on a fresh live local service (control-automation map run --file Fnn --fresh --keep-going --record): F19 7/7, F03 17/17, F05 25/25, F06 16/16, F09 20/20. Details are in the PR comment.

Second round of review changes (migration renumbered to 039, template conflict, PATCH only on change, draft validation, lock and delete changes), at 1b1a1d4 on Docker with PostgreSQL 15:

  • Full suite: 3231 passed, 8 skipped, 0 failed, 0 errors.
  • New tests in tests/test_factories.py: template re-enable (matching, omitted, different factory, no factory) on all four create endpoints; /v1/validate and draft create with an unknown factory, another org's factory and a member; PATCH of null and of the stored value; member and admin edits of an automation in a factory; a DELETE racing a PATCH 20 times with a client that gives each request its own session (PostgreSQL and SQLite); and, on PostgreSQL, a rename completing while an assignment holds the lock and a delete waiting for it. The race test fails on the previous commit on SQLite.
  • ruff format --check, ruff check, pycodestyle and pyright pass on the changed files.

Third round of review changes (migration renumbered to 040 after main took 039, draft guard, create-side dedupe, description constant), at 6cbf6c3 on Docker with PostgreSQL 15, after merging current main:

  • tests/test_factories.py, tests/test_factories_migration.py, tests/test_route_permissions.py, tests/test_capabilities_router.py, tests/test_draft_execution_regressions.py, tests/test_migration_history.py: 228 passed, 1 skipped (PostgreSQL row locks, on SQLite).
  • tests/test_preset_router.py, tests/test_router.py, tests/test_plugin_specs.py, tests/test_draft_router.py, tests/test_startup_migrations.py, tests/test_verify_automation_skill.py and tests/test_git_sync*.py: 1033 passed.
  • Before the fix, PATCH {"factory_id": ...} on a draft's hidden automation returned 200 and the factory then refused to delete (409). The new test test_draft_test_run_automation_cannot_be_moved_into_a_factory covers it on both databases.
  • Against a real PostgreSQL 15 container: alembic upgrade head reported 040 (head), heads shows one head, downgrade 039 and upgrade head round-trip, and \d automations shows the factory_id column, its index and the ON DELETE SET NULL foreign key.
  • ruff format --check, ruff check and pyright pass on the changed files. I did not run the full suite after this round.

What I did not run:

  • pre-commit run --all-files end to end; CI runs the full hook set.
  • The manual map (control-automation map run) after 1b1a1d4. Its F19, F03, F05, F06 and F09 results above are from 0abc0b3; the admin paths they drive did not change.
  • The manual map does not drive factory_id on /v1/preset/* or /v1/plugin-spec, or the member 403 cases (the local key holds manage_automations); only the unit tests cover them. The plugin-spec success path is not covered either, since it needs a fetchable manifest.

This PR was created by an AI agent (OpenHands) on behalf of @jpshackelford.

@jpshackelford can click here to continue refining the PR

🤖 Generated with Claude Code

Add a factories table and a nullable automations.factory_id, the factory
CRUD API under /v1/factories, factory assignment through PATCH /v1/{id},
and a factory_id filter on GET /v1.

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions github-actions Bot added the type: feat A new feature label Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Coverage

Warning

Your comment is too long (maximum is 65536 characters), so the coverage report was not added. See the job log for how to reduce it.

@jpshackelford

jpshackelford commented Nov 13, 2026 •

Copy link
Copy Markdown
Member Author

Manual test results (PR head 77a81a2)

I ran the service in local mode against SQLite and against a real PostgreSQL 15 container, and drove the real HTTP API with a script (59 checks, all passed on both). I also ran the PR's own tests (SQLite and PostgreSQL), and a few extra live checks for PostgreSQL-only and cross-feature behaviour. Results are grouped by how each item was verified.

1. Tested live (SQLite and PostgreSQL)

Factory CRUD

# What was checked Result
T1 Fresh install has zero factories and no default Pass
T2 Create with only name: 201, derived slug, automation_count: 0, timestamps; GET by slug matches Pass
T3 Slug rules: supplied slug lowercased; duplicate (supplied or derived) 409; bad slug, underivable name, blank or missing name, extra field all 422 Pass
T4 Rename changes only name; slug in PATCH 422; unknown factory 404; old slug 404 Pass

Automation ↔ factory relationship (each test created real automations and linked them to factories)

# What was checked Result
T5 Assign an automation to a factory, move it to another, clear it with null; omitting the field leaves it unchanged; unknown id 404; malformed id 422; every automation response carries factory_id Pass
T6 GET /v1?factory_id= returns only that factory's automations and combines with tag; a bad value is 422; automation_count agrees with the filter; factory list paging limits and offset Pass
T7 A soft-deleted member drops out of automation_count and out of the filter Pass
T8 Delete is 409 with the live count while a member exists; 204 once cleared; the factory is then 404; a recreated slug gets a new id; the soft-deleted automation ends with factory_id = NULL in the DB Pass
T9 Each factory_id change appears in /v1/{id}/changes with factory_id in changed_fields Pass
T10 A prompt-preset automation takes a factory; enable, disable, rename, trigger change, timeout change and a manual dispatch all keep factory_id Pass
T11 A plugin-spec automation (script runner, local git plugin repo, SQLite) takes a factory; enable/disable, rename and a plugin PATCH keep factory_id; the filter and automation_count include it; delete is 409 while it is a member and 204 after it is soft-deleted Pass
T12 A draft dispatched as a test run creates a hidden DRAFT-state automation; assigning it to a factory works, it counts in automation_count and the filter, and it blocks the factory's delete (409) Pass (see observations)
T13 A disabled member still counts and blocks delete (409); after it is soft-deleted the delete succeeds (204) Pass

Schema and migration

# What was checked Result
T14 Missing or wrong key rejected; Bearer works Pass
T15 Single head 038. SQLite: indexes and ON DELETE SET NULL FK present; downgrade 037 then upgrade head round-trips. PostgreSQL 15: ix_automations_factory_id, unique ix_factories_org_slug and FK fk_automations_factory_id with confdeltype = n (SET NULL) present; downgrade 037 removes the table and column, upgrade head restores them, and a pre-existing automation survives both with factory_id NULL Pass
T16 PostgreSQL: deleting a factory row in raw SQL while an automation points at it leaves the automation with factory_id NULL Pass

Concurrency and git sync

# What was checked Result
T17 PostgreSQL: 30 rounds of DELETE /factories/{slug} racing a PATCH assignment to it. 29 rounds: delete 204 and assign 404. 1 round: delete 409 and assign 200. No round left a dangling link, or a deleted factory that still had a member Pass
T18 Real git-sync cycle (local bare repo, PostgreSQL): a factory-linked automation exports with no factory field in automation.yaml; after renaming it in git and syncing again, the name updates and factory_id is unchanged Pass

2. Covered only by the PR's unit tests (86 passed with PostgreSQL, 112 on SQLite only)

These need a second org or a non-admin user, which local mode doesn't have, so I did not exercise them live:

  • 403 for a creator without manage_automations on factory_id assignment, and for factory create, rename and delete
  • 404 when assigning an automation to another org's factory
  • test_route_permissions.py lists the new routes as manage-only

3. Not tested

  • Plugin-spec automations from a remote GitHub source (I used a local git repo, which local mode allows).
  • Delete-versus-assign with more than two concurrent requests per round, or across several service replicas.

4. Out of scope for this PR (per #725), so not tested

Membership is data only here: assign, filter, count, audit and delete rules. Nothing in this PR makes a factory affect how its automations run. That belongs to follow-ons:

Observations (not defects)

  • A draft that was dispatched as a test run leaves a hidden DRAFT-state automation. If it is assigned to a factory, it keeps counting toward automation_count and blocks deleting the factory with a 409 that names a live automation the user may not recognise. This matches the documented behaviour (counts every state, including drafts), but may be worth a UI note.
  • POST /v1 with factory_id in the body returns 422. Assignment is PATCH-only, which matches the spec, but API clients may be surprised.
  • Ünïcode Fáctory gets the slug n-code-f-ctory. Accented letters are dropped rather than transliterated. It is valid and URL-safe.

@jpshackelford jpshackelford left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: simplify the factory API and make it easier for the Factory UI to use

Manual testing found nothing wrong with what #725 asks for. The results are in my earlier comment. I can see how slugs and uniqueness constraints could help API ergonomics later, but they would be novel here. No other API route or UI route addresses a resource by slug, and automation names are not unique either. It is probably best to simplify now. If we want to introduce either later, we can do it consistently across the whole API surface.

Walking the Automations dashboard in OpenHands/OpenHands to see how a factory page would use these endpoints found three gaps:

  1. The slug is a second handle nothing else in the app uses. The UI addresses automations and conversations by id (automations/:automationId). A factory has an id and a slug, so clients map between them: the factory routes take the slug, while automation PATCH, the list filter and other services (Enterprise worklogs) take the id. Slug rules add more: derivation, 409 on duplicates (QA Two collides with qa two), accents dropped (Ünïcode Fáctory becomes n-code-f-ctory), and immutability.
  2. Nothing advertises factories. The UI gates features on /v1/capabilities features, so it would have to probe for a 404.
  3. Creating an automation in a factory takes two requests. The create endpoints reject factory_id, so the UI creates, then PATCHes.

Changes

  • Drop the slug. Address factories by id: GET, PATCH and DELETE /factories/{id}, with the UI route /factories/:factoryId/automations. Remove the slug column and its unique (org_id, slug) index from migration 038, the slug derivation and validation, the 409 on duplicate slugs, and the 422 on slug in PATCH. Update AGENTS.md, the F19 feature map and the tests. This revises #725's slug requirements and acceptance criteria, so update the issue too.
  • Add factories to _STATIC_FEATURES in capabilities_router.py.
  • Accept factory_id on POST /v1, /v1/plugin-spec, /v1/preset/prompt and /v1/preset/plugin.
    • Move the check now in PATCH into one shared helper. It needs manage_automations (403 otherwise), and an unknown or other-org factory returns 404.
    • Members can still create automations, but get 403 if they set factory_id.
    • Drafts accept and store factory_id automatically, because they share the request models. Do not copy it onto the hidden test-run automation. The dashboard hides those, so a factory carrying one would block its delete (409) with nothing visible to remove. Add a test for this.

Not changing

  • Factory names need not be unique. Automation names aren't either.
  • Factory delete stays "cancel unless empty". The UI removes members one at a time with PATCH factory_id: null. A cascade would also purge worklogs once #726 lands, so it needs its own decision.
  • Deferred to the list page design: bulk assign, per-factory health, and a "no factory" filter.

Tests

  • Factory routes by id: create, get, rename, delete; unknown id 404; slug no longer accepted or returned.
  • The migration on SQLite and PostgreSQL has no slug column, and still upgrades, downgrades and keeps existing automations.
  • factories in capabilities.
  • factory_id on each create endpoint: assigned, member 403, unknown 404, other-org 404.
  • A draft carrying factory_id: the hidden test run gets none, and the factory delete is not blocked.

Drop the factory slug (column, unique index, derivation, 409, routes now use
the id). Advertise `factories` in capabilities. Accept factory_id on POST /v1,
/v1/plugin-spec, /v1/preset/prompt and /v1/preset/plugin through one shared
authorize_factory_change helper also used by PATCH. Drafts store factory_id but
their hidden test run does not carry it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jpshackelford

Copy link
Copy Markdown
Member Author

Addressed the review in 283eb28:

  • Slug dropped. Factories are addressed by id (GET/PATCH/DELETE /v1/factories/{id}). Removed the slug column and its unique index from migration 038, the slug derivation module, the duplicate-slug 409, and slug handling in the schemas. A slug in a request body now gets 422 (unknown field). Names need not be unique.
  • factories added to _STATIC_FEATURES in capabilities_router.py.
  • factory_id on create for POST /v1, /v1/plugin-spec, /v1/preset/prompt and /v1/preset/plugin. One shared helper, authorize_factory_change, is used by those and by PATCH: 403 without manage_automations, 404 for an unknown or other-org factory. Members can still create automations without a factory.
  • Drafts store factory_id; the hidden test-run automation does not get it, so a draft cannot block a factory delete (test added).
  • Updated AGENTS.md, the F19 feature map and the PR description.

Tests: the factory, migration, route-permission, capabilities, draft, preset, router and plugin-spec tests pass on both PostgreSQL and SQLite. The plugin-spec create path is covered for 403/404 only; its success path needs a fetchable manifest.

Not done: updating #725 to drop the slug requirements.

openhands-agent and others added 2 commits November 13, 2026 23:09
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jpshackelford

Copy link
Copy Markdown
Member Author

Re-ran the affected manual families of the verify-automation map (control-automation map run --file Fnn --fresh --keep-going --record, each on its own fresh local run, at 0abc0b3):

Family Why Result
F19 Factories rewritten for ids and factory_id on create 7/7 pass
F03 Capabilities default features list now includes factories 17/17 pass
F05 Automation CRUD POST /v1 and PATCH share the new factory check 25/25 pass
F06 Presets /v1/preset/prompt and /v1/preset/plugin accept factory_id 16/16 pass
F09 Drafts drafts accept factory_id 20/20 pass

Two map corrections came out of it:

  • F03's exact features check was missing factories (it would have failed), fixed.
  • My first F19 rewrite deleted the factory it had just recreated by the old id, so F19.delete failed. Fixed to use the new id; F19 then passed on a fresh run.

Not covered by the manual map: factory_id on /v1/preset/* and /v1/plugin-spec is exercised only by the unit tests (PostgreSQL and SQLite), and the member 403 cases are unit-test only because the local API key holds manage_automations. The F19 create-in step drives POST /v1 only.

@jpshackelford
jpshackelford marked this pull request as ready for review November 16, 2026 13:02
@VascoSch92

Copy link
Copy Markdown
Member

@jpshackelford Sorry I don't get it: what is the difference between a factory and a plugin? Is one a subset of the other? or are really different? If yes, what are the differences?

@jpshackelford

jpshackelford commented Nov 16, 2026 •

Copy link
Copy Markdown
Member Author

@VascoSch92 A factory is a group of related automations for a single project / purpose. For example there may be an OpenHands OSS factory that contains all of the OSS automations or you might group by SDK, Canvas, Automations. We expect some new features to be factory level, for example some dashboards fit at that level.

Eventually we want a plugin to have metadata to represent the factory and it could contain automations or references to other automation plugins. This allows reuse. For example the same core automation plugins might be included in factories used by many different teams that have small team-specific variations.

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was posted by an AI agent (OpenHands).

Scope: The Factory model groups automations, so this belongs in OpenHands/automation. The change matches issue #725 as currently written (a name, an id handle, a nullable automations.factory_id — the slug requirement was dropped and the issue body updated to match).

Fresh review at head 0abc0b3 — the asks from the earlier review on 77a81a2 are all landed: factories are addressed by id with no slug (schemas reject slug via extra="forbid", migration has no slug column), factories is in /v1/capabilities features, factory_id is accepted on all four create endpoints through the one shared authorize_factory_change check, and drafts store factory_id while the hidden test-run automation never receives it (tested, including that a draft cannot block a factory delete).

What I verified in the code and workspace:

  • Authorization — factory reads need view_automations; create/rename/delete need manage_automations, and test_route_permissions.py pins the exact manage-only route set. authorize_factory_change applies 403 (creator exception deliberately bypassed) and org-scoped 404 at every write boundary: POST /v1, /v1/plugin-spec, /v1/preset/prompt, /v1/preset/plugin and PATCH /v1/{id}. The list filter is org-scoped by the base query, so a foreign factory id leaks nothing.
  • Concurrency — delete takes the factory row FOR UPDATE before counting live automations, assignment takes FOR SHARE; lock order (factory, then automation rows) is consistent in both paths, so delete-versus-assign cannot interleave or deadlock on PostgreSQL, and SQLite is correctly guarded with using_sqlite().
  • Migration — 038 is the single head after 037 (chain intact), uses batch mode so the FK works on SQLite, has a complete downgrade, and the migration test upgrades a database holding an automation and a run and asserts both survive with the SET NULL FK and index in place.
  • Audit and git sync — the delete detaches soft-deleted automations through the ORM so each detach is audited; the serializer neither writes nor reads factory_id, tested in both directions.
  • Contracts — GET /v1/factories pages newest-first with a unique (created_at, id) tie-breaker and returns total; FactoryCreate/FactoryUpdate forbid extra fields; AGENTS.md, the F19 feature map and F03's default capabilities check are updated, and test_committed_feature_map_owns_and_drives_every_route passes.

Tests I ran in the workspace (Docker is unavailable here, so PostgreSQL params error at fixture setup — those are covered by this head's green CI unit-tests/backend/windows-tests runs and the full-suite run documented in the PR): tests/test_factories.py 52 passed on the SQLite param, tests/test_factories_migration.py + tests/test_route_permissions.py 3 passed, tests/test_preset_router.py 213 passed, tests/test_verify_automation_skill.py 59 passed. No tracked files were modified.

The two decisions flagged for confirmation in the PR description (any factory_id presence requires manage_automations; limit/offset pagination like the webhook list) are consistent with the earlier review's explicit request and with existing API conventions.

No material correctness, security, compatibility, or acceptance-criterion findings.

✅ APPROVED

@VascoSch92 VascoSch92 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! I left a few comments.

The one that blocks merging is the migration number, which now clashes with #756 on main. The rest is about factory_id edge cases (template re-enable, null/unchanged values, drafts) and the locking around create and delete.

Comment thread openhands/automation/router.py Resolved
Comment thread openhands/automation/router.py Resolved
Comment thread openhands/automation/draft_schemas.py Resolved
Comment thread openhands/automation/factory_router.py Outdated
Comment on lines +89 to +92
Membership is a factory-level act: features built on factories hand a
factory's automations access to the factory's data. It needs
``manage_automations`` (403), even for the automation's creator, and the
factory must belong to the caller's org (404).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If membership is what gives an automation access to the factory's data, guarding only factory_id doesn't fully cover it: the creator escape hatch still lets a member change the entrypoint or tarball_path of their automation after an admin has put it in a factory.

Nothing is exposed today since nothing hangs off factories yet, but I think we should settle it before #726. Maybe any edit to an automation that is in a factory should need manage_automations.

@jpshackelford jpshackelford Nov 16, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1b1a1d4: any PATCH of an automation that is in a factory now needs manage_automations (403 otherwise). A creator can no longer change entrypoint, tarball_path or anything else on it once an admin has put it in a factory.

What I looked at beyond PATCH, given that #726/#752 will hand the run a factory token:

  • Dispatch, delete and cancel are unchanged. Dispatch takes no body and runs the stored code, the same as the schedule would, so it adds runs but no new way to change what receives the token. Delete and cancel only remove or stop things.
  • Git sync is the other path that changes what runs. _update_automation_from_git applies entrypoint, setup_script_path and the tarball from the sync repo to an existing automation with no per-automation check. Repo push access is the gate. I haven't changed that here. Do you want it handled in this PR (for example, skipping git-sync updates for automations in a factory), or tracked for feat(factories): add the run token, factory lookup and delete notification #752?

A member who created an automation also loses the ability to edit it once an admin adds it to a factory. I think that follows from membership granting access to the factory's data, and I listed it under "Decisions to confirm" in the description.

This reply was created by an AI agent (OpenHands) on behalf of @jpshackelford.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision on the git sync question: not guarded in this PR. Reasons:

  • Git sync is configured by an admin (manage_automations), and the sync code already treats push access to that repo as a trust boundary ("Anyone who can push to the repo could otherwise start an automation as its owner"). The PATCH rule closes the member-creator escalation. It cannot cover every path that decides what code runs: a plugin-spec automation resolves its plugin ref at run time, and an https:// tarball URL can change its content. A guard in git sync alone would suggest a closure that does not exist.
  • Skipping imports for factory members is not small. content_hash would not advance, so the import would retry every cycle, and a later export could overwrite a legitimate repo edit. It needs its own design, and nothing consumes factory data yet.

So in 6cbf6c3 AGENTS.md now says the rule is enforced at the API only, and that git sync, plugin refs and external tarball URLs are trusted as the source of an automation's code. For #752, the token should be treated as carrying the authority of the automation's code, and I'll say so in its docs. If worklogs raises the stakes I'll open a separate issue for a git sync guard. @VascoSch92, tell me if you disagree and want it here. Leaving this thread open for your answer.

This reply was created by an AI agent (OpenHands) on behalf of @jpshackelford.

Comment thread openhands/automation/factory_router.py Resolved
Comment thread openhands/automation/factory_router.py Resolved
Comment thread openhands/automation/factory_router.py Resolved
Comment thread openhands/automation/schemas.py Resolved
- Template re-enable naming a factory the automation is not in is 409
  instead of silently dropping factory_id.
- PATCH checks manage_automations for factory_id only when it changes.
- Any PATCH of an automation in a factory needs manage_automations.
- Drafts and /v1/validate report factory_id_403/404 validation errors.
- Assignment takes FOR KEY SHARE (SQLite: a no-op write) right before the
  write, so a rename is not blocked by a slow create.
- Delete is one conditional DELETE, closing the SQLite delete-vs-assign race.
- Lock mode is a Literal; FactoryUpdate subclasses FactoryCreate.

Co-authored-by: openhands <openhands@all-hands.dev>
jpshackelford pushed a commit to OpenHands/OpenHands that referenced this pull request Nov 16, 2026
Recorded against an automation service built from OpenHands/automation#751.

Co-authored-by: openhands <openhands@all-hands.dev>
@jpshackelford

Copy link
Copy Markdown
Member Author

The UI for this is in OpenHands/OpenHands#19800

jpshackelford pushed a commit to OpenHands/OpenHands that referenced this pull request Nov 17, 2026
A factory's automations dashboard now shows the same Add Automation menu
(create and import) as the Automate dashboard. Only the org-level Git Sync
stays hidden.

An automation created from a factory is created in it: `factory_id` is
threaded through useStartAutomationSetup, the setup draft and form (preset,
plugin and custom bodies, and saved drafts), the import dialog's create
call, and the plugin-spec request type. Without a factory no `factory_id`
key is sent. Creating or importing refreshes the automation lists and the
factories query so counts update. Saved drafts stay hidden in a factory view.

The factory sub-nav Automations item now uses the Automate sidebar's
AutomationsIcon, so SubPageNavItem accepts an SVG component as well as a
Lucide icon.

Depends on OpenHands/automation#751.

Co-authored-by: openhands <openhands@all-hands.dev>

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was posted by an AI agent (OpenHands).

Fresh review of head 1b1a1d4 (the head moved after the earlier approval of 0abc0b3, and the review threads from that round are addressed in it). Scope gate passes: factories are automation definitions and belong in this repository; the Agent Canvas UI work lives in its own repo.

Verified on this head:

  • Migration 039: single Alembic head after #756 took 038 (checked programmatically across all 39 migration files: one head, no dangling revisions). Batch mode and generic sa.Uuid keep SQLite and PostgreSQL aligned; upgrade/downgrade and a pre-existing automation + run are covered by tests/test_factories_migration.py.
  • Authorization: one shared check (authorize_factory_change) guards factory_id on all four create endpoints and on PATCH, with 403 for missing manage_automations and 404 for unknown/other-org factories; validation reports factory_id_403/factory_id_404 on drafts and /v1/validate. Resending the stored value is not a change, and any PATCH of an automation that sits in a factory needs manage_automations, closing the creator escape hatch. Cross-org access returns 404 everywhere.
  • Template re-enable no longer silently drops factory_id: all four create endpoints return 409 when the request names a factory the existing automation is not in.
  • Delete vs assign race: PostgreSQL uses FOR UPDATE (delete) against FOR KEY SHARE (assignment), which conflict without blocking renames; SQLite takes the write lock with a no-op update and makes the membership check atomic with a single conditional DELETE ... WHERE NOT EXISTS. The race test uses per-request sessions over 20 rounds on both backends and asserts the (204, 404) / (409, 200) invariant; soft-deleted members are detached through the ORM so the change audit records it.
  • Drafts store factory_id, validate it before dispatch, and the hidden test-run automation deliberately carries none.

Tests: focused run here (tests/test_factories.py, test_factories_migration.py, test_route_permissions.py, test_capabilities_router.py, SQLite parametrization): 76 passed, 1 skipped. The PostgreSQL-parametrized cases need the Docker fixture, unavailable in this workspace; they are covered by the green unit-tests check on this exact head and the full-suite run documented in the PR. CI is green on 1b1a1d4 (unit-tests, windows-tests, backend).

No material correctness, security, or compatibility findings. The unresolved human thread on this topic is outdated and answered by the blanket PATCH rule, and the git-sync boundary is documented and tested.

✅ APPROVED

@VascoSch92 VascoSch92 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround. I went through the second round and all eight threads are addressed. I also tried mutating the locks: making lock_factory_for_assignment a no-op, or dropping the delete's FOR UPDATE, makes the race tests fail on both databases, so they really cover it.

One thing outside this PR blocks the merge: main now has two 038s. #715 landed yesterday with 038_add_webhook_protocol_options.py on top of 037, next to #756's 038_add_run_deferred_until.py, and main's Run tests is red with Multiple head revisions. Whatever fixes main will probably take 039, so this one will need another renumber after that.

The inline comments are mostly about duplication, plus one edge case with draft automations.

Comment thread openhands/automation/schemas.py Resolved
Comment thread openhands/automation/router.py Resolved
Comment thread openhands/automation/router.py Resolved
Comment thread tests/test_factories.py Resolved
Comment thread tests/test_factories.py Resolved
Comment thread openhands/automation/factory_router.py Resolved
Comment thread openhands/automation/factory_router.py Resolved
Co-authored-by: openhands <openhands@all-hands.dev>
…ory checks

- PATCH refuses a factory change on a DRAFT-state automation (409), so a
  draft's hidden test run cannot hold a factory open or receive its token.
- authorize_factory_for_create and a None-tolerant lock_factory_for_assignment
  leave one unconditional line per step in each create endpoint.
- FACTORY_ID_DESCRIPTION replaces five pasted descriptions.
- create_factory flushes like rename_factory; _count_live builds its own query.
- Drop the unasserted outcomes set from the race test.
- AGENTS.md: draft rule and the git-sync/plugin-ref trust boundary.

Co-authored-by: openhands <openhands@all-hands.dev>

@malhotra5 malhotra5 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

@jpshackelford
jpshackelford merged commit 03823b4 into main Nov 18, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feat A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Factories: a named group that automations can belong to

5 participants