Skip to content

Bug Report: MoveTables create allows same source and target keyspace, creating an uncompletable workflow whose cancel drops the source table #20368

Description

@lizztheblizz

Overview of the Issue

MoveTables create does not validate that the source and target keyspaces differ. SwitchTraffic does (via validateMoveTablesSourceKeyspace), but only after the workflow already exists. As a result, a workflow created with --source-keyspace X --target-keyspace X on the same table:

  • succeeds at create time (as long as the target table is empty),
  • can never be completed (switchtraffic rejects it with FAILED_PRECONDITION), and
  • when cancelled with --keep-data=false, issues DROP TABLE against what is physically the source table, deleting source data.

The guard that rejects source == target already exists, but is wired only into the SwitchTraffic path, not into create. Validating it at create would make this whole failure mode unreachable.

First reported by @chrisplim.

Reproduction Steps

  1. In an unsharded keyspace ks, start with an empty table t.
  2. Create a MoveTables workflow with source == target:
    vtctldclient MoveTables --workflow w --target-keyspace ks create --source-keyspace ks --tables t
    Result: the workflow is created successfully.
  3. Try to complete it:
    vtctldclient MoveTables --workflow w --target-keyspace ks switchtraffic
    Result: rpc error: code = FailedPrecondition desc = workflow ks.w is invalid: MoveTables source keyspace matches target keyspace (ks). The workflow can never be switched.
  4. Cancel it:
    vtctldclient MoveTables --workflow w --target-keyspace ks cancel --keep-data=false
    Result: cancel issues drop table ks.t. Because source and target are the same keyspace and table, this drops the source table.

Note: if t is non-empty, create instead fails the "all target tables are empty" check, so the trap only springs when the target table is empty.

Root cause (analysis)

  • The source == target guard validateMoveTablesSourceKeyspace (in go/vt/vtctl/workflow/server.go) has a single caller: WorkflowSwitchTraffic.
  • moveTablesCreate (same file) performs no equivalent check.
  • The cancel path is WorkflowDelete -> dropTargets -> removeTargetTables -> drop table <db>.<table> (in go/vt/vtctl/workflow/traffic_switcher.go). The target DB name is derived per keyspace (TabletDbName, go/vt/topo/topoproto/tablet.go), so when the source and target keyspaces are identical, the dropped table is the source.

Proposed fix

Invoke the existing source == target validation (or an equivalent returning the same FAILED_PRECONDITION) at the start of moveTablesCreate, so the degenerate workflow can never be created. Scope to create only; cancel's drop behavior is correct for normal workflows.

Binary Version

Vitess v22 (release-22.0)

Operating System and Environment details

Logic bug, environment-independent. Reproduced on a standard Vitess cluster with a single unsharded keyspace. Not OS- or architecture-specific.

Log Fragments

# switchtraffic on the same-keyspace workflow:
rpc error: code = FailedPrecondition desc = workflow ks.w is invalid: MoveTables source keyspace matches target keyspace (ks)

# cancel --keep-data=false then drops the source table:
drop table ks.t

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions