Add MySQL proof of concept#3997
Draft
shaunandrews wants to merge 9 commits into
Draft
Conversation
shaunandrews
changed the title
Adds a `studio convert --to mysql` command that converts an existing SQLite-backed Studio site to real MySQL in place, reusing the MySQL POC's provisioning, config-swap, and boot-check building blocks. Sequence (all reversible until the boot-verify gate passes): 1. Backup db.php, the sqlite-database-integration mu-plugin, wp-config.php, and a config snapshot to a timestamped dir; .ht.sqlite is never deleted. 2. Export the SQLite DB to a MySQL-shaped .sql via the existing AST-driver exporter (full single-file dump — keeps wp_users/wp_usermeta). 3. Provision the per-site MySQL DB, matching the exporter's per-table collation (utf8mb4_0900_ai_ci) so the schema default and imported tables agree. 4. Import the dump INTO MySQL via a new streaming helper. 5. Remove the SQLite integration and write MySQL DB_* constants. 6. Verify WordPress boots on MySQL (is_blog_installed) as a hard accept gate. 7. Roll back fully to the SQLite site on any failure. New/changed pieces: - `importSqlFileIntoMysql` streams a .sql on the mysql client's stdin and relaxes the session sql_mode (NO_ENGINE_SUBSTITUTION) so WordPress's legitimate `0000-00-00` datetime defaults load under MySQL 8 strict mode — the same accommodation `wp db import`/mysqldump round-trips rely on. - `runMysqlCommand` gains optional stdin-file streaming. - `provisionMysqlDatabase` gains an optional collation override (default unchanged for the create path). - `getWpConfigTransformerPath` exposes the bundled transformer path. - The convert command reserves known site/MySQL ports before allocating the new MySQL port so it can't collide with the site's own server port. Proven end to end by converting a real 274 MB SQLite runtime to MySQL 8.4 and serving it over HTTPS with content intact (posts/pages/options/users match the pre-conversion baseline). ## AI assistance - **AI assistance:** Yes - **Tool(s):** Claude Opus 4.8 via Claude Code - **Used for:** Implementing the convert command, the MySQL import helper, and the collation/sql_mode/port-reservation handling; root-causing the import and boot-check behavior against a real runtime.
Two linked bugs let a freshly-converted MySQL site break on the next `studio start`, even though the conversion itself imported cleanly: 1. paths.ts hardcoded the daemon home to ~/.studio, ignoring the DEV_CONFIG_DIR sandbox that the rest of the CLI honors via getConfigDirectory(). A sandboxed CLI (the MySQL POC, or start:test) therefore connected to the default ~/.studio daemon, which for the POC is the stock MySQL-unaware Studio.app daemon — it never launched mysqld and drove the isolated site through the wrong process manager. Derive PROCESS_MANAGER_HOME from getConfigDirectory() so the daemon socket lives alongside cli.json/shared.json. When DEV_CONFIG_DIR is unset getConfigDirectory() returns ~/.studio, so normal installs are byte-for -byte unchanged; an explicit STUDIO_PROCESS_MANAGER_HOME still wins. This finishes what #2958 set out to do ("redirect the entire config directory") but which only covered well-known-paths.ts. 2. ensureWpConfig() silently fell back to DB_NAME='wordpress' whenever the engine config was absent. Combined with (1), a start via the wrong daemon rewrote a converted site's wp-config.php back to 'wordpress', severing it from its studio_<id> database. Add a guard that refuses to overwrite an existing non-default DB_NAME when no MySQL engine config is provided, instead of corrupting the config. Proven on the real studio-native-local-runtime site: full stop -> start round-trip now boots on MySQL 8.4.10, serves HTTP 200, and keeps DB_NAME=studio_1322c571c44e43ca80826cd1 with all content intact. ## AI assistance - **AI assistance:** Yes - **Tool(s):** Claude Opus 4.8 via Claude Code - **Used for:** Root-causing the two bugs from source, writing the fixes and the guard unit test, and verifying the stop/start round-trip on the converted runtime.
The native-PHP runtime hardcoded its worker pool to 4, which is the concurrency ceiling for a site — the number of requests it can serve simultaneously. Nothing let a developer change it: not a flag, config field, or env var. Allow STUDIO_PHP_WORKER_POOL_SIZE to override the default of 4, so a larger machine can raise throughput (useful for exercising concurrent request paths like async fanout) and a constrained one can lower memory use. Invalid or non-positive values fall back to 4. Env var rather than a persisted per-site config field for now — kept deliberately small; it can be promoted to a `studio config` setting later if it earns it. The name is STUDIO_PHP_WORKER_POOL_SIZE (not NATIVE_PHP) to avoid reading as the unrelated "Studio Native" product; it configures Studio's own native-PHP runtime. Verified on studio-native-local-runtime: STUDIO_PHP_WORKER_POOL_SIZE=8 spawns 8 workers and serves HTTP 200; an unset value still defaults to 4. ## AI assistance - **AI assistance:** Yes - **Tool(s):** Claude Opus 4.8 via Claude Code - **Used for:** Implementing the env-var override and verifying worker count on the running site.
WordPress fires its own loopback requests — Action Scheduler's async queue runner, wp-cron, REST-to-self — as fire-and-forget: a non-blocking request with a ~0.01s timeout, where the client disconnects immediately by design and the PHP worker is meant to keep running and do the work. On the native-PHP runtime this never worked, which stalls every async workload (Action Scheduler jobs, and async agent fanout in particular). Two distinct causes, both fixed here: 1. `.local` custom domains resolve via multicast DNS (RFC 6762) with a ~5s timeout on macOS, even when the name is in /etc/hosts. A self-loopback to the site's own `.local` URL therefore spends 5-10s in DNS and blows past the 0.01s dispatch timeout, so the request never even connects. Add a native-PHP mu-plugin (0-loopback-dns-fast-path.php) that pins the site's OWN host to 127.0.0.1 via CURLOPT_RESOLVE for requests WordPress makes to itself. This does not change how `.local` resolves for the browser or anything external — it only short-circuits the site addressing itself, which is what 127.0.0.1 is for. Threaded the site host+port through writeStudioMuPluginsForNativePhpRuntime and its three call sites (server start, blueprint start, WP-CLI). 2. Both proxy layers (the HTTPS front door in proxy-server.ts and the worker pool proxy in php-server-child.ts) aborted the upstream PHP-worker request when the client disconnected. For a fire-and-forget request that means the worker is killed the instant it is kicked off. Decouple the upstream request's lifetime from the client: forward the body manually and, on a client abort, finish sending to the worker rather than tearing it down so it runs to completion. proxy-server.ts drops the http-proxy dependency for this path in favor of a raw http.request forward. Loopback latency drops from 5-10s to ~0.09s; a fire-and-forget dispatch now lands and the worker survives client disconnect. Verified end to end: a WordPress async fanout job self-drives to completion on this runtime, where before every branch dispatch died at DNS. ## AI assistance - **AI assistance:** Yes - **Tool(s):** Claude Opus 4.8 via Claude Code - **Used for:** Root-causing both failures from source and live runtime behavior, implementing the fixes, and building a deterministic loopback repro to verify.
# Conflicts: # apps/cli/commands/site/create.ts # apps/studio/src/modules/cli/lib/cli-site-creator.ts # packages/common/lib/database-engine.ts # packages/common/lib/mysql-binary-cdn-metadata.json # packages/common/lib/mysql-binary-metadata.ts
…n heartbeat Three robustness gaps in the native-PHP MySQL runtime, each fixed and verified against the live studio-native-local-runtime site. 1. `wp db` was broken: WP-CLI's db subcommands shell out to a bare `mysql`/ `mysqldump` resolved on PATH, but Studio bundles the MySQL client without installing it globally, so every `wp db *` died with `env: mysql: No such file or directory`. Prepend the bundled client's bin/ dir to the WP-CLI child's PATH for MySQL-engine sites (SQLite sites are untouched). Verified: `wp db query "SELECT 1"` and `wp db size` now return results. 2. The bundled mysqld ran with the default `time_zone=SYSTEM`, inheriting the host's local zone (e.g. EDT, UTC-4). WordPress and Action Scheduler store all `*_gmt` columns as PHP-computed UTC, so the database clock disagreed with the stored data by the host's UTC offset: `NOW()` returned local time while the rows were UTC. That makes UTC timestamps look hours in the future to any query comparing against the DB clock, and corrupts any code mixing MySQL `NOW()` with a `_gmt` column. Launch mysqld with `--default-time-zone=+00:00` so the server clock is UTC. Verified after a stop/start: `@@time_zone=+00:00` and `NOW()` == `UTC_TIMESTAMP()` == PHP `gmdate()`. 3. This runtime is a bare `php -S` worker pool with no cron ticker, so WordPress's `wp-cron.php` — the universal entrypoint a production host drives via system-cron or traffic — never fired on a schedule. Any async workload (Action Scheduler jobs, agent fan-out branches) only advanced when a user request happened to arrive, so branches could strand PENDING indefinitely on an idle site. Add a generic 60s WP-Cron heartbeat in the server child that loopbacks to `/wp-cron.php?doing_wp_cron` over the internal HTTP proxy (with the canonical Host header, avoiding the self-signed `.local` cert and the canonical redirect). The runtime fires generic WordPress cron; Action Scheduler drains as a consequence of being a well-behaved WP-Cron citizen — no plugin-specific coupling in the runtime. Verified: the "Starting WP-Cron heartbeat" log appears on start and a worker connection lands each 60s window with zero external traffic. The Action Scheduler future-dated-timestamp symptom was investigated and is NOT a runtime clock-skew bug in the claim path: AS compares PHP-UTC strings on both sides and never uses MySQL `NOW()`, so the tz fix above is correctness hygiene, not the drain fix. The remaining branch-drain edge (a stuck in-progress action holding the single claim slot while `pending_branch_count()` collapses concurrency back to 1, under agents-api's intentional 3600s long-branch reaper window) is app-layer (agents-api) behavior, not a Studio runtime defect. ## AI assistance - **AI assistance:** Yes - **Tool(s):** Claude Opus 4.8 via Claude Code - **Used for:** Root-causing all three gaps from source and live runtime evidence, writing the fixes, and verifying each via the actual CLI/DB/HTTP behavior on the running site.
shouldUsePrimaryWorker() pinned every non-GET/HEAD/OPTIONS request to worker 0. That method-based rule was an over-broad stand-in for "stateful admin request", but it swept in all of WordPress's own POST loopbacks — most importantly Action Scheduler's async queue runner (a POST to admin-ajax.php) and wp-cron. Async fanout fires N concurrent loopback POSTs to wake N workers for N branches; with the method pin all N serialized on worker 0, capping fanout concurrency at 1 regardless of pool size. Pin worker affinity by request PATH only. The one route that genuinely needs single-worker affinity is phpMyAdmin (file-based session store scoped to one worker via STUDIO_PHPMYADMIN_SESSION_PATH), and its existing /phpmyadmin path pin already covers all its methods. WordPress itself is stateless across the pool — sessions live in the shared MySQL database, not on any worker's disk — so its POSTs are safe to load-balance like GETs. Proven at the proxy level: 8 concurrent loopback POSTs now reach 8 distinct worker PIDs (was 1 before), matching GET behavior. AI assistance: Yes; Tool(s): Claude Code (Opus 4.8); Used for: root-cause analysis, the fix, and proxy/end-to-end concurrency proofs.
Contributor
Fanout concurrency: proxy POST-pin fixed; a second upstream constraint isolatedFix in this commit (proxy layer)
Now affinity is pinned by path only. phpMyAdmin (the one genuinely stateful route — file-based session store scoped to one worker) keeps its existing Proxy-level proof (8-worker pool,
Second constraint isolated (upstream — NOT the proxy, NOT this repo)With the proxy fixed, a 5-page Isolation proof (same runtime, same 8-worker pool,
So the branch-fanout mechanism, the AS concurrency policy, and this proxy fix are all correct; the remaining cap is agents-api constructing an HTTPS/:443 loopback target that doesn't match a plain-HTTP native-PHP worker-pool runtime. That fix belongs upstream in agents-api's AI assistance
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Changes
What Works Now
databaseEngine=mysql.~/.studio/mysql-bin/8.4.10/.~/.studio/mysql-data/<site-id>/.wp-config.phpwith real MySQL credentials.Big Remaining Work
Platform binaries
darwin-arm64..exeand.zip, but are untested.Binary delivery policy
Lifecycle hardening
mysqldafter app crashes.Feature boundaries
User experience
WP-CLI / developer tooling
studio wp ...command in this branch.Tests and CI
Upgrade story