This directory holds the tracked repo-level automation code. These scripts are the implementation layer behind several content workflows, mirrors, and publication tasks. Start with the tracked workflow docs in ../workflows/README.md, then use the tracked skill bundles under ../workflows/skills/ when you need execution help. This file explains what each script in this folder does.
- Start here when you need to understand a root-level automation workflow.
- Treat blueprint hardening and alphaXiv validation as workflow-and-skill processes, not root-script entrypoints. They currently have no dedicated root script by design because they depend on judgment and evidence review rather than fixed automation.
- Use
npm run ...entry points when they exist instead of calling lower-level scripts by habit. - Treat
archive_utils.pyas a support module, not a user-facing entrypoint.
The default rule is: root-level scripts should be generic repo automation. A script may show a package name in CLI help text as an example when it still accepts --package or another generic selector. A script becomes a Phase 1 exception when it hardcodes one package, one series, or one migration context into its behavior.
Current root-script status: no accepted package-specific root-script exceptions.
Retired or moved exceptions:
| Script | Decision | Current Location |
|---|---|---|
stitch_binding_gap_slides.py |
Removed. The one-off Binding Gap slide utility was not a root workflow. | Retired. |
sync_operating_agents_substack.py |
Moved to the Operating Agents umbrella package because it is a series adapter, not root automation. | ../publications/autonomous-agents/scripts/sync_operating_agents_substack.py |
update_initial_status.py |
Removed. Historical lifecycle bootstrap is no longer a day-to-day command. | Retired. |
Non-exceptions:
check_repo_links.pycan name legacy paths because its job is to detect migration residue.- Generic package scripts such as
render_publication_video.py,upload_publication_tldrs.py, andupload_publication_shorts.pycan mention example package names if the actual package remains a CLI argument.
These scripts keep the publication lifecycle contracts, manifests, and refinement cycles coherent. They are the automation layer behind the tracked stage ids, but they do not replace the workflow docs that define how the work should actually be done.
Routine lifecycle records from this layer belong in ../reviewable-runs/logs/, not in the retained proof surface under reviewable-runs/examples/proofs/. For scaffolded essay_iteration, the log records the scaffold event while the canonical working packet lives beside the leaf draft inside the package.
Tracked docs:
| Script | Role | Normal Entry Point | Notes |
|---|---|---|---|
lifecycle.py |
Shared lifecycle constants, stage-to-workflow mapping, and workflow-frontmatter loader used by the other lifecycle scripts. | imported by other scripts | This is the contract layer, not a direct user command. |
harness.py |
Shared Codex-facing helpers for policy loading, context resolution, stage capability checks, launch packs, and harness contract generation. | imported by other scripts | Keep the machine-readable harness surface consistent. |
init_status.py |
Initializes status.yaml manifests for publication packages. |
.venv/bin/python scripts/init_status.py --package <package> |
Use --recursive when bootstrapping many README-backed packages. |
sync_status_schema.py |
Adds missing lifecycle stages and leaf defaults to existing manifests after schema changes. | .venv/bin/python scripts/sync_status_schema.py |
Maintenance helper when the tracked stage list changes. |
check_status.py |
Prints the leaf-package lifecycle matrix across publications/. |
.venv/bin/python scripts/check_status.py |
Fast visibility into current package state. |
run_stage.py |
Inspects one lifecycle stage, resolves its workflow contract, can execute or mark the stage state, and writes a lightweight run log. | npm run run -- --package <package> --stage <stage> |
This is the stage runner surface. The autogenerated log is operational trace, not retained proof. |
next_action.py |
Resolves the next valid lifecycle step for one package and emits a stable JSON/text payload for harnesses. | npm run stage:next -- --package <package> |
The Phase 1.5 next-step surface. |
init_package.py |
Bootstraps a minimal leaf package from an idea and seeds a runnable status.yaml. |
npm run package:init -- --idea "..." |
Creates the harness-first idea-to-package scaffold. |
init_essay_iteration_packet.py |
Seeds the package-local essay packet beside the live draft using the tracked essay-iteration templates. | .venv/bin/python scripts/init_essay_iteration_packet.py --package <leaf-package> |
Used by stage:run for scaffolded essay_iteration passes. |
check_essay_iteration_packet.py |
Validates the package-local essay packet for required headings, anchors, policy limits, and machine-checkable voice violations. | npm run check:essay-iteration -- --package <leaf-package> |
Run this before a weaker model claims the packet is ready for review. |
check_harness.py |
Validates that harness_contract.yaml matches the generated contract and that the policy/script surface is coherent. |
npm run check:harness |
Use this after lifecycle, workflow, or package.json changes. |
check_reviewable_run_logs.py |
Validates the current lightweight Reviewable Runs log schema for lifecycle logs and harness launch packs. | npm run check:reviewable-runs |
Use this after changing run logging, launch packs, or Reviewable Runs log policy. |
start_refinement.py |
Starts a new refinement cycle by bumping revision, optionally syncing from archive, and resetting selected stages. | .venv/bin/python scripts/start_refinement.py --package <package> --reset <stage,...> |
The main automation entrypoint for a new bounded rewrite cycle. |
sync_from_archive.py |
Copies a published archive snapshot back into a package, updates status.yaml with the resolved markdown path and slug, and backfills referenced archive assets into the package-local assets/ folder. |
.venv/bin/python scripts/sync_from_archive.py --package <package> |
Used directly and by start_refinement.py. |
import_substack_draft_to_archive.py |
Manual-import helper for unpublished Substack drafts: fetches draft metadata, writes a tracked markdown and HTML snapshot into publication_strategy_and_archive/substack_archive/, localizes any referenced local assets, and updates metadata.json. |
.venv/bin/python scripts/import_substack_draft_to_archive.py --post-id <id> --markdown <path> |
Use when a draft exists on Substack but is intentionally not yet in the public archive. |
audit_package.py |
Runs an advisory structural and evidence-drift audit for one package. | .venv/bin/python scripts/audit_package.py --package <package> |
Used directly and by run_stage.py during stage resolution. |
These scripts maintain the tracked Substack archive in publication_strategy_and_archive/ and validate the Eleventy mirror.
Tracked workflow:
| Script | Role | Normal Entry Point | Related Skill |
|---|---|---|---|
sync_substack_archive.py |
Canonical archive refresh: discovers public posts, localizes assets, writes markdown and HTML snapshots, updates metadata.json, and regenerates the root README publication table. |
npm run sync:substack |
substack-archive-sync |
generate_editorial_atlas.py |
Generates publication_strategy_and_archive/editorial_atlas/atlas_data.json from complete model-specific publication_strategy_and_archive/editorial_atlas/rubric_assessments_*.yaml files and can verify that the checked-in atlas still matches the rubric sources. |
npm run sync:editorial-atlas |
none |
editorial_atlas_utils.py |
Shared rubric-source validation and atlas derivation helpers used by the generator and mirror checks. | imported by other scripts | none |
archive_utils.py |
Shared paths and helpers for the archive manifest, post_map.yaml, and README table generation. |
imported by other scripts | substack-archive-sync |
check_mirror.py |
Structural validation for the archive, editorial atlas, and built mirror output. Checks metadata keys, rubric-derived atlas generation, archive files, asset references, generated README section, and built internal links. | npm run check:mirror |
substack-archive-sync |
check_repo_links.py |
Validation pass for stale old-package paths, broken relative links, and outdated internal GitHub repo links after the publications/ migration. |
npm run check:repo-links |
none |
e2e_mirror_smoke.py |
Runtime smoke test for the built mirror at both root and prefixed path layouts. | npm run check:mirror |
substack-archive-sync |
This workflow republishes the Operating Agents essay set from repo markdown to live Substack posts.
Tracked workflow:
| Script | Role | Normal Entry Point | Related Skill |
|---|---|---|---|
../publications/autonomous-agents/scripts/sync_operating_agents_substack.py |
Republishes the preface and six essay posts, preserves the embedded video block, rewrites published essay links to Substack URLs, and retries on Substack rate limits. | .venv/bin/python publications/autonomous-agents/scripts/sync_operating_agents_substack.py |
operating-agents-substack-sync |
This workflow updates one existing Substack draft from a canonical local markdown file without driving the browser UI.
Tracked workflow:
| Script | Role | Normal Entry Point | Related Skill |
|---|---|---|---|
sync_markdown_post_to_substack.py |
Syncs one markdown file into one existing Substack draft, uploads local images, preserves or injects the draft's uploaded video block when the markdown uses a #### Video TL;DR section, and can reuse a logged-in Chrome session over CDP to mint the auth cookie string. |
.venv/bin/python scripts/sync_markdown_post_to_substack.py --publication-url https://<subdomain>.substack.com --post-id <id> --markdown <file> --chrome-debug-url http://127.0.0.1:9222 |
none |
These scripts handle the local production stage for narrated slide videos before any YouTube upload or replacement step.
Tracked workflows:
../workflows/youtube-production/README.md../workflows/youtube-channel-ops/README.md../workflows/youtube-series-operating-agents/README.md
| Script | Role | Typical Use | Related Skill |
|---|---|---|---|
align_slides_to_transcript.py |
Drafts slide timing from OCR plus an untimestamped transcript and audio duration. | first-pass alignment | youtube-slide-video-pipeline |
align_slides_to_captions.py |
Drafts slide timing from OCR plus timed SRT captions and can blend OCR from complementary slide directories that share the same filenames. | first-pass alignment with captions | youtube-slide-video-pipeline |
build_slideshow_from_captions.py |
Drafts a cue sheet from timed SRT captions plus slide OCR, can blend complementary OCR slide decks, and can render a first-pass MP4 while interleaving multiple render slide directories that share the same filenames. | caption-first draft render | youtube-slide-video-pipeline |
map_slides_to_transcript_lines.py |
Drafts transcript line ranges from OCR text files and a numbered transcript. | line-range drafting | youtube-slide-video-pipeline |
line_map_markdown_to_cues.py |
Converts a reviewed markdown line map plus timed captions into the final cue-sheet JSON used for rendering. | verified cue generation | youtube-slide-video-pipeline |
render_audio_slides_video.py |
Generates a rough evenly spaced template or renders the final MP4 from a cue sheet. | template or final render | youtube-slide-video-pipeline |
render_publication_video.py |
Thin convenience wrapper that renders one package's final publication video from its prepared cues. | package-level final render | youtube-slide-video-pipeline |
create_publication_shorts_from_longform.py |
Slices a verified long-form MP4 into named short clips from a manifest and renders portrait Shorts variants. | package-level Shorts derivation from one long-form video | youtube-slide-video-pipeline |
convert_landscape_videos_to_portrait.py |
Converts landscape MP4 clips into portrait 9:16 versions by centering the original frame over a blurred vertical background, keeping output in a standard H.264/AAC web-delivery envelope. |
prep horizontal clips for Shorts-style upload | youtube-slide-video-pipeline |
probe_shorts_media.py |
Probes portrait Shorts masters for dimensions, audio, duration, and JSON-reportable preflight status. | verify Shorts media before upload | youtube-data-api |
scrape_youtube_watch_transcript.py |
Scrapes the visible watch-page transcript panel into JSON when you need a browser fallback for transcript inspection. | transcript fallback or debugging | youtube-slide-video-pipeline |
These scripts are thin wrappers around workflow or skill behavior. They remain generic enough for the root script surface.
| Script | Role | Notes |
|---|---|---|
list_youtube_videos.py |
Lists videos from the authenticated YouTube channel. | Thin wrapper around the YouTube skill script layer. Use ../workflows/youtube-channel-ops/README.md for the canonical process. |
upload_publication_tldrs.py |
Uploads TL;DR video variants from one package into a playlist. | Thin publication wrapper around the YouTube API helper stack; package selection is an argument. |
upload_publication_shorts.py |
Uploads portrait Shorts variants for one package, adds them to a Shorts playlist, links them back to the main long-form video, can schedule release times from a manifest, and writes an upload/readback report. | Thin publication wrapper around the YouTube API helper stack; package selection is an argument. Defaults to dry-run and preflights local media. |
upload_ad_hoc_shorts.py |
Uploads arbitrary local portrait clips, or converts landscape clips first, then links them to a long-form video and writes an upload/readback report. | Use for downloaded or manually edited clips outside a package's canonical Shorts folder. Defaults to dry-run and preflights local media. |
Some adjacent automation lives outside this folder:
- YouTube channel-management, caption download, temporary upload transcript recovery, playlist diagnostics, generic publication bundle uploads, Shorts maintenance, and replacement uploads live under
../workflows/skills/youtube-data-api/scripts/and are documented canonically in../workflows/youtube-channel-ops/README.mdand../workflows/youtube-series-operating-agents/README.md. - PDF slide export lives under
../workflows/skills/pdf-slides-to-substack-images/scripts/. - NotebookLM, blueprint hardening, and alphaXiv validation are documented canonically under
../workflows/and do not use this rootscripts/folder as their main implementation surface.
If you only need the normal repo entrypoints, start with:
npm run sync:substack
npm run sync:editorial-atlas
npm run check:mirror
npm run check:repo-links
PYTHONUNBUFFERED=1 .venv/bin/python publications/autonomous-agents/scripts/sync_operating_agents_substack.pyFor YouTube production work, start with ../workflows/youtube-production/README.md before calling the lower-level scripts directly.