Skip to content

Make BrightBean scheduling infrastructure reproducible - #128

Open
plantsandchairs wants to merge 6 commits into
brightbeanxyz:mainfrom
plantsandchairs:fix-brightbean-llms-setup
Open

Make BrightBean scheduling infrastructure reproducible#128
plantsandchairs wants to merge 6 commits into
brightbeanxyz:mainfrom
plantsandchairs:fix-brightbean-llms-setup

Conversation

@plantsandchairs

Copy link
Copy Markdown

What

Scheduling-infra hardening from the 2026-07-18 kimi session, on top of the fork-main Meta OAuth fixes already awaiting upstream:

Infra (this PR's purpose):

  • 16121b5 .dockerignore (excludes .env, .git, .context from images) + fly.toml process groups (app + worker) + health checks + restart policy + tests/test_deployment_config.py
  • d6eee1f Fly health check middleware + asset ingestion script

Also included (already on plantsandchairs/main, not yet upstream):

  • f87d17f fall back to Instagram Business app credentials for Meta OAuth
  • a3c92ed use current Meta scope names (deprecated ones removed)
  • 0f9d429 merge origin/main

Verification

  • 1,043 tests passed; fly config validated; studio QA passed
  • Docker image confirmed free of .env, .git, .context
  • Free local Qwen review of the infra diff: APPROVE

Notes

After merge: prove the managed worker, then retire the old standalone Fly worker machine (standalone machines are not managed by process-group deploys).

plantsandchairs and others added 5 commits July 14, 2026 22:05
The existing PnC Meta app is configured through the Instagram API /
Instagram Business use case, not a separate Facebook Login app. Meta
credential resolution now falls back to PLATFORM_INSTAGRAM_APP_ID/SECRET
when the Facebook-named vars aren't set, so the existing app works
without a destructive secret rotation. Already deployed and health-checked
live prior to this commit; adding fly.toml so the deploy config is
version-controlled.

Co-Authored-By: Claude <claude-sonnet-5> <noreply@anthropic.com>
Meta renamed Instagram Graph API permissions with an "instagram_business_"
prefix (instagram_basic -> instagram_business_basic, etc.) as part of the
Instagram/Facebook Login platform unification. The old names are now
rejected outright ("Invalid Scopes") during the OAuth consent screen.
instagram_login.py already used the current names; instagram.py (the
provider actually reachable via this app's Facebook Login for Business
product) still had the old ones. Brought it in line.

Co-Authored-By: Claude <claude-sonnet-5> <noreply@anthropic.com>
Move the web and scheduler into explicit Fly process groups, protect the Docker build context, and validate the deployment contract before replacing the unmanaged worker.

Constraint: Preserve the live unmanaged worker until R2 and the managed replacement are proven.

Rejected: Deploying with 512 MB worker memory | FFmpeg workloads are configured for concurrent transcodes.

Confidence: high

Scope-risk: narrow

Directive: Do not remove machine 2862960a65d4d8 until the managed worker passes a gated publish-cycle smoke test.

Tested: 1043 pytest tests, Ruff, flyctl config validation, Docker secret and git exclusion, Django system check, worker command discovery, studio QA, local Qwen review.

Not-tested: R2 integration and production account OAuth remain blocked by rotated credentials and interactive consent.
@plantsandchairs
plantsandchairs requested a review from JanSchm as a code owner July 19, 2026 05:19

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6eee1f372

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ingest_assets.py
asset_type = metadata.get("type", "image" if asset["filename"].endswith((".png", ".jpg", ".jpeg", ".webp")) else "video")

# Check if already exists in DB
if MediaAsset.objects.filter(sha256=asset["sha256"]).exists():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the actual MediaAsset fields for ingestion

When any source directory contains a media file, this query runs before the --dry-run branch and raises FieldError because apps.media_library.models.MediaAsset does not define a sha256 field. The real ingest path has the same schema mismatch (sha256/project are passed to create() while required fields like filename and media_type are omitted), so the advertised ingestion script cannot create or deduplicate records against the current model.

Useful? React with 👍 / 👎.

Comment on lines +22 to +24
assert config["processes"] == {
"app": "gunicorn config.wsgi:application --bind 0.0.0.0:${PORT:-8000} --workers 2 --threads 2",
"worker": "python manage.py process_tasks",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the Fly process assertion in sync

The added test parses fly.toml and asserts the app process is exactly gunicorn config.wsgi:application --bind 0.0.0.0:${PORT:-8000} --workers 2 --threads 2, but the config introduced in this same commit sets it to sh -c "gunicorn config.wsgi:application --bind 0.0.0.0:$PORT --workers 2 --threads 2". As written, test_fly_runs_web_and_background_worker_process_groups fails immediately, so CI will reject the deployment-config change even though the runtime config may be intentional.

Useful? React with 👍 / 👎.

@JanSchm

JanSchm commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

@plantsandchairs Can you tell me the bigger idea behind your merge requests, so I have a better understanding in what feature direction you intend to application.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants