Skip to content

AGENTS.md

Latest commit

 

History

History
124 lines (105 loc) · 8.34 KB

AGENTS.md

File metadata and controls

124 lines (105 loc) · 8.34 KB
Raw

Project agent memory

This file is the project's committed home for project-intrinsic agent knowledge: build, test, release, architecture, and sharp-edge notes that should travel with the code. CLAUDE.md is a tracked symlink to this file.

  • Add durable project-specific notes here as they are discovered through real work.
  • Point to the authoritative file/command instead of copying what the codebase already shows.

What this repo is

foursight-core is not a standalone app: it is the shared core library behind the foursight (Fourfront) and foursight-cgap (CGAP) monitoring apps. It is a serverless AWS Chalice application (published to PyPI as foursight-core) that runs scheduled checks against a Portal/ElasticSearch environment and exposes both a legacy server-rendered UI and a newer React UI/API.

  • The central class is AppUtilsCore (foursight_core/app_utils.py:72), which multiply-inherits ReactApi and Routes. Downstream apps subclass it (their chalicelib_cgap/chalicelib_fourfront AppUtils) and instantiate it via AppUtilsCore.singleton(...) (app_utils.py:2105). A local AppUtils(AppUtilsCore) alias exists for backward-compatible imports.
  • foursight_core/check_setup.json is {} here on purpose — the real check-setup file is supplied by the downstream app and located at runtime (see CheckHandler, check_utils.py).
  • Root app.py is a minimal packaging/test stub (a bare Chalice app). Real route/app assembly lives in foursight_core/app.py (the shared Chalice app object with request helpers) plus foursight_core/app_utils.py.

Layout / key modules

Python package foursight_core/:

  • app_utils.py — largest module; AppUtilsCore, auth (check_authorization), legacy UI rendering (Jinja templates/), config-file location, singleton. Real app entry surface.
  • check_utils.py — CheckHandler: loads/validates the check-setup, discovers check modules, runs checks.
  • decorators.py — Decorators: the @check_function / @action_function decorators, the CheckResult/ActionResult factories, in-process check timeout (CHECK_TIMEOUT, < 900s Lambda cap), and a decorator registry for surfacing check kwargs without a run.
  • run_result.py — CheckResult/ActionResult base classes (result storage/retrieval semantics).
  • schedule_decorator.py — @schedule wrapper over Chalice Cron; supports per-stage schedule dicts and disabled stages (see module docstring).
  • deploy.py — Deploy: generates gitignored .chalice/config.json and runs chalice deploy; holds CONFIG_BASE (stages dev/prod, lambda memory/timeout).
  • environment.py / stage.py / identity.py — env/bucket resolution, chalice stage, and applying the Global Application Configuration (GAC) secrets to os.environ (identity.py documents the GAC→Foursight key mapping).
  • fs_connection.py / s3_connection.py / es_connection.py / abstract_connection.py — connections to Fourfront (FF) keys, S3 results bucket, and ElasticSearch; also optional Redis session tokens.
  • routes.py + route_prefixes.py — legacy Chalice routes; route_prefixes handles the /api prefix quirk under chalice local (CHALICE_LOCAL).
  • boto_s3.py / boto_sqs.py / sqs_utils.py — AWS clients and the SQS check-runner queue.
  • checks/ — the checks bundled with core (ECS, codebuild, scaling, access-key expiration) plus checks/helpers/confchecks.py, the template a downstream project copies to bind the decorators to its own foursight_prefix.
  • templates/ — Jinja templates for the legacy UI.

React lives in two distinct trees — do not confuse them:

  • react/ (repo root) — the React frontend source (Create React App, react/src/{api,ui,...}). Dev: make react-run-local (npm start, port 3000).
  • foursight_core/react/api/ — the Python backend for the React app: react_routes.py (route table), react_route_decorator.py (the @route decorator: auth default-on, CORS for local, path prefixing), react_api*.py, plus AWS/auth helpers (cognito.py, auth.py, jwt_utils.py, aws_*.py, gac.py, envs.py).
  • foursight_core/react/ui/ — the built frontend, generated and copied in by scripts/react_build.sh (make react); checked into git. Do not hand-edit; rebuild instead. The build de-hashes main.*.js/css to stable main.js/main.css and sets PUBLIC_URL=/api/react.

Config / deployment conventions

  • Dependencies via Poetry (pyproject.toml, poetry.lock); poetry.toml pins virtualenvs.create = false (see release sharp edge below). Python >=3.8.1,<3.13.
  • .chalice/ holds deployed.json and policy-{dev,prod}.json; config.json is generated by deploy.py and gitignored. Chalice stage comes from the chalice_stage env var (default dev).
  • Secrets/config are pulled from the AWS Global Application Configuration (GAC) at runtime via identity.py — not hardcoded env vars.

Build / test / lint commands

  • make build (installs poetry + deps, builds React) / make build-noreact / make build-for-ga.
  • make test → pytest -vv -m "not integrated" && pytest -vv -m "integrated". Markers: unit, integrated, integratedx (see pytest.ini). conftest.py sets up a simulated env with mocked boto/EnvUtils — most unit tests run against it.
  • make lint → flake8 foursight_core and flake8 tests (config in .flake8).
  • Tests in tests/ mirror the package (test_react_*, test_check_*, test_deploy.py, …); tests/test_checks/ holds sample checks.
  • Docs (Sphinx) under docs/, published via .github/workflows/main-deploy-docs.yml.

Sharp edges (AWS / integration / operational)

  • In-process check timeout is 870s (decorators.py), deliberately under the 900s Lambda ceiling — keep it below.
  • Two Chalice apps / two UIs run side by side (legacy + React). React-specific code is confined to foursight_core/react/; the only crossovers into legacy code are the ReactApi mixin in AppUtilsCore and a React branch in auth0_callback (routes.py).
  • chalice local prefix quirk: locally there is no /api prefix, so route_prefixes.py injects it when CHALICE_LOCAL is set (upstream Chalice issue #838). CORS is enabled only for local dev.
  • Auth spans Auth0 (/callback) and AWS Cognito (react/api/cognito.py); the @route decorator defaults to requiring authorization — pass authorize=False for public routes.

Automatic tag-and-publish-on-master release workflow

.github/workflows/main-CI.yml's publish job (needs: build, runs only on push to master) reads the version from pyproject.toml (poetry version -s) and independently checks both for its git tag and for an existing PyPI release (curl to https://pypi.org/pypi/foursight-core/<version>/json; only HTTP 200/404 are treated as present/absent, anything else fails the job closed). It creates the tag only if missing and publishes only if PyPI doesn't already have the version, in the same job run. It deliberately does not rely on .github/workflows/main-publish.yml's tag-triggered on: push: tags event, because GitHub Actions does not start a new workflow run from a tag pushed with the default GITHUB_TOKEN (anti-recursion rule) — main-publish.yml remains for manual/ workflow_dispatch publishing only.

make build-for-ga uses POETRY_VIRTUALENVS_CREATE=true poetry install, never poetry config --local virtualenvs.create true — the latter rewrites the tracked poetry.toml ([virtualenvs] create = false), dirtying the release checkout and making publish-to-pypi's clean-tree check fail. The workflow also asserts git diff --exit-code right after dependency install so a future regression here fails loud with the filename. publish-to-pypi (from dcicutils.scripts.publish_to_pypi) tolerates exactly one dirty file named gitinfo.json, which is why the publish job (like main-publish.yml) writes foursight_core/gitinfo.json with the release commit info right before publishing — any other tracked-file diff still fails the check.

Maintaining this file

Keep this file for knowledge useful to almost every future agent session in this project. Do not repeat what the codebase already shows; point to the authoritative file or command instead. Prefer rewriting or pruning existing entries over appending new ones. When updating this file, preserve this bar for all agents and keep entries concise.