Skip to content

fix(sync): run warehouse-backed syncs on postgres#179

Open
jonathanhaaswriter wants to merge 6 commits intosnowflake-app-state-postgres-20260327from
snowflake-sync-runtime-postgres-20260327
Open

fix(sync): run warehouse-backed syncs on postgres#179
jonathanhaaswriter wants to merge 6 commits intosnowflake-app-state-postgres-20260327from
snowflake-sync-runtime-postgres-20260327

Conversation

@jonathanhaaswriter
Copy link
Copy Markdown
Collaborator

Summary

  • switch direct/API/scheduled sync paths to the generic warehouse interface and make Postgres the default warehouse when Postgres DSNs are configured
  • add SQL dialect helpers so sync upserts, relationship persistence, schedule storage, and change-tracking tables work on Postgres-backed warehouses
  • update warehouse/status health surfaces and regression tests to cover the Postgres-backed sync runtime

Validation

  • go test ./...
  • make verify (race tests completed; remaining verify targets run manually because the installed golangci-lint binary targets Go 1.25)
  • GOTOOLCHAIN=go1.26.1 go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.8.0 run --timeout 5m ./...
  • make api-contract-compat
  • make cloudevents-contract-compat
  • make report-contract-compat
  • make entity-facet-contract-compat
  • make agent-sdk-contract-compat
  • make graph-ontology-guardrails

@jonathanhaaswriter
Copy link
Copy Markdown
Collaborator Author

I think there are two issues here:

  • JOB_DATABASE_URL now implicitly flips the warehouse backend to Postgres, so setting it just for app-state/jobs can silently disable Snowflake unless WAREHOUSE_BACKEND is also forced
  • the post-sync toxic-combination scan is now routed through application.Warehouse, but the implementation still uses Snowflake-specific SQL, so Postgres-backed syncs will just warn/skip that analysis

Can we decouple warehouse selection from JOB_DATABASE_URL and either port or gate the toxic-combination scan for Postgres?

@jonathanhaaswriter jonathanhaaswriter force-pushed the snowflake-app-state-postgres-20260327 branch from bd7bddb to 1a95faa Compare March 28, 2026 01:19
@jonathanhaaswriter jonathanhaaswriter force-pushed the snowflake-sync-runtime-postgres-20260327 branch from 08d9da8 to 337654a Compare March 28, 2026 01:41
@jonathanhaaswriter jonathanhaaswriter force-pushed the snowflake-app-state-postgres-20260327 branch from 1a95faa to 51e755a Compare March 28, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant