chore(docker): stack local evo-flow + clickhouse para tracking de eventos (EVO-1571)#114
Conversation
…ntos (EVO-1571) Adiciona docker-compose.evoflow.yaml (override sobre o docker-compose.yml principal) que sobe clickhouse + a app evo-flow (RUN_MODE=api + QUEUE_MODE=direct + WRITE_MODE=ch-sync) na rede default do projeto, integrado ao CRM. O evo-flow auto-cria o DB Postgres dedicado evo_campaign + roda migrations no start (sem db:create oficial). Documenta as vars no .env.example (EVO_FLOW_ENABLED, EVO_FLOW_API_URL, AUTH_APIKEY_INTEGRATION_LOCAL, EVO_FLOW_ALLOW_INSECURE). Subir: docker compose -f docker-compose.yml -f docker-compose.evoflow.yaml up -d.
Reviewer's GuideAdds a new docker-compose override to run evo-flow and ClickHouse in the main evo-crm docker stack, wiring env vars and bootstrap commands so the local event tracking flow works end-to-end. Sequence diagram for local contact event tracking with evo-flow and ClickHousesequenceDiagram
actor User
participant EvoCRM as Evo_CRM_FE+API
participant EvoFlow as Evo_Flow_API
participant ClickHouse
User->>EvoCRM: Update contact
EvoCRM->>EvoFlow: POST /events (AUTH_APIKEY_INTEGRATION_LOCAL)
EvoFlow->>ClickHouse: INSERT INTO contact_events (WRITE_MODE=ch-sync)
ClickHouse-->>EvoFlow: insert ok
EvoFlow-->>EvoCRM: 200 OK
User->>EvoCRM: Open contact Events tab
EvoCRM->>EvoFlow: GET /contacts/:id/events
EvoFlow->>ClickHouse: SELECT * FROM contact_events
ClickHouse-->>EvoFlow: rows
EvoFlow-->>EvoCRM: events
EvoCRM-->>User: Render contact.updated events
Flow diagram for evo-flow container bootstrap with dedicated Postgres DBflowchart TD
A[Evo-flow container start] --> B[Node pg Client connects
to Postgres database=postgres]
B --> C[CREATE DATABASE evo_campaign]
C --> D[If exists
log db already exists]
D --> E[Run typeorm migration:run
-d dist/database/ormconfig.js]
E --> F[exec node dist/main.js
RUN_MODE=api
QUEUE_MODE=direct
WRITE_MODE=ch-sync]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="docker-compose.evoflow.yaml" line_range="52-53" />
<code_context>
+ - sh
+ - -c
+ - |
+ node -e 'const{Client}=require("pg");const db=process.env.POSTGRES_DB_DATABASE;const c=new Client({host:process.env.POSTGRES_DB_HOST,port:+(process.env.POSTGRES_DB_PORT||5432),user:process.env.POSTGRES_DB_USERNAME,password:process.env.POSTGRES_DB_PASSWORD,database:"postgres"});c.connect().then(()=>c.query("CREATE DATABASE "+db)).then(()=>console.log("[init] created "+db)).catch(e=>console.log("[init] "+(e.code==="42P04"?"db already exists":e.message))).finally(()=>c.end());'
+ node_modules/.bin/typeorm migration:run -d dist/database/ormconfig.js || echo "[init] migration:run failed (continuing)"
+ exec node dist/main.js
+ environment:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Inline Node DB bootstrap script is hard to maintain and lacks failure signaling.
The `node -e` database creation command is difficult to read, test, and debug, and `migration:run` failures are deliberately ignored, so the container can appear healthy with an incomplete schema. Consider moving this logic into a dedicated JS/TS bootstrap file in the repo and calling it here, and decide whether migration failures should instead cause the container to exit so misconfigurations fail fast.
Suggested implementation:
```
# Recorrência: o banco Postgres do evo-flow é DEDICADO (evo_campaign) — NÃO
# compartilha o evo_community do CRM (as migrations do evo-flow criam tabelas
# contacts/labels/custom_attribute_definitions que colidiriam com o Chatwoot;
# o evo-flow lê o CRM via HTTP, não via Postgres). Como o evo-flow não tem
# passo oficial de db:create, o script bootstrap cuida de:
# - criar o DB se faltar
# - rodar as migrations (idempotente)
# - subir a app
# Se as migrations falharem, o container sai com erro para que a má
# configuração seja detectada rapidamente.
command:
- sh
- -c
- |
set -e
node dist/bootstrap.js
```
```
CLICKHOUSE_PASSWORD: password
CLICKHOUSE_DB: default
```
To fully implement your suggestion, the repository also needs a dedicated bootstrap script (e.g. `src/bootstrap.ts` compiled to `dist/bootstrap.js`) that encapsulates the previous inline behavior and exits non-zero on migration failure. For example, it should:
1. Use `pg` to connect to the Postgres server, create the target database if it does not exist, and log informative messages.
2. Run `typeorm migration:run -d dist/database/ormconfig.js` (via `child_process.spawn`/`exec`) and propagate any non-zero exit code.
3. Start the application (e.g. `require('./main');` or `import('./main');`) only if the above steps succeed.
Also ensure the Docker build produces `dist/bootstrap.js` and that the working directory in the container is where `dist/` is located so `node dist/bootstrap.js` resolves correctly.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| node -e 'const{Client}=require("pg");const db=process.env.POSTGRES_DB_DATABASE;const c=new Client({host:process.env.POSTGRES_DB_HOST,port:+(process.env.POSTGRES_DB_PORT||5432),user:process.env.POSTGRES_DB_USERNAME,password:process.env.POSTGRES_DB_PASSWORD,database:"postgres"});c.connect().then(()=>c.query("CREATE DATABASE "+db)).then(()=>console.log("[init] created "+db)).catch(e=>console.log("[init] "+(e.code==="42P04"?"db already exists":e.message))).finally(()=>c.end());' | ||
| node_modules/.bin/typeorm migration:run -d dist/database/ormconfig.js || echo "[init] migration:run failed (continuing)" |
There was a problem hiding this comment.
suggestion (bug_risk): Inline Node DB bootstrap script is hard to maintain and lacks failure signaling.
The node -e database creation command is difficult to read, test, and debug, and migration:run failures are deliberately ignored, so the container can appear healthy with an incomplete schema. Consider moving this logic into a dedicated JS/TS bootstrap file in the repo and calling it here, and decide whether migration failures should instead cause the container to exit so misconfigurations fail fast.
Suggested implementation:
# Recorrência: o banco Postgres do evo-flow é DEDICADO (evo_campaign) — NÃO
# compartilha o evo_community do CRM (as migrations do evo-flow criam tabelas
# contacts/labels/custom_attribute_definitions que colidiriam com o Chatwoot;
# o evo-flow lê o CRM via HTTP, não via Postgres). Como o evo-flow não tem
# passo oficial de db:create, o script bootstrap cuida de:
# - criar o DB se faltar
# - rodar as migrations (idempotente)
# - subir a app
# Se as migrations falharem, o container sai com erro para que a má
# configuração seja detectada rapidamente.
command:
- sh
- -c
- |
set -e
node dist/bootstrap.js
CLICKHOUSE_PASSWORD: password
CLICKHOUSE_DB: default
To fully implement your suggestion, the repository also needs a dedicated bootstrap script (e.g. src/bootstrap.ts compiled to dist/bootstrap.js) that encapsulates the previous inline behavior and exits non-zero on migration failure. For example, it should:
- Use
pgto connect to the Postgres server, create the target database if it does not exist, and log informative messages. - Run
typeorm migration:run -d dist/database/ormconfig.js(viachild_process.spawn/exec) and propagate any non-zero exit code. - Start the application (e.g.
require('./main');orimport('./main');) only if the above steps succeed.
Also ensure the Docker build producesdist/bootstrap.jsand that the working directory in the container is wheredist/is located sonode dist/bootstrap.jsresolves correctly.
dpaes
left a comment
dpaes
left a comment
There was a problem hiding this comment.
Review (EVO-1571 · local evo-flow stack) — change requested
The intent here is great and the wiring is mostly right, but the stack can't be brought up from a clean checkout as documented.
🔴 Blocker — build: context: ./evo-flow points at a path that isn't in the repo
docker-compose.evoflow.yaml builds the evo-flow service from ./evo-flow, but evo-flow is no longer part of this umbrella: it's commented out in .gitmodules and there is no evo-flow tree on this branch (nor on develop). It was removed as a submodule in the rc4/rc5 cleanup.
Result: a fresh clone running the documented command
docker compose -f docker-compose.yml -f docker-compose.evoflow.yaml up -d
fails with unable to prepare context: path "./evo-flow" not found. It works on your machine only because you still have a local ./evo-flow from when it was a submodule.
AC#6 ("documented how to bring it up") isn't satisfiable on a clean checkout yet. Options:
- Reference a published image (
image: ...) instead of a localbuild context; or - Document an explicit
git clone https://github.com/evolution-foundation/evo-flow ./evo-flowstep (and gitignore the dir); or - Re-add evo-flow as a submodule (note: this contradicts the rc4/rc5 removal decision).
🟡 Note — submodule pointers (merge process, not a defect)
This PR doesn't bump the evo-ai-crm-community / evo-ai-frontend-community pointers, so even once ./evo-flow is resolved, the umbrella builds CRM/FE from their old commits — without the proxy fix (#106) and traits/CSP changes (#116) — until the pointers are bumped after those merge.
🟢 Minor — dev-default credentials committed
EVOAI_CRM_API_TOKEN (a fixed UUID default), AUTH_APIKEY_INTEGRATION_LOCAL=evoflow-dev-key, and the ClickHouse password are committed in the compose. They're local-only and overridable, so acceptable — just flagging for awareness.
✅ Confirmed correct
- Service hostnames (
postgres,evo-auth:3001,evo-crm:3000) match the maindocker-compose.yml. - Dedicated
evo_campaignDB rationale (avoiding the Chatwootevo_communitytable collision) is sound. - The idempotent
CREATE DATABASE+migration:runone-liner is fine. .env.exampledocumentation is clear.
… pointers (EVO-1571) - docker-compose.evoflow.yaml: document the `git clone ... ./evo-flow` prerequisite. evo-flow is no longer a submodule (removed in rc4/rc5) and ./evo-flow is gitignored, so on a fresh clone the `build: context: ./evo-flow` path is missing and the documented "up" command failed. - bump evo-ai-frontend-community -> 58305bc (EVO-1571 #116 + review fixes: i18n traits/noData + aria-controls test). - bump evo-ai-crm-community -> 509ac56 (includes #106 "proxy le eventos do envelope data.events"), so the umbrella stops building CRM/FE from pre-#106/#116 commits. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… pointers (EVO-1571) - docker-compose.evoflow.yaml: document the `git clone ... ./evo-flow` prerequisite. evo-flow is no longer a submodule (removed in rc4/rc5) and ./evo-flow is gitignored, so on a fresh clone the `build: context: ./evo-flow` path is missing and the documented "up" command failed. - bump evo-ai-frontend-community -> 58305bc (EVO-1571 #116 + review fixes: i18n traits/noData + aria-controls test).
daniloleonecarneiro
force-pushed
the
danilocarneiro/evo-1571-historicotracking-fix-end-to-end-do-fluxo-de-eventos-do
branch
from
afa7813 to
e59af7d
Compare
dpaes
left a comment
dpaes
left a comment
There was a problem hiding this comment.
🔴 Re-review (round 2) — Request changes
Thanks for the rework. The fresh-clone build blocker is resolved ✅ — .gitignore now has evo-flow/, .gitmodules keeps evo-flow commented out, and docker-compose.evoflow.yaml documents the git clone … ./evo-flow prerequisite. That one is closed.
The round-2 changes introduced one new blocker: the two submodule pointer bumps are broken.
evo-ai-frontend-community→58305bc(hard break): this commit exists on no remote branch — it's an orphan from a rebase of #116 (whose real head isd0e6d52f…;58305bcisn't even an ancestor of it). After merging this PR,git submodule update --initon a fresh umbrella clone fails onevo-ai-frontend-communitybecause the recorded commit can't be fetched.evo-ai-crm-community→509ac56(wrong commit): not ondevelop— it lives on the EVO-1570 branch (refactor: unify reporting-event mapping), so it would pull another card's unmerged WIP into the umbrella. The #106 fix is already ondevelopas13348b9(currentdevelopHEAD isdc142a4).
Requested change — remove both gitlink bumps from this PR (don't re-point them). There is no correct FE SHA to target: #116 is still open and a squash-merge produces a new develop SHA, so any value committed now is wrong by construction. Umbrella pointers are bumped in a batched chore(submodules), not in a feature PR. Keep this PR scoped to docker-compose.evoflow.yaml + .env.example (+ the .gitignore/.gitmodules changes, already good) and drop the two submodule pointer changes → then it's good to merge.
Holding the PR open for the rework.
PR #114 should only ship the local evo-flow docker stack (docker-compose.evoflow.yaml + .env.example). The submodule pointer bumps did not belong here: - evo-ai-frontend-community 58305bc -> 306e974: 58305bc is an orphan from a rebase of #116 (real head d0e6d52), not in any remote branch. - evo-ai-crm-community 509ac56 -> 48c9e93: 509ac56 is EVO-1570 WIP (reporting-event mapping), not on develop. Umbrella submodule pointers belong in a batched chore(submodules), not in a feature PR. Restored both gitlinks to their develop values. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dpaes
left a comment
dpaes
left a comment
There was a problem hiding this comment.
Re-review round 3 (EVO-1571 · local evo-flow docker stack) — approved ✅
Re-reviewed against origin/develop (8d31ecb) and the current head (28a3248). Both prior blockers are resolved.
✅ Round-2 blocker (broken submodule gitlinks): resolved
Commit 28a3248 drops the two gitlink bumps (evo-ai-frontend 58305bc, evo-ai-crm 509ac56) and restores both pointers to their develop values. The PR now touches only 2 files — docker-compose.evoflow.yaml (new) and .env.example — with no gitlink diff. This is exactly the requested change: umbrella submodule pointers belong in a batched chore(submodules), not a feature PR.
✅ Round-1 blocker (fresh-clone build): holds on develop
Verified on origin/develop: .gitignore ignores evo-flow/, .gitmodules has the evo-flow entry commented out, and there is no evo-flow gitlink in the tree. The compose header documents the git clone … ./evo-flow prerequisite. A fresh clone + the documented up command works.
✅ Compose wiring
Checked against the main docker-compose.yml: postgres (with a pg_isready healthcheck), evo-crm, and evo-auth all exist, so depends_on: { postgres: service_healthy } and the hostnames evo-crm:3000 / evo-auth:3001 resolve. Postgres credential defaults match (POSTGRES_USERNAME:-postgres / POSTGRES_PASSWORD:-evoai_dev_password). AC#6 met.
🟡 Non-blocking (Sourcery also flagged this)
The inline command: uses a node -e one-liner for CREATE DATABASE and runs typeorm migration:run … || echo "failed (continuing)", which swallows a migration failure (the container can start with an incomplete schema). Acceptable for a dev-only override that is validated end-to-end per the card's validation steps. Optional hardening: move the bootstrap to a small script and let migration failures fail fast. Minor: clickhouse/clickhouse-server:latest is unpinned; dev-default secrets are committed (overridable). None blocking.
🧵 Note (out of this PR's scope)
End-to-end in the umbrella still requires a batched chore(submodules) bumping the CRM (#106, already on develop) and FE (#116 post-merge SHA) pointers. This PR correctly does not do that. Approving.
dpaes
deleted the
danilocarneiro/evo-1571-historicotracking-fix-end-to-end-do-fluxo-de-eventos-do
branch
2 participants
Summary
Adiciona o setup local que faz o fluxo de Histórico/Tracking rodar end-to-end (complementa os PRs de código: evo-flow
direct, proxy 7.7, UI traits 7.8).docker-compose.evoflow.yaml(novo) — override sobre odocker-compose.ymlprincipal que sobeclickhouse+ a appevo-flow(RUN_MODE=api+QUEUE_MODE=direct+WRITE_MODE=ch-sync) na rede default do projeto, então oevo-crmalcançahttp://evo-flow:3334. O evo-flow auto-cria o DB Postgres dedicadoevo_campaign+ roda migrations no start (não hádb:createoficial; oevo_communitydo CRM não serve — colide com tabelas do Chatwoot)..env.example— documentaEVO_FLOW_ENABLED,EVO_FLOW_API_URL,AUTH_APIKEY_INTEGRATION_LOCAL,EVO_FLOW_ALLOW_INSECURE.Por que
Sem isso, o
evo-flownão está em nenhum compose (odocker-compose.flow.yamlé só infra Kafka/Temporal, em rede isolada, e não sobe a app), e os listeners do CRM viram no-op sem as vars — a aba "Eventos" do contato não funciona.Como subir
(ou
COMPOSE_FILE=docker-compose.yml:docker-compose.evoflow.yamlno.envparadocker compose updireto)Test plan
contact.updated.docker exec ... clickhouse-client -q "SELECT count() FROM evo_campaign.contact_events"cresce a cada edição.Notas
QUEUE_MODE=directdepende do PR do evo-flow (DirectQueueProcessor) — buildar a imagem do evo-flow a partir dessa branch.Relacionado: EVO-1571
Summary by Sourcery
New Features:
Summary by Sourcery
New Features: