-
Notifications
You must be signed in to change notification settings - Fork 728
chore: optimize search sync worker (CM-750) #3697
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Uroš Marolt <[email protected]>
Signed-off-by: Uroš Marolt <[email protected]>
… aggregates Signed-off-by: Uroš Marolt <[email protected]>
Signed-off-by: Uroš Marolt <[email protected]>
Signed-off-by: Uroš Marolt <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR is being reviewed by Cursor Bugbot
Details
Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.
To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, lgtm. Left a few minor comments/nitpicks.
backend/src/database/migrations/V1765279142__optimize-segments-agg-tables.sql
Show resolved
Hide resolved
…g schedules Signed-off-by: Uroš Marolt <[email protected]>
services/apps/profiles_worker/src/workflows/member/refreshMemberDisplayAggregates.ts
Show resolved
Hide resolved
...ices/apps/profiles_worker/src/workflows/organization/refreshOrganizationDisplayAggregates.ts
Show resolved
Hide resolved
Signed-off-by: Uroš Marolt <[email protected]>
services/apps/profiles_worker/src/activities/calculateLeafSegmentAggregates.ts
Show resolved
Hide resolved
Signed-off-by: Uroš Marolt <[email protected]>
Signed-off-by: Uroš Marolt <[email protected]>
Note
Adds Temporal workflows for leaf→project/group aggregate rollups, optimizes segment aggregate tables and drops
indexed_entities, and simplifies search sync to direct OpenSearch updates.memberSegmentsAggandorganizationSegmentsAgg(segmentId,memberId|organizationId), add supporting indexes, and drop redundant ones.indexed_entitiestable.activities/workflows) and schedules (every 5 min for leaf; every 10 min rollups) guarded byIS_DEV_ENV.projectandproject-groupsegments using child workflows and segment hierarchy.insertMemberSegmentAggregatesand new batch rollup functions:calculateMemberAggregatesForSegment,calculateOrganizationAggregatesForSegment.activityRelationsandgetSegmentHierarchyutilities.indexed_entitiesusage and "withAggs" options; services now sync entities directly to OpenSearch.syncMemberactivity and update bot-affiliation fix workflow to use it.trigger-job:localscript tocron_service.Written by Cursor Bugbot for commit 26aa942. This will update automatically on new commits. Configure here.