Extract core, separate from UI, refactor data models and pipeline. Almost a full rewrite.#3
Open
Extract core, separate from UI, refactor data models and pipeline. Almost a full rewrite.#3
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…n files Move state management from App.tsx (~3500 lines) into two Zustand stores: - conversation-store.ts: conversations, selections, workflow actions - ui-store.ts: dialog state, presets, import state, dimensions Extract workflow pipeline from App.tsx into domain-organized files: - parse.ts, count-tokens.ts, segment.ts: pre-AI workflow steps - component-identification.ts: discover component list per dimension - component-classification.ts: classify each part into a component - color.ts: assign colors to components - summarize.ts, analyze.ts: AI summary and analysis generation - pipeline.ts: event dispatcher with per-event handlers - runner.ts, types.ts, dimensions.ts: shared infrastructure App.tsx shrinks to ~1150 lines of JSX, effects, and handler wiring. Each workflow domain file is a vertical slice (activity + step runner). The pipeline dispatcher replaces a 390-line monolithic function with a 20-line switch and 9 self-contained event handlers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the WorkflowRunner class and Activity<T> type. Replace with: - Notify: plain callback type (id, update) => void - startStep/endStep/updateState/markComplete: free functions - timed(): generic async timing wrapper Reorganize workflow files by domain instead of abstraction layer: - parse.ts, count-tokens.ts, segment.ts, color.ts, summarize.ts, analyze.ts each contain the activity logic + step runner as a vertical slice - component-identification.ts: discover component list (calls identifyComponents) - component-classification.ts: classify parts (calls mapComponentsToIds) - pipeline.ts: event dispatcher + composite sequences Each domain file calls the core lib directly — no intermediate Activity<T> wrapper or class instantiation needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Plan 1 - Componentisation Cleanup: Split componentisation.ts into component-types, component-identification, component-classification, component-coloring. Rename static-componentisation to static-components. Replace getComponentisationConfig with direct getAIConfig. Rename prompt key "component-mapping" to "component-classification". Plan 2 - Simplified Pipeline: Replace WorkflowEvent enum (9 values) with PipelineStep enum (6 ordered values). Replace 8 handler functions + dispatcher with runPipelineFrom(startStep). Summary/analysis become standalone on-demand functions. Plan 3 - First-Class Dimensions: Remove legacy top-level component fields (components, componentMapping, componentTimeline, componentColors, targetDimension) from WorkflowState. dimensions is now the single source of truth. Add accessor helpers (getDimension, getDefaultDimension, getAllComponents). Plan 4 - First-Class Groups: Groups become lightweight metadata (Group type with name + fileIds). Stored in separate groups store slice, not as concatenated WorkflowStates. Remove isGrouped, sourceConversations, messageSourceMap from WorkflowState. Plan 5 - Store Decomposition: Extract buildBaseContext to workflow/context.ts, file drop parsing to lib/file-import.ts, exportPromptsAsPreset to lib/export-builder.ts, orchestration functions to workflow/orchestrate.ts. Store shrinks from 788 to 332 lines. Additional: Rename SourceInfo→OriginInfo, messageSourceMap→messageOriginMap (provenance). Rename sourceConversations→memberFiles, sourceWorkflowStates→memberWorkflowStates. Define Source discriminated union type (FileSource | GroupSource) in source-types.ts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Allows turning off or adjusting reasoning/thinking for OpenAI reasoning models (gpt-5 series, o-series) via VITE_AI_THINKING=none|low|medium|high. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Store methods no longer accept setSelectedId or selectedId parameters. They just mutate data and return intent (e.g., groupConversations returns the group ID). App.tsx handles all navigation reactively: - useEffect auto-selects when selectedId is null or invalid - Wrapper functions in App.tsx call setSelectedId after store returns - No more stale closure captures from passing selectedId into async functions This fixes the bug where conversations would disappear after processing (caused by setSelectedId being called too late, after workflows completed, with a stale selectedId value). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Extract messageOriginMap into separate groupDisplayData memo (remove as-any cast) - Fix WorkflowDetailModal opening for group entries in sidebar - Strengthen color prompt to enforce named colors only (no hex codes) - Apply-prompts-to-all now also copies component list and colors as presets - Enable summary/analysis generation for groups (stored on Group object) - Virtual group state includes aiSummary/analysis from Group Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add VITE_AI_BASE_URL and VITE_AI_API_MODE env vars to allow switching between OpenAI and any OpenAI-compatible provider. Centralize model creation in ai-config.ts via createModel() helper, removing duplicate createOpenAI() calls across all pipeline files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Logs now show [identifying-components] and [classifying-components] as distinct phases instead of a single [finding-components]. Step timings record each separately. The UI progress indicator still shows one combined "Find components" step. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Small cycle button above waffle chart in Components tab. Setting is shared across WaffleChart, StackedBarChartView, and ComponentComparisonView via UI store. Default is 0 (unchanged behavior). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ComparisonLegend is a separate function component that didn't have access to the percentPrecision prop. Now receives it explicitly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Step runners (identify, classify, color) accept optional dimNames[] to process only specific dimensions instead of all - runPipelineFrom passes dimNames through to step runners - Extract runDimensionSteps to eliminate duplication across processNewFile, resumeFromPause, and runPipelineFrom - Move customPrompt, customColoringPrompt, customComponents from WorkflowState top-level into DimensionData (dimensions are now self-contained) - Rewrite applyPromptsToAll to diff per-dimension prompts and only reprocess dimensions that actually changed - App.tsx passes [dimName] when editing prompts/components/colors for a single dimension Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add reprocessTarget() to orchestrate.ts: resolves group-or-file,
fans out to member files for groups, processes single file otherwise
- Replace all if (selectedGroup) { loop } blocks in App.tsx handlers
with calls to handleReprocessTarget
- Coloring prompt reprocessing now works for groups (was missing)
- Remove stale fields from Group type: customPrompt,
customColoringPrompt, customSegmentationPrompt, segmentationThreshold
(component prompts live on member files' dimensions, not the group)
- Remove unused runPipelineFrom import from App.tsx
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Components now read from Zustand stores directly instead of receiving 30+ callback props threaded through App.tsx. This eliminates the controlled+fallback dual-state pattern and makes each component self-contained. New files: - stores/url-store.ts: URL state as a Zustand store (replaces useUrlState hook) - hooks/useWorkflowActions.ts: extracted handler functions from App.tsx - lib/message-filters.ts: shared filter logic (was duplicated 3x) Key changes: - ConversationList: 36 props → 0 (reads stores directly) - ConversationView: removed 24 URL-state props and dual-state pattern - ComponentComparisonView: removed 14 props and 6 local state shadows - App.tsx: 1278 → 632 lines (layout shell + effects + dialogs only) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pipeline (pipeline.ts) — just the step chain: - runDimensionSteps: Segment → Identify → (Classify + Color in parallel) - processNewFile: Parse → CountTokens → Static → runDimensionSteps - resumeFromPause: runDimensionSteps from Segment Orchestration (orchestrate.ts) — who runs what, where results go: - runPipelineFrom, reprocessWithRunner, reprocessTarget (moved from pipeline) - runWorkflows batch processing (moved from pipeline) - generateSummaryOnDemand/generateAnalysisOnDemand/rerunSummary (moved) - New store-level functions: generateAnalysisForTarget, generateSummaryForTarget, rerunSummaryForTarget — handle group-vs-file dispatch with group-aware notify routing UI layer (useWorkflowActions.ts) — thin wrappers only: - All handlers now call store actions, never pipeline functions directly - Removed buildBaseContext, Notify, runPipelineFrom imports - Group fan-out handled by orchestrate, not UI code Also: - Rename runner.ts → notify.ts (it's progress/state helpers, not a runner) - Classify + Color run in parallel after Identify (both only need component list, not each other) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WorkflowDetailModal now reads conversation data from the store and calls action functions directly instead of receiving ~20 callback props. - Props reduced from 28 to 3 (isOpen, onClose, conversationId) - Reads conv data, group info, member files from useConversationStore - Calls openPromptEditor, generateAnalysis, etc. from useWorkflowActions - ConversationList call site simplified from 28 props to 3 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ai-config.ts: LanguageModelV2 → LanguageModel (renamed in ai SDK) - GroupFileOrderEditor: optional chaining on possibly-undefined array access - url-fetch.ts: non-null assertion on array element - workflow/types.ts: widen stepTimings to accept sub-step keys - claude-transcripts-parser: type assertions for Zod union narrowing (ClaudeContent catch-all schema prevents TS discriminated union narrowing) - codex-transcripts-parser: same Zod union narrowing fix - export-import tests: remove null analysis values (should be undefined) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- WorkflowDetailModal: derive dimensions from first member file for groups (was undefined because groups aren't in conversations array) - openPromptEditor/openComponentsEditor/openColoringPromptEditor: for groups, read current prompt from first member file - applyPrompt/applyComponents/applyColoringPrompt: for groups, update all member files' dimensions in the store (was only updating by group ID which matched nothing) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The diff was only comparing prompts and customComponents between source and target dimensions. When both had undefined prompts (default), nothing would run even if the actual component lists were different. Now compares the actual components array too, so "apply to all" correctly reprocesses targets whose components differ from the source. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add "segments-asc" and "segments-desc" sort options that flatten messages into individual parts sorted by token count - Change default coloring prompt to request hex codes with a reference palette, instead of restricting to named color strings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sion data Both runClassifyComponents and runAssignColors run in parallel and were replacing dims[dimName] with a spread of the original object. Whichever finished last would wipe the other's work, causing either missing componentMapping (no waffles) or missing componentColors. Fix: mutate the existing dimData object in place instead of replacing it. Also add store/dimensions to __debug window object for easier debugging. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each dimension-level step (Identify, Classify, Color) now checks its inputs against its existing outputs and skips if they match, rather than relying on the orchestrator to decide what to run. This removes the branching logic from applyPromptsToAll (segChanged/identifyDims/ colorDims) — it now copies all state from source and runs the full pipeline, letting each step skip itself. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The expandedDimension state was keyed by dimension name alone, so expanding "default" on one conversation expanded it everywhere. Now keyed by "conversationId:dimName" so each is independent. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move selection state and API key state from conversation-store to ui-store. Extract export actions as standalone functions. Remove 7 orchestration passthrough methods from conversation-store — hooks now call orchestrate functions directly via an external accessor, making conversation-store a pure data/CRUD layer. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reorganize the codebase into a clear layered directory structure: - model/ — domain types and schemas (schema, types, dimensions, export-schema, presets) - operations/ — pure transforms (aggregation, token-counting, static-components, etc.) - parsers/ — pluggable input format adapters - stages/ — processing pipeline stages with ai/ infrastructure nested inside - pipeline/ — orchestration (step sequencing, notify, orchestrate) - stores/ — state management + actions (moved useWorkflowActions here) - ui/ — React components, hooks, and UI-specific lib files - lib/ — generic utilities (id-generator) Each algorithm file and its workflow wrapper are merged into one file per stage. The dependency rule is enforced: model → nothing, operations → model, parsers → model, stages → model + operations, pipeline → stages, stores → pipeline, ui → stores. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move browser I/O (downloadExport, exportPromptsAsPreset) from operations/ to ui/lib/export-download.ts - Move workflow-logger and stage-logger from stages/ai/ to pipeline/ (they're cross-cutting infrastructure, not AI-specific) - Replace ProcessingPhase/ProcessingStep with unified Stage and StageGroup types; consistent -ing verb forms throughout - Rename Workflow* types to Pipeline* (PipelineState, PipelineCallbacks, PipelineOptions, etc.) to match directory naming - Rename DimensionData.components to discoveredComponents; add getEffectiveComponents() helper to consolidate override logic - Add I/O concern note to stages/ai/preset-loader.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Write docs/architecture.md: directory structure, layers, dependency rules, stage descriptions, pipeline flow, and key concepts - Archive 12 completed/historical plan docs to docs/archive/ - Update CAPABILITIES.md file locations to new structure - Update tech-stack.md: useState → Zustand - Update system-overview.md and dimension-scoped-pipeline-design.md with cross-references to architecture.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each dimension-level stage (identify, classify, color) now operates on a single DimensionData instead of internally looping over all dimensions. The pipeline owns the iteration: - identifyForDimension(conversation, dimData, config, id) - classifyForDimension(conversation, dimData, config, id) - colorForDimension(dimData, config, id, presetColors?) pipeline/pipeline.ts handles: which dimensions to process, parallelism (classify + color run in parallel per dimension), error collection, and timing. Stages are now simpler — pure single-dimension logic with no orchestration concerns. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The dimension loop in pipeline/pipeline.ts silently skipped all work when dims["default"] didn't exist (new files start with empty dimensions). Added ensureDimension() and createEmptyDimension() to model/dimensions.ts as the canonical way to initialize dimensions. Replaced 7 inline empty DimensionData literals across stores/actions, pipeline/orchestrate, and ui/App with the factory function. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Stages now return results instead of mutating ctx/dimData. The pipeline is defined as a PIPELINE array of StageDescriptors, and a runner handles sequencing, dimension loops, parallelism, logging, timing, and store updates. Classify+color parallel execution is race-free by construction — results are merged after both complete. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the declarative PIPELINE array + runner with an imperative runPipeline function that reads top-to-bottom like pseudocode. Absorb all orchestration (reprocessTarget, applyPromptsToAll, batch processing, on-demand summary/analysis) into pipeline.ts and delete orchestrate.ts entirely. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each stage checks if its work is already done and skips if so. Callers signal "re-run this stage" by clearing its outputs (e.g., dim.discoveredComponents = [] to force re-identify). Delete PipelineStep enum — no longer needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Delete processNewFile and resumeFromPause — they were identical try/catch wrappers around runPipeline. Error handling is now inside runPipeline itself. - Remove redundant markComplete calls from reprocessTarget and applyPromptsToAll — the pipeline's final updateState already pushes the complete state. - Extract makeGroupAwareCallbacks to deduplicate streaming callback setup for summary/analysis. - Fix duplicate ctx.aiSummary = "" in rerunSummaryForTarget. - Simplify resumePipelinesWithApiKey to call runPipeline directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.