From 9a2024b300e5bcd686efb59be8501b9b825f5a43 Mon Sep 17 00:00:00 2001 From: Marius van Niekerk Date: Wed, 24 Jun 2026 21:07:30 -0400 Subject: [PATCH] feat(parser): migrate opencode-family providers OpenCode, Kilo, and MiMoCode share the same storage/session, message, part, and legacy SQLite source model. Moving them behind one concrete provider keeps that shared source contract explicit instead of spreading it across sync-only classifier paths. The provider preserves storage-first discovery, hybrid SQLite fallback, duplicate filtering, child-file changed-path classification, SQLite virtual paths, composite source mtimes, storage fingerprints, and fork-specific ID relabeling. fix(parser): classify removed opencode storage sessions OpenCode-family storage sessions are watched recursively, so delete and rename-style events for the primary session JSON need to map back to the same provider source even after the file no longer exists. Without that syntactic fallback, provider-path sync can miss stale storage sources until a broader resync. Move OpenCode, Kilo, and MiMoCode into shadow comparison on this branch so the stack continues as a real migration rather than an additive provider implementation. Validation: go test -tags "fts5" ./internal/parser -run 'Test(OpenCodeProvider|OpenCodeFamilyProvider|ProviderMigration)' -count=1; go test -tags "fts5" ./internal/parser -count=1; go vet ./...; git diff --check fix(sync): classify removed opencode session files The provider path now handles deleted OpenCode-family storage session JSONs, but the legacy SyncPaths classifier is still active during the migration. It needs the same syntactic fallback so watcher-driven sync remains behaviorally equivalent while both forms run. Validation: go test -tags "fts5" ./internal/sync -run 'TestEngine_ClassifyPathsOpenCodeFamilyRemovedSessionFile' -count=1; go test -tags "fts5" ./internal/parser -run 'Test(OpenCodeProvider|OpenCodeFamilyProvider|ProviderMigration)' -count=1; go test -tags "fts5" ./internal/sync -count=1; go vet ./...; git diff --check test(sync): compare opencode family shadow parity OpenCode, Kilo, and MiMoCode share the OpenCode-format provider implementation on this branch, so add source-level migration coverage for all three storage-mode source shapes. The table test compares provider observation with each legacy parser and verifies session/message/data-version parity while preserving provider-computed storage fingerprints. Validation: go test -tags "fts5" ./internal/parser ./internal/sync -run 'TestObserveProviderSourceMatchesOpenCodeFamilyLegacyParsers|TestOpenCode|TestParseOpenCode|TestParseKilo|TestParseMiMoCode|TestDiscoverKilo|TestDiscoverMiMoCode|TestProviderMigrationModes' -count=1; go test -tags "fts5" ./internal/parser ./internal/sync -count=1; go fmt ./...; go vet ./...; ./custom-gcl run --config .golangci.nilaway.yml ./internal/parser/... ./internal/sync/...; git diff --check refactor(parser): fold opencode family into providers OpenCode, Kilo, and MiMoCode share one on-disk format (storage/session JSON plus message/part files, with a legacy SQLite fallback exposed as # virtual paths). They were still shims: the concrete provider delegated to package-level free functions and the agents stayed LegacyOnly, which violated the migration manifest invariant (a concrete provider must not remain legacy-only) and left the runtime on the legacy sync dispatch. Make the three providers authoritative and own their behavior. One shared openCodeFormatProvider implementation is parameterized per agent by a format struct (SQLite filename, storage session subdir, ID prefix); Kilo and MiMoCode reuse the OpenCode storage and SQLite readers and only relabel the parsed session onto their own agent and ID prefix, so the parse/discover/find logic is not duplicated three times. The 15 legacy free functions (Discover/Find/ParseFile/ParseSession/ ParseSQLiteVirtualPath for each of opencode/kilo/mimocode) are deleted. ParseOpenCodeFile/ParseOpenCodeSession move to unexported helpers (parseOpenCodeStorageFile/parseOpenCodeDBSession) that the provider spec drives. SQLite virtual-path resolution now goes through the provider-neutral ParseVirtualSourcePathForBase, so engine, parsediff, and resume callers no longer reference deleted parsers. Remove the opencode-family legacy engine dispatch: the classify blocks and classifyOpenCodeFormatPath, the processFile arm and processOpenCodeFormat, the DB-backed sync pass, single-session resync, and orphaned helpers. Runtime now routes through provider changed-path classification and processProviderFile. Because these agents now flow through file discovery, the resync empty-discovery guard tracks a non-container discovered count so a self-preserving storage store cannot mask plain file-backed sessions whose directories went empty, and parse-diff discovers them through the provider facade. fix(sync): skip provider-authoritative agents in parse-diff db synthesis parseDiffDatabaseSources synthesized a raw opencode.db/kilo.db source so the legacy processOpenCode fan-out re-parsed every DB session. Once those agents became provider-authoritative, parseDiffProviderSources already enumerates their DB sessions through the provider, which applies the storage-ID filter that drops a file-backed storage session's stale db row. Re-adding the raw db then double-counted those sessions and parsed the filtered storage row, surfacing a spurious ParseError and an extra examined file. Skip agents that have dropped their DiscoverFunc; the Kiro data.sqlite3 synthesis still runs because Kiro keeps its legacy DiscoverFunc until its own fold. refactor(parser): delete opencode legacy whole-database parser ParseOpenCodeDB parsed every session in an OpenCode SQLite database, but the provider (and the Kilo/MiMoCode reuse) routes per-session through parseOpenCodeDBSession, so the free function survived only as test-exercised dead production code. Delete ParseOpenCodeDB along with the orphaned loadOpenCodeSessions and the OpenCodeSession bundle type; loadOpenCodeProjects stays since the per-session path also resolves worktrees through it. The retained parse tests reproduce the whole-database walk with the provider's own primitives (ListOpenCodeSessionMeta + parseOpenCodeDBSession). fix(sync): preserve parse-diff virtual sqlite identity OpenCode-family providers expose SQLite sessions as per-session virtual sources. Parse-diff was still collapsing those db#session paths to the shared database path before error attribution, presence sweep, and limit accounting, which could apply one session's parse failure or omitted sample to every sibling in the DB. Keep exact source keys for OpenCode, Kilo, and MiMoCode provider virtual SQLite paths while retaining shared-base grouping for true physical multi-session jobs. Source existence checks still stat the physical DB path so virtual identities do not look missing. Validation: go test -tags "fts5" ./internal/sync -run 'TestParseDiffProviderVirtualSQLite(ErrorUsesExactSource|PresenceUsesExactSource|LimitUsesExactSource)|TestStripVirtualSourceSuffixVisualStudioCopilot' -count=1; go test -tags "fts5" ./internal/sync -run 'TestParseDiff(CoversMixedOpenCodeRoot|CoversMixedKiloRoot|ProviderVirtualSQLite|PresenceSweep|LimitNewestFirst|ReportHasFailures)' -count=1; go test -tags "fts5" ./internal/parser -run 'Test(OpenCodeProvider|OpenCodeFamilyProvider|Kilo|MiMoCode)' -count=1; go vet ./...; git diff --check --- internal/parser/discovery.go | 50 -- internal/parser/kilo.go | 28 +- internal/parser/kilo_test.go | 49 +- internal/parser/mimocode.go | 28 +- internal/parser/mimocode_test.go | 49 +- internal/parser/opencode.go | 113 +--- internal/parser/opencode_provider.go | 744 +++++++++++++++++++++ internal/parser/opencode_provider_test.go | 347 ++++++++++ internal/parser/opencode_test.go | 87 ++- internal/parser/provider.go | 6 + internal/parser/provider_migration.go | 6 +- internal/parser/provider_shim_scan_test.go | 1 - internal/parser/types.go | 6 - internal/parser/types_test.go | 39 +- internal/server/resume.go | 4 +- internal/sync/engine.go | 722 +------------------- internal/sync/engine_integration_test.go | 38 +- internal/sync/engine_test.go | 108 ++- internal/sync/parsediff.go | 45 +- internal/sync/parsediff_compare_test.go | 144 ++++ internal/sync/progress.go | 16 +- 21 files changed, 1633 insertions(+), 997 deletions(-) create mode 100644 internal/parser/opencode_provider.go create mode 100644 internal/parser/opencode_provider_test.go diff --git a/internal/parser/discovery.go b/internal/parser/discovery.go index 848e825d5..42a329c97 100644 --- a/internal/parser/discovery.go +++ b/internal/parser/discovery.go @@ -300,22 +300,6 @@ func ResolveOpenCodeSource(root string) OpenCodeSource { return resolveOpenCodeFormatSource(openCodeFmt, root) } -// DiscoverOpenCodeSessions finds all file-backed OpenCode session -// JSON files under storage/session. -func DiscoverOpenCodeSessions(root string) []DiscoveredFile { - return discoverOpenCodeFormatSessions(openCodeFmt, root) -} - -// FindOpenCodeSourceFile locates a single OpenCode session source -// path or SQLite backing file by raw session ID. Returns "" when -// the session is not present under this root so the caller -// (Engine.FindSourceFile) can continue searching later configured -// roots — important when an early hybrid root with an unrelated -// opencode.db could otherwise shadow a session in a later root. -func FindOpenCodeSourceFile(root, sessionID string) string { - return findOpenCodeFormatSourceFile(openCodeFmt, root, sessionID) -} - // OpenCodeStorageSessionIDs returns the set of session IDs that // have a JSON file under storage/session/*/ in the given root. // Returns nil for non-storage roots. In hybrid roots (storage and @@ -344,12 +328,6 @@ func OpenCodeSQLiteVirtualPath( return dbPath + "#" + sessionID } -func ParseOpenCodeSQLiteVirtualPath( - sourcePath string, -) (dbPath, sessionID string, ok bool) { - return parseOpenCodeFormatVirtualPath(openCodeFmt.dbName, sourcePath) -} - func openCodeSessionProject(path string) string { data, err := os.ReadFile(path) if err == nil { @@ -372,14 +350,6 @@ func ResolveKiloSource(root string) OpenCodeSource { return resolveOpenCodeFormatSource(kiloFmt, root) } -func DiscoverKiloSessions(root string) []DiscoveredFile { - return discoverOpenCodeFormatSessions(kiloFmt, root) -} - -func FindKiloSourceFile(root, sessionID string) string { - return findOpenCodeFormatSourceFile(kiloFmt, root, sessionID) -} - func KiloStorageSessionIDs(root string) map[string]struct{} { return openCodeFormatStorageSessionIDs(kiloFmt, root) } @@ -392,26 +362,12 @@ func KiloSQLiteVirtualPath(dbPath, sessionID string) string { return OpenCodeSQLiteVirtualPath(dbPath, sessionID) } -func ParseKiloSQLiteVirtualPath( - sourcePath string, -) (dbPath, sessionID string, ok bool) { - return parseOpenCodeFormatVirtualPath(kiloFmt.dbName, sourcePath) -} - // ResolveMiMoCodeSource detects whether a MiMoCode root is using // file-backed storage (storage/session_diff) or SQLite storage. func ResolveMiMoCodeSource(root string) OpenCodeSource { return resolveOpenCodeFormatSource(mimoFmt, root) } -func DiscoverMiMoCodeSessions(root string) []DiscoveredFile { - return discoverOpenCodeFormatSessions(mimoFmt, root) -} - -func FindMiMoCodeSourceFile(root, sessionID string) string { - return findOpenCodeFormatSourceFile(mimoFmt, root, sessionID) -} - func MiMoCodeStorageSessionIDs(root string) map[string]struct{} { return openCodeFormatStorageSessionIDs(mimoFmt, root) } @@ -424,12 +380,6 @@ func MiMoCodeSQLiteVirtualPath(dbPath, sessionID string) string { return OpenCodeSQLiteVirtualPath(dbPath, sessionID) } -func ParseMiMoCodeSQLiteVirtualPath( - sourcePath string, -) (dbPath, sessionID string, ok bool) { - return parseOpenCodeFormatVirtualPath(mimoFmt.dbName, sourcePath) -} - // ResolveCodexShallowWatchRoots returns directories that should be watched // shallowly (root only) for live Codex updates, in addition to the recursive // watch on the configured sessions root. Codex writes title renames to diff --git a/internal/parser/kilo.go b/internal/parser/kilo.go index 364eff937..b7b31607e 100644 --- a/internal/parser/kilo.go +++ b/internal/parser/kilo.go @@ -3,28 +3,8 @@ package parser import "strings" // Kilo uses OpenCode's storage format, but sessions are exposed as a -// distinct agent with the kilo: ID prefix. -func ParseKiloFile( - sessionPath, machine string, -) (*ParsedSession, []ParsedMessage, error) { - sess, msgs, err := ParseOpenCodeFile(sessionPath, machine) - if err != nil || sess == nil { - return sess, msgs, err - } - relabelOpenCodeSessionAsKilo(sess) - return sess, msgs, nil -} - -func ParseKiloSession( - dbPath, sessionID, machine string, -) (*ParsedSession, []ParsedMessage, error) { - sess, msgs, err := ParseOpenCodeSession(dbPath, sessionID, machine) - if err != nil || sess == nil { - return sess, msgs, err - } - relabelOpenCodeSessionAsKilo(sess) - return sess, msgs, nil -} +// distinct agent with the kilo: ID prefix. The OpenCode-format provider +// owns parsing and relabels results through relabelOpenCodeSessionAsKilo. func ListKiloSessionMeta(dbPath string) ([]OpenCodeSessionMeta, error) { metas, err := ListOpenCodeSessionMeta(dbPath) @@ -43,7 +23,9 @@ func KiloSourceMtime(sourcePath string) (int64, error) { if sourcePath == "" { return 0, nil } - if dbPath, sessionID, ok := ParseKiloSQLiteVirtualPath(sourcePath); ok { + if dbPath, sessionID, ok := parseOpenCodeFormatVirtualPath( + kiloFmt.dbName, sourcePath, + ); ok { return openCodeSQLiteSessionMtime(dbPath, sessionID) } return openCodeStorageSessionMtime(sourcePath) diff --git a/internal/parser/kilo_test.go b/internal/parser/kilo_test.go index ed382ddaf..d67deb7a4 100644 --- a/internal/parser/kilo_test.go +++ b/internal/parser/kilo_test.go @@ -1,6 +1,7 @@ package parser import ( + "context" "path/filepath" "testing" @@ -8,7 +9,7 @@ import ( "github.com/stretchr/testify/require" ) -func TestParseKiloFileRelabelsOpenCodeSession(t *testing.T) { +func TestKiloProviderParseRelabelsOpenCodeSession(t *testing.T) { root := t.TempDir() sessionPath := filepath.Join( root, "storage", "session", "global", "ses_kilo.json", @@ -46,9 +47,25 @@ func TestParseKiloFileRelabelsOpenCodeSession(t *testing.T) { }, }) - sess, msgs, err := ParseKiloFile(sessionPath, "testmachine") + provider, ok := NewProvider(AgentKilo, ProviderConfig{ + Roots: []string{root}, + Machine: "testmachine", + }) + require.True(t, ok) + source, found, err := provider.FindSource(context.Background(), FindSourceRequest{ + FullSessionID: "kilo:ses_kilo", + }) + require.NoError(t, err) + require.True(t, found) + + outcome, err := provider.Parse(context.Background(), ParseRequest{ + Source: source, + Machine: "testmachine", + }) require.NoError(t, err) - require.NotNil(t, sess) + require.Len(t, outcome.Results, 1) + sess := outcome.Results[0].Result.Session + msgs := outcome.Results[0].Result.Messages require.Len(t, msgs, 1) assert.Equal(t, "kilo:ses_kilo", sess.ID) @@ -58,7 +75,7 @@ func TestParseKiloFileRelabelsOpenCodeSession(t *testing.T) { assert.Equal(t, "Hello from Kilo", msgs[0].Content) } -func TestDiscoverKiloSessions(t *testing.T) { +func TestKiloProviderDiscoversSessions(t *testing.T) { root := t.TempDir() sessionPath := filepath.Join( root, "storage", "session", "global", "ses_kilo.json", @@ -72,24 +89,28 @@ func TestDiscoverKiloSessions(t *testing.T) { }, }) - files := DiscoverKiloSessions(root) - require.Len(t, files, 1) + provider, ok := NewProvider(AgentKilo, ProviderConfig{Roots: []string{root}}) + require.True(t, ok) + sources, err := provider.Discover(context.Background()) + require.NoError(t, err) + require.Len(t, sources, 1) - assert.Equal(t, sessionPath, files[0].Path) - assert.Equal(t, "kiloapp", files[0].Project) - assert.Equal(t, AgentKilo, files[0].Agent) + assert.Equal(t, sessionPath, sources[0].DisplayPath) + assert.Equal(t, "kiloapp", sources[0].ProjectHint) + assert.Equal(t, AgentKilo, sources[0].Provider) } -func TestParseKiloSQLiteVirtualPath(t *testing.T) { +func TestKiloSQLiteVirtualPathRoundTrips(t *testing.T) { wantDBPath := filepath.Join(t.TempDir(), "kilo.db") - virtual := wantDBPath + "#ses_kilo" - dbPath, sessionID, ok := ParseKiloSQLiteVirtualPath(virtual) + virtual := KiloSQLiteVirtualPath(wantDBPath, "ses_kilo") + dbPath, sessionID, ok := parseOpenCodeFormatVirtualPath(kiloFmt.dbName, virtual) require.True(t, ok) assert.Equal(t, wantDBPath, dbPath) assert.Equal(t, "ses_kilo", sessionID) - _, _, ok = ParseKiloSQLiteVirtualPath( - filepath.Join(t.TempDir(), "opencode.db") + "#ses_kilo", + _, _, ok = parseOpenCodeFormatVirtualPath( + kiloFmt.dbName, + filepath.Join(t.TempDir(), "opencode.db")+"#ses_kilo", ) assert.False(t, ok) } diff --git a/internal/parser/mimocode.go b/internal/parser/mimocode.go index 115a9177e..7df6ddd6b 100644 --- a/internal/parser/mimocode.go +++ b/internal/parser/mimocode.go @@ -4,28 +4,8 @@ import "strings" // MiMoCode uses OpenCode's storage format, but stores sessions under // storage/session_diff and is exposed as a distinct agent with the -// mimocode: ID prefix. -func ParseMiMoCodeFile( - sessionPath, machine string, -) (*ParsedSession, []ParsedMessage, error) { - sess, msgs, err := ParseOpenCodeFile(sessionPath, machine) - if err != nil || sess == nil { - return sess, msgs, err - } - relabelOpenCodeSessionAsMiMoCode(sess) - return sess, msgs, nil -} - -func ParseMiMoCodeSession( - dbPath, sessionID, machine string, -) (*ParsedSession, []ParsedMessage, error) { - sess, msgs, err := ParseOpenCodeSession(dbPath, sessionID, machine) - if err != nil || sess == nil { - return sess, msgs, err - } - relabelOpenCodeSessionAsMiMoCode(sess) - return sess, msgs, nil -} +// mimocode: ID prefix. The OpenCode-format provider owns parsing and +// relabels results through relabelOpenCodeSessionAsMiMoCode. func ListMiMoCodeSessionMeta(dbPath string) ([]OpenCodeSessionMeta, error) { metas, err := ListOpenCodeSessionMeta(dbPath) @@ -44,7 +24,9 @@ func MiMoCodeSourceMtime(sourcePath string) (int64, error) { if sourcePath == "" { return 0, nil } - if dbPath, sessionID, ok := ParseMiMoCodeSQLiteVirtualPath(sourcePath); ok { + if dbPath, sessionID, ok := parseOpenCodeFormatVirtualPath( + mimoFmt.dbName, sourcePath, + ); ok { return openCodeSQLiteSessionMtime(dbPath, sessionID) } return openCodeStorageSessionMtime(sourcePath) diff --git a/internal/parser/mimocode_test.go b/internal/parser/mimocode_test.go index b0b9c6ce1..7217186db 100644 --- a/internal/parser/mimocode_test.go +++ b/internal/parser/mimocode_test.go @@ -1,6 +1,7 @@ package parser import ( + "context" "path/filepath" "testing" @@ -8,7 +9,7 @@ import ( "github.com/stretchr/testify/require" ) -func TestParseMiMoCodeFileRelabelsOpenCodeSession(t *testing.T) { +func TestMiMoCodeProviderParseRelabelsOpenCodeSession(t *testing.T) { root := t.TempDir() sessionPath := filepath.Join( root, "storage", "session_diff", "global", "ses_mimo.json", @@ -46,9 +47,25 @@ func TestParseMiMoCodeFileRelabelsOpenCodeSession(t *testing.T) { }, }) - sess, msgs, err := ParseMiMoCodeFile(sessionPath, "testmachine") + provider, ok := NewProvider(AgentMiMoCode, ProviderConfig{ + Roots: []string{root}, + Machine: "testmachine", + }) + require.True(t, ok) + source, found, err := provider.FindSource(context.Background(), FindSourceRequest{ + FullSessionID: "mimocode:ses_mimo", + }) + require.NoError(t, err) + require.True(t, found) + + outcome, err := provider.Parse(context.Background(), ParseRequest{ + Source: source, + Machine: "testmachine", + }) require.NoError(t, err) - require.NotNil(t, sess) + require.Len(t, outcome.Results, 1) + sess := outcome.Results[0].Result.Session + msgs := outcome.Results[0].Result.Messages require.Len(t, msgs, 1) assert.Equal(t, "mimocode:ses_mimo", sess.ID) @@ -58,7 +75,7 @@ func TestParseMiMoCodeFileRelabelsOpenCodeSession(t *testing.T) { assert.Equal(t, "Hello from MiMoCode", msgs[0].Content) } -func TestDiscoverMiMoCodeSessions(t *testing.T) { +func TestMiMoCodeProviderDiscoversSessions(t *testing.T) { root := t.TempDir() sessionPath := filepath.Join( root, "storage", "session_diff", "global", "ses_mimo.json", @@ -72,24 +89,28 @@ func TestDiscoverMiMoCodeSessions(t *testing.T) { }, }) - files := DiscoverMiMoCodeSessions(root) - require.Len(t, files, 1) + provider, ok := NewProvider(AgentMiMoCode, ProviderConfig{Roots: []string{root}}) + require.True(t, ok) + sources, err := provider.Discover(context.Background()) + require.NoError(t, err) + require.Len(t, sources, 1) - assert.Equal(t, sessionPath, files[0].Path) - assert.Equal(t, "mimoapp", files[0].Project) - assert.Equal(t, AgentMiMoCode, files[0].Agent) + assert.Equal(t, sessionPath, sources[0].DisplayPath) + assert.Equal(t, "mimoapp", sources[0].ProjectHint) + assert.Equal(t, AgentMiMoCode, sources[0].Provider) } -func TestParseMiMoCodeSQLiteVirtualPath(t *testing.T) { +func TestMiMoCodeSQLiteVirtualPathRoundTrips(t *testing.T) { wantDBPath := filepath.Join(t.TempDir(), "mimocode.db") - virtual := wantDBPath + "#ses_mimo" - dbPath, sessionID, ok := ParseMiMoCodeSQLiteVirtualPath(virtual) + virtual := MiMoCodeSQLiteVirtualPath(wantDBPath, "ses_mimo") + dbPath, sessionID, ok := parseOpenCodeFormatVirtualPath(mimoFmt.dbName, virtual) require.True(t, ok) assert.Equal(t, wantDBPath, dbPath) assert.Equal(t, "ses_mimo", sessionID) - _, _, ok = ParseMiMoCodeSQLiteVirtualPath( - filepath.Join(t.TempDir(), "opencode.db") + "#ses_mimo", + _, _, ok = parseOpenCodeFormatVirtualPath( + mimoFmt.dbName, + filepath.Join(t.TempDir(), "opencode.db")+"#ses_mimo", ) assert.False(t, ok) } diff --git a/internal/parser/opencode.go b/internal/parser/opencode.go index dfd7297ec..b7d00e3c9 100644 --- a/internal/parser/opencode.go +++ b/internal/parser/opencode.go @@ -5,7 +5,6 @@ import ( "database/sql" "encoding/json" "fmt" - "log" "os" "path/filepath" "regexp" @@ -18,12 +17,6 @@ import ( const openCodeStorageFingerprintPrefix = "opencode-storage:v1:" -// OpenCodeSession bundles a parsed session with its messages. -type OpenCodeSession struct { - Session ParsedSession - Messages []ParsedMessage -} - // OpenCodeSessionMeta is lightweight metadata for a session, // used to detect changes without parsing messages or parts. type OpenCodeSessionMeta struct { @@ -35,8 +28,9 @@ type OpenCodeSessionMeta struct { // OpenCodeSQLiteSessionExists reports whether a session row with // the given ID is present in the OpenCode SQLite database at // dbPath. Returns false when the file is missing, the schema is -// unexpected, or no row matches. Used by FindOpenCodeSourceFile -// so callers can distinguish "this DB has the session" from +// unexpected, or no row matches. Used by the OpenCode-format +// provider's source lookup so callers can distinguish "this DB has +// the session" from // "this DB exists but doesn't have it" — the latter must let // resolution continue to other configured roots. func OpenCodeSQLiteSessionExists(dbPath, sessionID string) bool { @@ -106,61 +100,10 @@ func ListOpenCodeSessionMeta( return metas, rows.Err() } -// ParseOpenCodeDB opens the OpenCode SQLite database read-only -// and returns all sessions with messages. -func ParseOpenCodeDB( - dbPath, machine string, -) ([]OpenCodeSession, error) { - if _, err := os.Stat(dbPath); os.IsNotExist(err) { - return nil, nil - } - - db, err := openOpenCodeDB(dbPath) - if err != nil { - return nil, err - } - defer db.Close() - - projects, err := loadOpenCodeProjects(db) - if err != nil { - return nil, fmt.Errorf( - "loading opencode projects: %w", err, - ) - } - - sessions, err := loadOpenCodeSessions(db) - if err != nil { - return nil, fmt.Errorf( - "loading opencode sessions: %w", err, - ) - } - - var results []OpenCodeSession - for _, s := range sessions { - worktree := projects[s.projectID] - parsed, msgs, err := buildOpenCodeSession( - db, s, worktree, dbPath, machine, - ) - if err != nil { - log.Printf( - "opencode session %s: %v", s.id, err, - ) - continue - } - if parsed == nil { - continue - } - results = append(results, OpenCodeSession{ - Session: *parsed, - Messages: msgs, - }) - } - return results, nil -} - -// ParseOpenCodeSession parses a single session by ID from the -// OpenCode database. -func ParseOpenCodeSession( +// parseOpenCodeDBSession parses a single session by ID from the +// OpenCode SQLite database. The OpenCode-format provider owns this +// path; Kilo and MiMoCode reuse it and relabel the result. +func parseOpenCodeDBSession( dbPath, sessionID, machine string, ) (*ParsedSession, []ParsedMessage, error) { if _, err := os.Stat(dbPath); os.IsNotExist(err) { @@ -196,9 +139,11 @@ func ParseOpenCodeSession( ) } -// ParseOpenCodeFile parses a file-backed OpenCode storage session -// rooted at storage/session//.json. -func ParseOpenCodeFile( +// parseOpenCodeStorageFile parses a file-backed OpenCode storage +// session rooted at storage/session//.json. The +// OpenCode-format provider owns this path; Kilo and MiMoCode reuse it +// and relabel the result. +func parseOpenCodeStorageFile( sessionPath, machine string, ) (*ParsedSession, []ParsedMessage, error) { raw, err := os.ReadFile(sessionPath) @@ -317,36 +262,6 @@ type openCodeSessionRow struct { timeUpdated int64 } -func loadOpenCodeSessions( - db *sql.DB, -) ([]openCodeSessionRow, error) { - rows, err := db.Query(` - SELECT s.id, s.project_id, - COALESCE(s.parent_id, ''), - COALESCE(s.title, ''), - s.time_created, s.time_updated - FROM session s - ORDER BY s.time_created - `) - if err != nil { - return nil, err - } - defer rows.Close() - - var sessions []openCodeSessionRow - for rows.Next() { - var s openCodeSessionRow - if err := rows.Scan( - &s.id, &s.projectID, &s.parentID, - &s.title, &s.timeCreated, &s.timeUpdated, - ); err != nil { - return nil, err - } - sessions = append(sessions, s) - } - return sessions, rows.Err() -} - func loadOneOpenCodeSession( db *sql.DB, sessionID string, ) (openCodeSessionRow, error) { @@ -1043,7 +958,9 @@ func OpenCodeSourceMtime(sourcePath string) (int64, error) { if sourcePath == "" { return 0, nil } - if dbPath, sessionID, ok := ParseOpenCodeSQLiteVirtualPath(sourcePath); ok { + if dbPath, sessionID, ok := parseOpenCodeFormatVirtualPath( + openCodeFmt.dbName, sourcePath, + ); ok { return openCodeSQLiteSessionMtime(dbPath, sessionID) } return openCodeStorageSessionMtime(sourcePath) diff --git a/internal/parser/opencode_provider.go b/internal/parser/opencode_provider.go new file mode 100644 index 000000000..0620fd2d6 --- /dev/null +++ b/internal/parser/opencode_provider.go @@ -0,0 +1,744 @@ +package parser + +import ( + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" +) + +var _ Provider = (*openCodeFormatProvider)(nil) + +type openCodeFormatProviderFactory struct { + def AgentDef + spec openCodeProviderSpec +} + +func newOpenCodeProviderFactory(def AgentDef) ProviderFactory { + return openCodeFormatProviderFactory{ + def: cloneAgentDef(def), + spec: openCodeProviderSpecForAgent(AgentOpenCode), + } +} + +func newKiloProviderFactory(def AgentDef) ProviderFactory { + return openCodeFormatProviderFactory{ + def: cloneAgentDef(def), + spec: openCodeProviderSpecForAgent(AgentKilo), + } +} + +func newMiMoCodeProviderFactory(def AgentDef) ProviderFactory { + return openCodeFormatProviderFactory{ + def: cloneAgentDef(def), + spec: openCodeProviderSpecForAgent(AgentMiMoCode), + } +} + +func (f openCodeFormatProviderFactory) Definition() AgentDef { + return cloneAgentDef(f.def) +} + +func (f openCodeFormatProviderFactory) Capabilities() Capabilities { + return openCodeFormatProviderCapabilities() +} + +func (f openCodeFormatProviderFactory) NewProvider(cfg ProviderConfig) Provider { + cfg = cfg.Clone() + return &openCodeFormatProvider{ + ProviderBase: ProviderBase{ + Def: cloneAgentDef(f.def), + Caps: openCodeFormatProviderCapabilities(), + Config: cfg, + }, + sources: newOpenCodeFormatSourceSet(cfg.Roots, f.spec), + } +} + +type openCodeFormatProvider struct { + ProviderBase + sources openCodeFormatSourceSet +} + +func (p *openCodeFormatProvider) Discover(ctx context.Context) ([]SourceRef, error) { + return p.sources.Discover(ctx) +} + +func (p *openCodeFormatProvider) WatchPlan(ctx context.Context) (WatchPlan, error) { + return p.sources.WatchPlan(ctx) +} + +func (p *openCodeFormatProvider) SourcesForChangedPath( + ctx context.Context, + req ChangedPathRequest, +) ([]SourceRef, error) { + return p.sources.SourcesForChangedPath(ctx, req) +} + +func (p *openCodeFormatProvider) FindSource( + ctx context.Context, + req FindSourceRequest, +) (SourceRef, bool, error) { + req = providerFindRequestWithRawSessionID(p.Def, req) + return p.sources.FindSource(ctx, req) +} + +func (p *openCodeFormatProvider) Fingerprint( + ctx context.Context, + source SourceRef, +) (SourceFingerprint, error) { + return p.sources.Fingerprint(ctx, source) +} + +func (p *openCodeFormatProvider) Parse( + ctx context.Context, + req ParseRequest, +) (ParseOutcome, error) { + if err := ctx.Err(); err != nil { + return ParseOutcome{}, err + } + path, ok := p.sources.pathFromSource(req.Source) + if !ok { + return ParseOutcome{}, fmt.Errorf("%s source path unavailable", p.Def.Type) + } + + machine := firstNonEmptyJSONLString(req.Machine, p.Config.Machine) + var ( + sess *ParsedSession + msgs []ParsedMessage + err error + ) + if dbPath, sessionID, ok := p.sources.spec.parseVirtual(path); ok { + sess, msgs, err = p.sources.spec.parseSQLite(dbPath, sessionID, machine) + } else { + sess, msgs, err = p.sources.spec.parseFile(path, machine) + } + if err != nil { + return ParseOutcome{}, err + } + if sess == nil { + return ParseOutcome{ + ResultSetComplete: true, + SkipReason: SkipNoSession, + }, nil + } + if req.Fingerprint.Hash != "" { + sess.File.Hash = req.Fingerprint.Hash + } + return ParseOutcome{ + Results: []ParseResultOutcome{{ + Result: ParseResult{ + Session: *sess, + Messages: msgs, + }, + DataVersion: DataVersionCurrent, + }}, + ResultSetComplete: true, + }, nil +} + +// openCodeProviderSpec parameterizes the one shared OpenCode-format +// provider implementation for OpenCode and its Kilo and MiMoCode forks. +// All three reuse the same discovery, source-lookup, fingerprinting, +// and parsing code; they differ only in the per-agent SQLite filename, +// the storage/ that holds session JSON, and the agent +// label/ID prefix applied via relabel. Kilo and MiMoCode parse through +// the OpenCode storage and SQLite readers, then relabel the result onto +// their own agent and ID prefix. +type openCodeProviderSpec struct { + agent AgentType + format openCodeFormat + dbName string + listSQLite func(string) ([]OpenCodeSessionMeta, error) + sourceMtime func(string) (int64, error) + relabel func(*ParsedSession) +} + +func openCodeProviderSpecForAgent(agent AgentType) openCodeProviderSpec { + switch agent { + case AgentOpenCode: + return openCodeProviderSpec{ + agent: AgentOpenCode, + format: openCodeFmt, + dbName: openCodeFmt.dbName, + listSQLite: ListOpenCodeSessionMeta, + sourceMtime: OpenCodeSourceMtime, + } + case AgentKilo: + return openCodeProviderSpec{ + agent: AgentKilo, + format: kiloFmt, + dbName: kiloFmt.dbName, + listSQLite: ListKiloSessionMeta, + sourceMtime: KiloSourceMtime, + relabel: relabelOpenCodeSessionAsKilo, + } + case AgentMiMoCode: + return openCodeProviderSpec{ + agent: AgentMiMoCode, + format: mimoFmt, + dbName: mimoFmt.dbName, + listSQLite: ListMiMoCodeSessionMeta, + sourceMtime: MiMoCodeSourceMtime, + relabel: relabelOpenCodeSessionAsMiMoCode, + } + default: + return openCodeProviderSpec{} + } +} + +// resolve detects the OpenCode storage backend for a root. +func (spec openCodeProviderSpec) resolve(root string) OpenCodeSource { + return resolveOpenCodeFormatSource(spec.format, root) +} + +// discover lists file-backed storage session JSON files under a root. +func (spec openCodeProviderSpec) discover(root string) []DiscoveredFile { + return discoverOpenCodeFormatSessions(spec.format, root) +} + +// find locates a session source path (storage JSON or SQLite virtual +// path) by raw session ID under a root. +func (spec openCodeProviderSpec) find(root, sessionID string) string { + return findOpenCodeFormatSourceFile(spec.format, root, sessionID) +} + +// watchRoots returns the directories that should be watched for live +// updates under a configured root. +func (spec openCodeProviderSpec) watchRoots(root string) []string { + return resolveOpenCodeFormatWatchRoots(spec.format, root) +} + +// storageIDs returns the set of session IDs present as storage JSON +// under a root, used to skip duplicate SQLite metas in hybrid roots. +func (spec openCodeProviderSpec) storageIDs(root string) map[string]struct{} { + return openCodeFormatStorageSessionIDs(spec.format, root) +} + +// parseVirtual splits an opencode-format SQLite virtual path +// (#) when the DB base name matches this agent. +func (spec openCodeProviderSpec) parseVirtual( + sourcePath string, +) (dbPath, sessionID string, ok bool) { + return parseOpenCodeFormatVirtualPath(spec.dbName, sourcePath) +} + +// parseFile parses a file-backed storage session and relabels it onto +// this agent's ID prefix when the agent is a fork of OpenCode. +func (spec openCodeProviderSpec) parseFile( + sessionPath, machine string, +) (*ParsedSession, []ParsedMessage, error) { + sess, msgs, err := parseOpenCodeStorageFile(sessionPath, machine) + if err != nil || sess == nil { + return sess, msgs, err + } + if spec.relabel != nil { + spec.relabel(sess) + } + return sess, msgs, nil +} + +// parseSQLite parses a single SQLite-backed session and relabels it +// onto this agent's ID prefix when the agent is a fork of OpenCode. +func (spec openCodeProviderSpec) parseSQLite( + dbPath, sessionID, machine string, +) (*ParsedSession, []ParsedMessage, error) { + sess, msgs, err := parseOpenCodeDBSession(dbPath, sessionID, machine) + if err != nil || sess == nil { + return sess, msgs, err + } + if spec.relabel != nil { + spec.relabel(sess) + } + return sess, msgs, nil +} + +type openCodeFormatSource struct { + Root string + Path string +} + +type openCodeFormatSourceSet struct { + roots []string + spec openCodeProviderSpec +} + +func newOpenCodeFormatSourceSet( + roots []string, + spec openCodeProviderSpec, +) openCodeFormatSourceSet { + return openCodeFormatSourceSet{ + roots: cleanJSONLRoots(roots), + spec: spec, + } +} + +func (s openCodeFormatSourceSet) Discover(ctx context.Context) ([]SourceRef, error) { + var sources []SourceRef + seen := make(map[string]struct{}) + for _, root := range s.roots { + if err := ctx.Err(); err != nil { + return nil, err + } + src := s.spec.resolve(root) + storageIDs := map[string]struct{}{} + if src.Mode == OpenCodeSourceStorage { + for _, file := range s.spec.discover(root) { + source, ok := s.sourceRef(root, file.Path, false) + if !ok { + continue + } + source.ProjectHint = file.Project + addJSONLSource(source, &sources, seen) + } + storageIDs = s.spec.storageIDs(root) + } + if src.DBPath == "" || !IsRegularFile(src.DBPath) { + continue + } + dbSources, err := s.sqliteSources(ctx, root, src.DBPath, storageIDs) + if err != nil { + return nil, err + } + for _, source := range dbSources { + addJSONLSource(source, &sources, seen) + } + } + sortJSONLSources(sources) + return sources, nil +} + +func (s openCodeFormatSourceSet) WatchPlan(context.Context) (WatchPlan, error) { + roots := make([]WatchRoot, 0, len(s.roots)) + for _, root := range s.roots { + for _, watchRoot := range s.spec.watchRoots(root) { + roots = append(roots, WatchRoot{ + Path: watchRoot, + Recursive: true, + IncludeGlobs: []string{ + "*.json", + s.spec.dbName, + s.spec.dbName + "-*", + }, + DebounceKey: string(s.spec.agent) + ":opencode:" + watchRoot, + }) + } + } + return WatchPlan{Roots: roots}, nil +} + +func (s openCodeFormatSourceSet) SourcesForChangedPath( + ctx context.Context, + req ChangedPathRequest, +) ([]SourceRef, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + pathExists := true + if _, err := os.Stat(req.Path); err != nil { + if !os.IsNotExist(err) { + return nil, nil + } + pathExists = false + } + for _, root := range s.roots { + sources, ok, err := s.sourcesForChangedPathInRoot( + ctx, root, req.Path, pathExists, + ) + if err != nil || ok { + return sources, err + } + } + return nil, nil +} + +func (s openCodeFormatSourceSet) FindSource( + ctx context.Context, + req FindSourceRequest, +) (SourceRef, bool, error) { + if err := ctx.Err(); err != nil { + return SourceRef{}, false, err + } + for _, path := range []string{req.StoredFilePath, req.FingerprintKey} { + if path == "" { + continue + } + for _, root := range s.roots { + if source, ok := s.sourceRef(root, path, true); ok { + return source, true, nil + } + } + } + if req.RawSessionID == "" { + return SourceRef{}, false, nil + } + for _, root := range s.roots { + path := s.spec.find(root, req.RawSessionID) + if path == "" { + continue + } + if source, ok := s.sourceRef(root, path, false); ok { + return source, true, nil + } + } + return SourceRef{}, false, nil +} + +func (s openCodeFormatSourceSet) Fingerprint( + ctx context.Context, + source SourceRef, +) (SourceFingerprint, error) { + if err := ctx.Err(); err != nil { + return SourceFingerprint{}, err + } + path, ok := s.pathFromSource(source) + if !ok { + return SourceFingerprint{}, fmt.Errorf("%s source path unavailable", s.spec.agent) + } + mtime, err := s.spec.sourceMtime(path) + if err != nil { + return SourceFingerprint{}, err + } + fingerprint := SourceFingerprint{ + Key: firstNonEmptyJSONLString(source.FingerprintKey, source.Key, path), + MTimeNS: mtime, + } + if dbPath, _, ok := s.spec.parseVirtual(path); ok { + info, err := os.Stat(dbPath) + if err != nil { + return SourceFingerprint{}, fmt.Errorf("stat %s: %w", dbPath, err) + } + fingerprint.Size = info.Size() + return fingerprint, nil + } + info, err := os.Stat(path) + if err != nil { + return SourceFingerprint{}, fmt.Errorf("stat %s: %w", path, err) + } + if info.IsDir() { + return SourceFingerprint{}, fmt.Errorf("stat %s: source is a directory", path) + } + fingerprint.Size = info.Size() + fingerprint.Hash, err = openCodeProviderStorageFingerprint(path) + if err != nil { + return SourceFingerprint{}, err + } + return fingerprint, nil +} + +func (s openCodeFormatSourceSet) pathFromSource(source SourceRef) (string, bool) { + switch src := source.Opaque.(type) { + case openCodeFormatSource: + return src.Path, src.Path != "" + case *openCodeFormatSource: + if src != nil && src.Path != "" { + return src.Path, true + } + } + for _, candidate := range []string{ + source.DisplayPath, + source.FingerprintKey, + source.Key, + } { + for _, root := range s.roots { + if ref, ok := s.sourceRef(root, candidate, false); ok { + src := ref.Opaque.(openCodeFormatSource) + return src.Path, true + } + } + } + return "", false +} + +func (s openCodeFormatSourceSet) sqliteSources( + ctx context.Context, + root string, + dbPath string, + storageIDs map[string]struct{}, +) ([]SourceRef, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + metas, err := s.spec.listSQLite(dbPath) + if err != nil { + return nil, err + } + sources := make([]SourceRef, 0, len(metas)) + for _, meta := range metas { + if _, exists := storageIDs[meta.SessionID]; exists { + continue + } + source, ok := s.sourceRef(root, meta.VirtualPath, false) + if !ok { + continue + } + sources = append(sources, source) + } + return sources, nil +} + +func (s openCodeFormatSourceSet) sourcesForChangedPathInRoot( + ctx context.Context, + root string, + path string, + pathExists bool, +) ([]SourceRef, bool, error) { + rel, ok := relUnder(root, path) + if !ok { + return nil, false, nil + } + base := filepath.Base(rel) + if rel == s.spec.dbName || strings.HasPrefix(base, s.spec.dbName+"-") { + dbPath := filepath.Join(root, s.spec.dbName) + if !IsRegularFile(dbPath) { + return nil, true, nil + } + storageIDs := map[string]struct{}{} + if s.spec.resolve(root).Mode == OpenCodeSourceStorage { + storageIDs = s.spec.storageIDs(root) + } + sources, err := s.sqliteSources(ctx, root, dbPath, storageIDs) + return sources, true, err + } + + src := s.spec.resolve(root) + if src.Mode != OpenCodeSourceStorage { + return nil, false, nil + } + parts := strings.Split(rel, string(filepath.Separator)) + sessionSubdir := filepath.Base(src.SessionRoot) + switch { + case pathExists && + len(parts) == 4 && + parts[0] == "storage" && + parts[1] == sessionSubdir && + strings.HasSuffix(parts[3], ".json"): + source, ok := s.sourceRef(root, path, false) + if !ok { + return nil, true, nil + } + return []SourceRef{source}, true, nil + case !pathExists && + len(parts) == 4 && + parts[0] == "storage" && + parts[1] == sessionSubdir && + strings.HasSuffix(parts[3], ".json"): + source, ok := s.sourceRefFromStoragePath(root, path) + if !ok { + return nil, true, nil + } + return []SourceRef{source}, true, nil + case len(parts) == 4 && + parts[0] == "storage" && + parts[1] == "message" && + strings.HasSuffix(parts[3], ".json"): + source, ok := s.sourceForRawID(root, parts[2]) + if !ok { + return nil, false, nil + } + return []SourceRef{source}, true, nil + case len(parts) == 4 && + parts[0] == "storage" && + parts[1] == "part" && + strings.HasSuffix(parts[3], ".json"): + sessionID := "" + if pathExists { + sessionID = readOpenCodeProviderStorageSessionID(path) + } + if sessionID == "" { + sessionID = findOpenCodeProviderStorageSessionIDByMessageID(root, parts[2]) + } + if sessionID == "" { + return nil, false, nil + } + source, ok := s.sourceForRawID(root, sessionID) + if !ok { + return nil, false, nil + } + return []SourceRef{source}, true, nil + case !pathExists && + len(parts) == 3 && + parts[0] == "storage" && + parts[1] == "message": + source, ok := s.sourceForRawID(root, parts[2]) + if !ok { + return nil, false, nil + } + return []SourceRef{source}, true, nil + case !pathExists && + len(parts) == 3 && + parts[0] == "storage" && + parts[1] == "part": + sessionID := findOpenCodeProviderStorageSessionIDByMessageID(root, parts[2]) + if sessionID == "" { + return nil, false, nil + } + source, ok := s.sourceForRawID(root, sessionID) + if !ok { + return nil, false, nil + } + return []SourceRef{source}, true, nil + } + return nil, false, nil +} + +func (s openCodeFormatSourceSet) sourceForRawID(root, sessionID string) (SourceRef, bool) { + path := s.spec.find(root, sessionID) + if path == "" { + return SourceRef{}, false + } + return s.sourceRef(root, path, false) +} + +func (s openCodeFormatSourceSet) sourceRef( + root string, + path string, + promoteVirtual bool, +) (SourceRef, bool) { + root = filepath.Clean(root) + path = filepath.Clean(path) + if dbPath, sessionID, ok := s.spec.parseVirtual(path); ok { + if _, under := relUnder(root, dbPath); !under { + return SourceRef{}, false + } + if promoteVirtual { + if selected := s.spec.find(root, sessionID); selected != "" && + selected != path { + return s.sourceRef(root, selected, false) + } + } + if !OpenCodeSQLiteSessionExists(dbPath, sessionID) { + return SourceRef{}, false + } + return s.newSourceRef(root, path, ""), true + } + if !s.isStorageSessionPath(root, path, true) { + return SourceRef{}, false + } + return s.sourceRefFromStoragePath(root, path) +} + +func (s openCodeFormatSourceSet) sourceRefFromStoragePath( + root string, + path string, +) (SourceRef, bool) { + if !s.isStorageSessionPath(root, path, false) { + return SourceRef{}, false + } + return s.newSourceRef(root, path, openCodeSessionProject(path)), true +} + +func (s openCodeFormatSourceSet) newSourceRef( + root string, + path string, + project string, +) SourceRef { + return SourceRef{ + Provider: s.spec.agent, + Key: path, + DisplayPath: path, + FingerprintKey: path, + ProjectHint: project, + Opaque: openCodeFormatSource{ + Root: root, + Path: path, + }, + } +} + +func (s openCodeFormatSourceSet) isStorageSessionPath( + root string, + path string, + requireExisting bool, +) bool { + rel, ok := relUnder(root, path) + if !ok { + return false + } + src := s.spec.resolve(root) + if src.Mode != OpenCodeSourceStorage { + return false + } + parts := strings.Split(rel, string(filepath.Separator)) + return len(parts) == 4 && + parts[0] == "storage" && + parts[1] == filepath.Base(src.SessionRoot) && + strings.HasSuffix(parts[3], ".json") && + (!requireExisting || IsRegularFile(path)) +} + +func openCodeProviderStorageFingerprint(sessionPath string) (string, error) { + root := filepath.Dir(filepath.Dir(filepath.Dir(filepath.Dir(sessionPath)))) + sessionID := strings.TrimSuffix(filepath.Base(sessionPath), filepath.Ext(sessionPath)) + msgs, err := loadOpenCodeStorageMessages(root, sessionID) + if err != nil { + return "", err + } + parts, err := loadOpenCodeStorageParts(root, msgs) + if err != nil { + return "", err + } + return buildOpenCodeStorageFingerprint(msgs, parts), nil +} + +func readOpenCodeProviderStorageSessionID(path string) string { + raw, err := os.ReadFile(path) + if err != nil { + return "" + } + var data struct { + SessionID string `json:"sessionID"` + } + if err := json.Unmarshal(raw, &data); err != nil { + return "" + } + return data.SessionID +} + +func findOpenCodeProviderStorageSessionIDByMessageID( + openCodeDir, messageID string, +) string { + messageRoot := filepath.Join(openCodeDir, "storage", "message") + entries, err := os.ReadDir(messageRoot) + if err != nil { + return "" + } + for _, entry := range entries { + if !entry.IsDir() { + continue + } + path := filepath.Join(messageRoot, entry.Name(), messageID+".json") + if info, err := os.Stat(path); err == nil && !info.IsDir() { + return entry.Name() + } + } + return "" +} + +func openCodeFormatProviderCapabilities() Capabilities { + return Capabilities{ + Source: SourceCapabilities{ + DiscoverSources: CapabilitySupported, + WatchSources: CapabilitySupported, + ClassifyChangedPath: CapabilitySupported, + FindSource: CapabilitySupported, + CompositeFingerprint: CapabilitySupported, + IncrementalAppend: CapabilityNotApplicable, + MultiSessionSource: CapabilityNotApplicable, + PerSessionErrors: CapabilityNotApplicable, + ExcludedSessions: CapabilityNotApplicable, + ForceReplaceOnParse: CapabilityNotApplicable, + }, + Content: ContentCapabilities{ + FirstMessage: CapabilitySupported, + Cwd: CapabilitySupported, + Relationships: CapabilitySupported, + Thinking: CapabilitySupported, + ToolCalls: CapabilitySupported, + PerMessageTokenUsage: CapabilitySupported, + Model: CapabilitySupported, + }, + } +} diff --git a/internal/parser/opencode_provider_test.go b/internal/parser/opencode_provider_test.go new file mode 100644 index 000000000..4ee13b120 --- /dev/null +++ b/internal/parser/opencode_provider_test.go @@ -0,0 +1,347 @@ +package parser + +import ( + "context" + "database/sql" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestOpenCodeFamilyProviderFactoriesReplaceLegacyAdapter(t *testing.T) { + for _, agent := range []AgentType{ + AgentOpenCode, + AgentKilo, + AgentMiMoCode, + } { + t.Run(string(agent), func(t *testing.T) { + factory, ok := ProviderFactoryByType(agent) + require.True(t, ok) + require.NotNil(t, factory) + + provider, ok := NewProvider(agent, ProviderConfig{ + Roots: []string{t.TempDir()}, + Machine: "devbox", + }) + require.True(t, ok) + require.NotNil(t, provider) + }) + } +} + +func TestOpenCodeProviderStorageSourceMethods(t *testing.T) { + root := t.TempDir() + sessionPath := writeOpenCodeProviderStorageSession( + t, root, "session", "ses_provider", "opencode-app", "Provider Session", + ) + + provider, ok := NewProvider(AgentOpenCode, ProviderConfig{ + Roots: []string{root}, + Machine: "devbox", + }) + require.True(t, ok) + + plan, err := provider.WatchPlan(context.Background()) + require.NoError(t, err) + require.Len(t, plan.Roots, 1) + assert.Equal(t, filepath.Join(root, "storage"), plan.Roots[0].Path) + assert.True(t, plan.Roots[0].Recursive) + + discovered, err := provider.Discover(context.Background()) + require.NoError(t, err) + require.Len(t, discovered, 1) + source := discovered[0] + assert.Equal(t, AgentOpenCode, source.Provider) + assert.Equal(t, sessionPath, source.DisplayPath) + assert.Equal(t, sessionPath, source.FingerprintKey) + assert.Equal(t, "opencode_app", source.ProjectHint) + + found, ok, err := provider.FindSource(context.Background(), FindSourceRequest{ + FullSessionID: "remote~opencode:ses_provider", + }) + require.NoError(t, err) + require.True(t, ok) + assert.Equal(t, sessionPath, found.DisplayPath) + + messagePath := filepath.Join( + root, "storage", "message", "ses_provider", "msg_1.json", + ) + partPath := filepath.Join(root, "storage", "part", "msg_1", "prt_1.json") + for _, tc := range []struct { + name string + path string + }{ + {name: "session", path: sessionPath}, + {name: "message", path: messagePath}, + {name: "part", path: partPath}, + } { + t.Run(tc.name, func(t *testing.T) { + changed, err := provider.SourcesForChangedPath( + context.Background(), + ChangedPathRequest{ + Path: tc.path, + EventKind: "write", + WatchRoot: filepath.Join(root, "storage"), + }, + ) + require.NoError(t, err) + require.Len(t, changed, 1) + assert.Equal(t, sessionPath, changed[0].DisplayPath) + }) + } + + fingerprint, err := provider.Fingerprint(context.Background(), found) + require.NoError(t, err) + assert.Equal(t, sessionPath, fingerprint.Key) + assert.Positive(t, fingerprint.Size) + assert.Positive(t, fingerprint.MTimeNS) + assert.True(t, HasOpenCodeStorageFingerprint(fingerprint.Hash)) + + outcome, err := provider.Parse(context.Background(), ParseRequest{ + Source: found, + Fingerprint: fingerprint, + }) + require.NoError(t, err) + require.True(t, outcome.ResultSetComplete) + require.Len(t, outcome.Results, 1) + result := outcome.Results[0] + assert.Equal(t, DataVersionCurrent, result.DataVersion) + assert.Equal(t, "opencode:ses_provider", result.Result.Session.ID) + assert.Equal(t, AgentOpenCode, result.Result.Session.Agent) + assert.Equal(t, "opencode_app", result.Result.Session.Project) + assert.Equal(t, "devbox", result.Result.Session.Machine) + assert.Equal(t, fingerprint.Hash, result.Result.Session.File.Hash) + assert.Len(t, result.Result.Messages, 1) + + require.NoError(t, os.Remove(sessionPath), "remove storage session") + removed, err := provider.SourcesForChangedPath( + context.Background(), + ChangedPathRequest{ + Path: sessionPath, + EventKind: "remove", + WatchRoot: filepath.Join(root, "storage"), + }, + ) + require.NoError(t, err) + require.Len(t, removed, 1) + assert.Equal(t, sessionPath, removed[0].DisplayPath) + assert.Equal(t, "global", removed[0].ProjectHint) +} + +func TestOpenCodeProviderSQLiteSourceMethods(t *testing.T) { + root := t.TempDir() + dbPath, seeder, db := newTestDBAt(t, filepath.Join(root, "opencode.db")) + defer db.Close() + seeder.AddProject("prj_1", "/home/user/code/sqlite-app") + seeder.AddSession("ses_sqlite", "prj_1", "", "SQLite Session", 1700000000000, 1700000060000) + seeder.AddMessage("msg_1", "ses_sqlite", 1700000000000, 1700000000000, `{"role":"user"}`) + seeder.AddPart("prt_1", "msg_1", "ses_sqlite", 1700000000000, 1700000000000, `{"type":"text","text":"Hello from sqlite"}`) + virtualPath := OpenCodeSQLiteVirtualPath(dbPath, "ses_sqlite") + + provider, ok := NewProvider(AgentOpenCode, ProviderConfig{ + Roots: []string{root}, + Machine: "devbox", + }) + require.True(t, ok) + + plan, err := provider.WatchPlan(context.Background()) + require.NoError(t, err) + require.Len(t, plan.Roots, 1) + assert.Equal(t, root, plan.Roots[0].Path) + assert.True(t, plan.Roots[0].Recursive) + + discovered, err := provider.Discover(context.Background()) + require.NoError(t, err) + require.Len(t, discovered, 1) + assert.Equal(t, virtualPath, discovered[0].DisplayPath) + assert.Equal(t, virtualPath, discovered[0].FingerprintKey) + + for _, path := range []string{dbPath, dbPath + "-wal"} { + changed, err := provider.SourcesForChangedPath( + context.Background(), + ChangedPathRequest{Path: path, EventKind: "write", WatchRoot: root}, + ) + require.NoError(t, err) + require.Len(t, changed, 1) + assert.Equal(t, virtualPath, changed[0].DisplayPath) + } + + found, ok, err := provider.FindSource(context.Background(), FindSourceRequest{ + FullSessionID: "host~opencode:ses_sqlite", + }) + require.NoError(t, err) + require.True(t, ok) + assert.Equal(t, virtualPath, found.DisplayPath) + + fingerprint, err := provider.Fingerprint(context.Background(), found) + require.NoError(t, err) + assert.Equal(t, virtualPath, fingerprint.Key) + assert.Equal(t, int64(1700000060000)*1_000_000, fingerprint.MTimeNS) + + outcome, err := provider.Parse(context.Background(), ParseRequest{ + Source: found, + Fingerprint: fingerprint, + }) + require.NoError(t, err) + require.True(t, outcome.ResultSetComplete) + require.Len(t, outcome.Results, 1) + result := outcome.Results[0] + assert.Equal(t, DataVersionCurrent, result.DataVersion) + assert.Equal(t, "opencode:ses_sqlite", result.Result.Session.ID) + assert.Equal(t, "sqlite_app", result.Result.Session.Project) + assert.Equal(t, "devbox", result.Result.Session.Machine) + assert.Equal(t, "Hello from sqlite", result.Result.Messages[0].Content) + + require.NoError(t, db.Close()) + require.NoError(t, os.Remove(dbPath), "remove sqlite db") + removed, err := provider.SourcesForChangedPath( + context.Background(), + ChangedPathRequest{Path: dbPath, EventKind: "remove", WatchRoot: root}, + ) + require.NoError(t, err) + assert.Empty(t, removed, "removed sqlite DBs have no stateless virtual source list") +} + +func TestOpenCodeProviderHybridDiscoveryFiltersSQLiteDuplicate(t *testing.T) { + root := t.TempDir() + storagePath := writeOpenCodeProviderStorageSession( + t, root, "session", "ses_dup", "storage-app", "Storage Session", + ) + dbPath, seeder, db := newTestDBAt(t, filepath.Join(root, "opencode.db")) + defer db.Close() + seeder.AddProject("prj_1", "/home/user/code/sqlite-app") + seeder.AddSession("ses_dup", "prj_1", "", "Duplicate", 1700000000000, 1700000010000) + seeder.AddSession("ses_db_only", "prj_1", "", "DB Only", 1700000000000, 1700000020000) + virtualOnly := OpenCodeSQLiteVirtualPath(dbPath, "ses_db_only") + + provider, ok := NewProvider(AgentOpenCode, ProviderConfig{Roots: []string{root}}) + require.True(t, ok) + + discovered, err := provider.Discover(context.Background()) + require.NoError(t, err) + require.Len(t, discovered, 2) + assert.ElementsMatch(t, []string{storagePath, virtualOnly}, []string{ + discovered[0].DisplayPath, + discovered[1].DisplayPath, + }) + + found, ok, err := provider.FindSource(context.Background(), FindSourceRequest{ + StoredFilePath: OpenCodeSQLiteVirtualPath(dbPath, "ses_dup"), + FullSessionID: "opencode:ses_dup", + }) + require.NoError(t, err) + require.True(t, ok) + assert.Equal(t, storagePath, found.DisplayPath) +} + +func TestOpenCodeFamilyProviderRelabelsForks(t *testing.T) { + for _, tc := range []struct { + agent AgentType + sessionSubdir string + prefix string + project string + }{ + {agent: AgentKilo, sessionSubdir: "session", prefix: "kilo:", project: "kilo-app"}, + {agent: AgentMiMoCode, sessionSubdir: "session_diff", prefix: "mimocode:", project: "mimo-app"}, + } { + t.Run(string(tc.agent), func(t *testing.T) { + root := t.TempDir() + sessionPath := writeOpenCodeProviderStorageSession( + t, root, tc.sessionSubdir, "ses_provider", tc.project, "Provider Session", + ) + provider, ok := NewProvider(tc.agent, ProviderConfig{ + Roots: []string{root}, + Machine: "devbox", + }) + require.True(t, ok) + source, ok, err := provider.FindSource(context.Background(), FindSourceRequest{ + FullSessionID: "host~" + tc.prefix + "ses_provider", + }) + require.NoError(t, err) + require.True(t, ok) + assert.Equal(t, sessionPath, source.DisplayPath) + + outcome, err := provider.Parse(context.Background(), ParseRequest{ + Source: source, + }) + require.NoError(t, err) + require.True(t, outcome.ResultSetComplete) + require.Len(t, outcome.Results, 1) + result := outcome.Results[0].Result + assert.Equal(t, tc.prefix+"ses_provider", result.Session.ID) + assert.Equal(t, tc.agent, result.Session.Agent) + assert.Equal(t, strings.ReplaceAll(tc.project, "-", "_"), result.Session.Project) + + require.NoError(t, os.Remove(sessionPath), "remove storage session") + removed, err := provider.SourcesForChangedPath( + context.Background(), + ChangedPathRequest{ + Path: sessionPath, + EventKind: "rename", + WatchRoot: filepath.Join(root, "storage"), + }, + ) + require.NoError(t, err) + require.Len(t, removed, 1) + assert.Equal(t, sessionPath, removed[0].DisplayPath) + }) + } +} + +func writeOpenCodeProviderStorageSession( + t *testing.T, + root, sessionSubdir, sessionID, project, title string, +) string { + t.Helper() + sessionPath := filepath.Join( + root, "storage", sessionSubdir, "global", sessionID+".json", + ) + writeOpenCodeStorageFile(t, sessionPath, map[string]any{ + "id": sessionID, + "directory": filepath.Join("/home/user/code", project), + "title": title, + "time": map[string]any{ + "created": int64(1700000000000), + "updated": int64(1700000060000), + }, + }) + writeOpenCodeStorageFile(t, filepath.Join( + root, "storage", "message", sessionID, "msg_1.json", + ), map[string]any{ + "id": "msg_1", + "sessionID": sessionID, + "role": "user", + "time": map[string]any{ + "created": int64(1700000000000), + }, + }) + writeOpenCodeStorageFile(t, filepath.Join( + root, "storage", "part", "msg_1", "prt_1.json", + ), map[string]any{ + "id": "prt_1", + "sessionID": sessionID, + "messageID": "msg_1", + "type": "text", + "text": "Hello from storage", + "time": map[string]any{ + "created": int64(1700000000000), + }, + }) + return sessionPath +} + +func newTestDBAt( + t *testing.T, + dbPath string, +) (string, *OpenCodeSeeder, *sql.DB) { + t.Helper() + db, err := sql.Open("sqlite3", dbPath) + require.NoError(t, err, "open test db") + _, err = db.Exec(openCodeSchema) + require.NoError(t, err, "create schema") + return dbPath, &OpenCodeSeeder{db: db, t: t}, db +} diff --git a/internal/parser/opencode_test.go b/internal/parser/opencode_test.go index 1e7e95888..950a418bc 100644 --- a/internal/parser/opencode_test.go +++ b/internal/parser/opencode_test.go @@ -12,6 +12,29 @@ import ( "github.com/stretchr/testify/require" ) +// parseOpenCodeAll parses every session in an OpenCode SQLite database using +// the same per-session primitives the provider uses (ListOpenCodeSessionMeta + +// parseOpenCodeDBSession), reproducing the deleted ParseOpenCodeDB +// whole-database free function for the retained parse tests. +func parseOpenCodeAll(dbPath, machine string) ([]ParseResult, error) { + metas, err := ListOpenCodeSessionMeta(dbPath) + if err != nil { + return nil, err + } + var out []ParseResult + for _, m := range metas { + sess, msgs, err := parseOpenCodeDBSession(dbPath, m.SessionID, machine) + if err != nil { + return nil, err + } + if sess == nil { + continue + } + out = append(out, ParseResult{Session: *sess, Messages: msgs}) + } + return out, nil +} + // openCodeSchema matches the real OpenCode database schema. // Role and part type live inside the JSON data columns. const openCodeSchema = ` @@ -113,7 +136,7 @@ func newTestDB(t *testing.T) (string, *OpenCodeSeeder, *sql.DB) { // seedHybridSQLiteDB creates an OpenCode-shaped SQLite DB at // dbPath containing a single session row with the given ID. Used -// by tests that exercise FindOpenCodeSourceFile in hybrid and +// by tests that exercise OpenCode-format source lookup in hybrid and // pure-SQLite roots, where a real DB file (not just a marker) is // required. func seedHybridSQLiteDB(t *testing.T, dbPath, sessionID string) { @@ -166,7 +189,7 @@ func TestParseOpenCodeDB_StandardSession(t *testing.T) { defer db.Close() seedStandardSession(t, seeder) - sessions, err := ParseOpenCodeDB(dbPath, "testmachine") + sessions, err := parseOpenCodeAll(dbPath, "testmachine") require.NoError(t, err, "ParseOpenCodeDB") assertEq(t, "sessions len", len(sessions), 1) @@ -278,10 +301,10 @@ func TestParseOpenCodeFile_StorageSession(t *testing.T) { }, }) - sess, msgs, err := ParseOpenCodeFile( + sess, msgs, err := parseOpenCodeStorageFile( sessionPath, "testmachine", ) - require.NoError(t, err, "ParseOpenCodeFile") + require.NoError(t, err, "parseOpenCodeStorageFile") require.NotNil(t, sess, "expected non-nil session") assertEq(t, "ID", sess.ID, "opencode:ses_storage") @@ -363,10 +386,10 @@ func TestParseOpenCodeFile_StorageSessionInvalidChildFails( root, "storage", "part", "msg_1", "prt_bad.json", ), []byte(`{"id":"prt_bad"`), 0o644), "write invalid part") - sess, msgs, err := ParseOpenCodeFile( + sess, msgs, err := parseOpenCodeStorageFile( sessionPath, "testmachine", ) - require.Error(t, err, "expected ParseOpenCodeFile error") + require.Error(t, err, "expected parseOpenCodeStorageFile error") assert.Nil(t, sess, "session, want nil") assert.Nil(t, msgs, "msgs, want nil") } @@ -397,10 +420,10 @@ func TestParseOpenCodeFile_MissingPartDirAllowed(t *testing.T) { }, }) - sess, msgs, err := ParseOpenCodeFile( + sess, msgs, err := parseOpenCodeStorageFile( sessionPath, "testmachine", ) - require.NoError(t, err, "ParseOpenCodeFile") + require.NoError(t, err, "parseOpenCodeStorageFile") assert.Nil(t, sess, "session, want nil") assert.Nil(t, msgs, "msgs, want nil") } @@ -429,10 +452,10 @@ func TestParseOpenCodeFile_StorageMessageMissingIDFails(t *testing.T) { }, }) - sess, msgs, err := ParseOpenCodeFile( + sess, msgs, err := parseOpenCodeStorageFile( sessionPath, "testmachine", ) - require.Error(t, err, "expected ParseOpenCodeFile error") + require.Error(t, err, "expected parseOpenCodeStorageFile error") assert.Nil(t, sess, "session, want nil") assert.Nil(t, msgs, "msgs, want nil") } @@ -472,10 +495,10 @@ func TestParseOpenCodeFile_StoragePartMissingIDFails(t *testing.T) { }, }) - sess, msgs, err := ParseOpenCodeFile( + sess, msgs, err := parseOpenCodeStorageFile( sessionPath, "testmachine", ) - require.Error(t, err, "expected ParseOpenCodeFile error") + require.Error(t, err, "expected parseOpenCodeStorageFile error") assert.Nil(t, sess, "session, want nil") assert.Nil(t, msgs, "msgs, want nil") } @@ -531,8 +554,8 @@ func TestParseOpenCodeFile_StoragePartOrderingUsesStartTime( }, }) - _, msgs, err := ParseOpenCodeFile(sessionPath, "testmachine") - require.NoError(t, err, "ParseOpenCodeFile") + _, msgs, err := parseOpenCodeStorageFile(sessionPath, "testmachine") + require.NoError(t, err, "parseOpenCodeStorageFile") require.Len(t, msgs, 1, "messages len") assertEq(t, "msg[0].Content", msgs[0].Content, "first\nsecond") } @@ -590,8 +613,8 @@ func TestParseOpenCodeFile_StoragePartOrderingPrefersStartOverCreated( }, }) - _, msgs, err := ParseOpenCodeFile(sessionPath, "testmachine") - require.NoError(t, err, "ParseOpenCodeFile") + _, msgs, err := parseOpenCodeStorageFile(sessionPath, "testmachine") + require.NoError(t, err, "parseOpenCodeStorageFile") require.Len(t, msgs, 1, "messages len") assertEq(t, "msg[0].Content", msgs[0].Content, "first\nsecond") } @@ -653,8 +676,8 @@ func TestParseOpenCodeFile_StorageStepFinishTokens(t *testing.T) { }, }) - sess, msgs, err := ParseOpenCodeFile(sessionPath, "testmachine") - require.NoError(t, err, "ParseOpenCodeFile") + sess, msgs, err := parseOpenCodeStorageFile(sessionPath, "testmachine") + require.NoError(t, err, "parseOpenCodeStorageFile") require.NotNil(t, sess, "want one parsed session") require.Len(t, msgs, 1, "messages") @@ -700,7 +723,7 @@ func TestParseOpenCodeDB_TitleFallback(t *testing.T) { 1700000020000, 1700000020000, `{"type":"text","text":"Refactor the auth module"}`) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") assertEq(t, "sessions len", len(sessions), 2) @@ -733,7 +756,7 @@ func TestParseOpenCodeDB_ToolParts(t *testing.T) { seeder.AddPart("prt_t", "msg_a", "ses_tools", 1700000011000, 1700000011000, `{"type":"tool","tool":"read","callID":"call_1","state":{"input":{"file_path":"main.go"}}}`) seeder.AddPart("prt_txt", "msg_a", "ses_tools", 1700000012000, 1700000012000, `{"type":"text","text":"Here is the file content."}`) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") assertEq(t, "sessions len", len(sessions), 1) @@ -760,7 +783,7 @@ func TestParseOpenCodeDB_EmptySession(t *testing.T) { seeder.AddProject("prj_1", "/tmp/proj") seeder.AddSession("ses_empty", "prj_1", "", "", 1700000000000, 1700000000000) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") assertEq(t, "sessions len", len(sessions), 0) @@ -769,7 +792,7 @@ func TestParseOpenCodeDB_EmptySession(t *testing.T) { func TestParseOpenCodeDB_NonexistentDB(t *testing.T) { dbPath := filepath.Join(t.TempDir(), "nonexistent.db") - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "expected nil error") assert.Nil(t, sessions, "expected nil sessions") } @@ -788,7 +811,7 @@ func TestParseOpenCodeDB_ProjectFromWorktree(t *testing.T) { seeder.AddMessage("msg_1", "ses_git", 1700000000000, 1700000000000, `{"role":"user"}`) seeder.AddPart("prt_1", "msg_1", "ses_git", 1700000000000, 1700000000000, `{"type":"text","text":"hello"}`) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") assertEq(t, "sessions len", len(sessions), 1) @@ -800,8 +823,8 @@ func TestParseOpenCodeSession_SingleSession(t *testing.T) { defer db.Close() seedStandardSession(t, seeder) - sess, msgs, err := ParseOpenCodeSession(dbPath, "ses_abc", "testmachine") - require.NoError(t, err, "ParseOpenCodeSession") + sess, msgs, err := parseOpenCodeDBSession(dbPath, "ses_abc", "testmachine") + require.NoError(t, err, "parseOpenCodeDBSession") require.NotNil(t, sess, "expected non-nil session") assertEq(t, "ID", sess.ID, "opencode:ses_abc") @@ -835,7 +858,7 @@ func TestParseOpenCodeDB_OrdinalContinuity(t *testing.T) { seeder.AddMessage("msg_5", "ses_ord", 1700000040000, 1700000040000, `{"role":"user"}`) seeder.AddPart("prt_5", "msg_5", "ses_ord", 1700000040000, 1700000040000, `{"type":"text","text":"follow up"}`) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") assertEq(t, "sessions len", len(sessions), 1) @@ -866,10 +889,10 @@ func TestParseOpenCodeDB_ParentSession(t *testing.T) { seeder.AddMessage("msg_c", "ses_child", 1700000020000, 1700000020000, `{"role":"user"}`) seeder.AddPart("prt_c", "msg_c", "ses_child", 1700000020000, 1700000020000, `{"type":"text","text":"child msg"}`) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") - var child *OpenCodeSession + var child *ParseResult for i := range sessions { if sessions[i].Session.ID == "opencode:ses_child" { child = &sessions[i] @@ -951,7 +974,7 @@ func TestParseOpenCodeDB_TokenUsage(t *testing.T) { 1700000020000, 1700000020000, `{"type":"text","text":"answer2"}`) - sessions, err := ParseOpenCodeDB(dbPath, "testmachine") + sessions, err := parseOpenCodeAll(dbPath, "testmachine") require.NoError(t, err, "ParseOpenCodeDB") require.Len(t, sessions, 1, "sessions len") s := sessions[0] @@ -1046,7 +1069,7 @@ func TestParseOpenCodeDB_UnknownTokensShape(t *testing.T) { 1700000005000, 1700000005000, `{"type":"text","text":"answer"}`) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") require.Len(t, sessions, 1, "sessions len") @@ -1103,7 +1126,7 @@ func TestParseOpenCodeDB_ZeroTokens(t *testing.T) { 1700000005000, 1700000005000, `{"type":"text","text":"sorry, request failed"}`) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") require.Len(t, sessions, 1, "sessions len") @@ -1156,7 +1179,7 @@ func TestParseOpenCodeDB_NoTokenUsage(t *testing.T) { 1700000005000, 1700000005000, `{"type":"text","text":"oops"}`) - sessions, err := ParseOpenCodeDB(dbPath, "m") + sessions, err := parseOpenCodeAll(dbPath, "m") require.NoError(t, err, "ParseOpenCodeDB") require.Len(t, sessions, 1, "sessions len") diff --git a/internal/parser/provider.go b/internal/parser/provider.go index d3740085f..f8705fc57 100644 --- a/internal/parser/provider.go +++ b/internal/parser/provider.go @@ -374,8 +374,14 @@ func providerFactoryForDef(def AgentDef) ProviderFactory { return newGptmeProviderFactory(def) case AgentKimi: return newKimiProviderFactory(def) + case AgentKilo: + return newKiloProviderFactory(def) + case AgentMiMoCode: + return newMiMoCodeProviderFactory(def) case AgentOpenHands: return newOpenHandsProviderFactory(def) + case AgentOpenCode: + return newOpenCodeProviderFactory(def) case AgentOpenClaw: return newOpenClawProviderFactory(def) case AgentOMP, AgentPi: diff --git a/internal/parser/provider_migration.go b/internal/parser/provider_migration.go index 2a8ba596c..90e33269c 100644 --- a/internal/parser/provider_migration.go +++ b/internal/parser/provider_migration.go @@ -22,9 +22,9 @@ var providerMigrationModes = map[AgentType]ProviderMigrationMode{ AgentCodex: ProviderMigrationLegacyOnly, AgentCopilot: ProviderMigrationLegacyOnly, AgentGemini: ProviderMigrationLegacyOnly, - AgentMiMoCode: ProviderMigrationLegacyOnly, - AgentOpenCode: ProviderMigrationLegacyOnly, - AgentKilo: ProviderMigrationLegacyOnly, + AgentMiMoCode: ProviderMigrationProviderAuthoritative, + AgentOpenCode: ProviderMigrationProviderAuthoritative, + AgentKilo: ProviderMigrationProviderAuthoritative, AgentOpenHands: ProviderMigrationProviderAuthoritative, AgentCursor: ProviderMigrationProviderAuthoritative, AgentIflow: ProviderMigrationProviderAuthoritative, diff --git a/internal/parser/provider_shim_scan_test.go b/internal/parser/provider_shim_scan_test.go index f4d3cb8e8..8e4aa92c4 100644 --- a/internal/parser/provider_shim_scan_test.go +++ b/internal/parser/provider_shim_scan_test.go @@ -55,7 +55,6 @@ var pendingShimProviderFiles = map[string]bool{ "gemini_provider.go": true, "kiro_ide_provider.go": true, "kiro_provider.go": true, - "opencode_provider.go": true, "positron_provider.go": true, "shelley_provider.go": true, "visualstudio_copilot_provider.go": true, diff --git a/internal/parser/types.go b/internal/parser/types.go index 238674cb3..3d261b97d 100644 --- a/internal/parser/types.go +++ b/internal/parser/types.go @@ -166,8 +166,6 @@ var Registry = []AgentDef{ "storage/part", }, FileBased: true, - DiscoverFunc: DiscoverMiMoCodeSessions, - FindSourceFunc: FindMiMoCodeSourceFile, WatchRootsFunc: ResolveMiMoCodeWatchRoots, }, { @@ -183,8 +181,6 @@ var Registry = []AgentDef{ "storage/part", }, FileBased: true, - DiscoverFunc: DiscoverOpenCodeSessions, - FindSourceFunc: FindOpenCodeSourceFile, WatchRootsFunc: ResolveOpenCodeWatchRoots, }, { @@ -200,8 +196,6 @@ var Registry = []AgentDef{ "storage/part", }, FileBased: true, - DiscoverFunc: DiscoverKiloSessions, - FindSourceFunc: FindKiloSourceFile, WatchRootsFunc: ResolveKiloWatchRoots, }, { diff --git a/internal/parser/types_test.go b/internal/parser/types_test.go index 6dff16319..0e834d177 100644 --- a/internal/parser/types_test.go +++ b/internal/parser/types_test.go @@ -504,8 +504,11 @@ func TestOpenCodeRegistryEntry(t *testing.T) { def, ok := AgentByType(AgentOpenCode) require.True(t, ok, "AgentOpenCode missing from Registry") require.True(t, def.FileBased, "OpenCode FileBased") - require.NotNil(t, def.DiscoverFunc, "OpenCode DiscoverFunc") - require.NotNil(t, def.FindSourceFunc, "OpenCode FindSourceFunc") + // OpenCode is a migrated, provider-authoritative agent: source + // discovery and lookup live on the concrete provider, not on legacy + // AgentDef hooks. + require.Nil(t, def.DiscoverFunc, "OpenCode DiscoverFunc") + require.Nil(t, def.FindSourceFunc, "OpenCode FindSourceFunc") want := []string{ "storage/session", "storage/message", @@ -541,8 +544,11 @@ func TestMiMoCodeRegistryEntry(t *testing.T) { def, ok := AgentByType(AgentMiMoCode) require.True(t, ok, "AgentMiMoCode missing from Registry") require.True(t, def.FileBased, "MiMoCode FileBased") - require.NotNil(t, def.DiscoverFunc, "MiMoCode DiscoverFunc") - require.NotNil(t, def.FindSourceFunc, "MiMoCode FindSourceFunc") + // MiMoCode is a migrated, provider-authoritative agent: source + // discovery and lookup live on the concrete provider, not on legacy + // AgentDef hooks. + require.Nil(t, def.DiscoverFunc, "MiMoCode DiscoverFunc") + require.Nil(t, def.FindSourceFunc, "MiMoCode FindSourceFunc") assert.Equal(t, "MIMOCODE_DIR", def.EnvVar) assert.Equal(t, "mimocode_dirs", def.ConfigKey) assert.Equal(t, []string{".local/share/mimocode"}, def.DefaultDirs) @@ -610,11 +616,11 @@ func TestResolveMiMoCodeSourcePrefersStorage(t *testing.T) { []byte(`{"id":"ses_test","directory":"/home/user/code/my-app"}`), 0o644)) - discovered := DiscoverMiMoCodeSessions(root) + discovered := discoverOpenCodeFormatSessions(mimoFmt, root) require.Len(t, discovered, 1) require.Equal(t, AgentMiMoCode, discovered[0].Agent) - require.Equal(t, path, FindMiMoCodeSourceFile(root, "ses_test")) + require.Equal(t, path, findOpenCodeFormatSourceFile(mimoFmt, root, "ses_test")) } func TestResolveOpenCodeSourceFallsBackToSQLiteOnBrokenStoragePath( @@ -661,7 +667,7 @@ func TestDiscoverOpenCodeSessions(t *testing.T) { data := []byte(`{"id":"ses_test","directory":"/home/user/code/my-app"}`) require.NoError(t, os.WriteFile(path, data, 0o644), "write session") - got := DiscoverOpenCodeSessions(root) + got := discoverOpenCodeFormatSessions(openCodeFmt, root) require.Len(t, got, 1, "len") require.Equal(t, path, got[0].Path, "Path") require.Equal(t, "my_app", got[0].Project, "Project") @@ -676,7 +682,7 @@ func TestDiscoverOpenCodeSessionsIgnoresNestedJSON(t *testing.T) { require.NoError(t, os.WriteFile(path, []byte(`{"id":"ses_test"}`), 0o644), "write session") require.NoError(t, os.WriteFile(filepath.Join(dir, "nested", "meta.json"), []byte(`{"id":"meta"}`), 0o644), "write nested json") - got := DiscoverOpenCodeSessions(root) + got := discoverOpenCodeFormatSessions(openCodeFmt, root) require.Len(t, got, 1, "len") require.Equal(t, path, got[0].Path, "Path") } @@ -688,7 +694,7 @@ func TestFindOpenCodeSourceFilePrefersStorage(t *testing.T) { require.NoError(t, os.WriteFile(path, []byte(`{"id":"ses_123"}`), 0o644), "write session") require.NoError(t, os.WriteFile(filepath.Join(root, "opencode.db"), []byte("x"), 0o644), "write db marker") - got := FindOpenCodeSourceFile(root, "ses_123") + got := findOpenCodeFormatSourceFile(openCodeFmt, root, "ses_123") require.Equal(t, path, got, "FindOpenCodeSourceFile()") } @@ -701,7 +707,7 @@ func TestFindOpenCodeSourceFileFallsBackToSQLiteInHybridRoot(t *testing.T) { dbPath := filepath.Join(root, "opencode.db") seedHybridSQLiteDB(t, dbPath, "ses_456") - got := FindOpenCodeSourceFile(root, "ses_456") + got := findOpenCodeFormatSourceFile(openCodeFmt, root, "ses_456") want := OpenCodeSQLiteVirtualPath(dbPath, "ses_456") require.Equal(t, want, got, "FindOpenCodeSourceFile()") } @@ -720,7 +726,7 @@ func TestFindOpenCodeSourceFileReturnsEmptyWhenSessionMissing(t *testing.T) { dbPath := filepath.Join(root, "opencode.db") seedHybridSQLiteDB(t, dbPath, "ses_unrelated") - got := FindOpenCodeSourceFile(root, "ses_missing") + got := findOpenCodeFormatSourceFile(openCodeFmt, root, "ses_missing") assert.Empty(t, got, "FindOpenCodeSourceFile()") } @@ -729,11 +735,11 @@ func TestFindOpenCodeSourceFilePureSQLiteOnlyForExistingSession(t *testing.T) { dbPath := filepath.Join(root, "opencode.db") seedHybridSQLiteDB(t, dbPath, "ses_present") - got := FindOpenCodeSourceFile(root, "ses_present") + got := findOpenCodeFormatSourceFile(openCodeFmt, root, "ses_present") assert.Equal(t, OpenCodeSQLiteVirtualPath(dbPath, "ses_present"), got, "FindOpenCodeSourceFile(present)") - got = FindOpenCodeSourceFile(root, "ses_absent") + got = findOpenCodeFormatSourceFile(openCodeFmt, root, "ses_absent") assert.Empty(t, got, "FindOpenCodeSourceFile(absent)") } @@ -897,17 +903,18 @@ func TestResolveOpenCodeWatchRootsMissingRoot(t *testing.T) { func TestParseOpenCodeSQLiteVirtualPath(t *testing.T) { dbPath := filepath.Join("/tmp", "opencode.db") virtual := OpenCodeSQLiteVirtualPath(dbPath, "ses_123") - gotDB, gotSessionID, ok := ParseOpenCodeSQLiteVirtualPath(virtual) + gotDB, gotSessionID, ok := parseOpenCodeFormatVirtualPath(openCodeFmt.dbName, virtual) require.True(t, ok, "expected virtual path to parse") assert.Equal(t, dbPath, gotDB, "db path") assert.Equal(t, "ses_123", gotSessionID, "session ID") hashDBPath := filepath.Join("/tmp", "opencode#dev", "opencode.db") hashVirtual := OpenCodeSQLiteVirtualPath(hashDBPath, "ses_456") - gotDB, gotSessionID, ok = ParseOpenCodeSQLiteVirtualPath(hashVirtual) + gotDB, gotSessionID, ok = parseOpenCodeFormatVirtualPath(openCodeFmt.dbName, hashVirtual) require.True(t, ok, "expected virtual path with # in db path to parse") assert.Equal(t, hashDBPath, gotDB, "db path with #") assert.Equal(t, "ses_456", gotSessionID, "session ID with #") - _, _, ok = ParseOpenCodeSQLiteVirtualPath( + _, _, ok = parseOpenCodeFormatVirtualPath( + openCodeFmt.dbName, "/tmp/project#dir/storage/session/global/ses_123.json", ) assert.False(t, ok, "expected real storage path with # to be rejected") diff --git a/internal/server/resume.go b/internal/server/resume.go index 3d887a1e7..86c4131df 100644 --- a/internal/server/resume.go +++ b/internal/server/resume.go @@ -382,10 +382,10 @@ func resolveResumeDir(session *db.Session) string { } func isVirtualSessionPath(path string) bool { - if _, _, ok := parser.ParseKiroSQLiteVirtualPath(path); ok { + if _, _, ok := parser.ParseVirtualSourcePathForBase(path, "data.sqlite3"); ok { return true } - if _, _, ok := parser.ParseOpenCodeSQLiteVirtualPath(path); ok { + if _, _, ok := parser.ParseVirtualSourcePathForBase(path, "opencode.db"); ok { return true } return false diff --git a/internal/sync/engine.go b/internal/sync/engine.go index e04fb7d10..a6831962e 100644 --- a/internal/sync/engine.go +++ b/internal/sync/engine.go @@ -2,7 +2,6 @@ package sync import ( "context" - "encoding/json" "errors" "fmt" "log" @@ -886,27 +885,12 @@ func isUnder(dir, path string) (string, bool) { } // classifyContainerPath runs the container- and SQLite-style classifiers that -// resolve a path whether or not it currently exists on disk (OpenCode-format -// stores, Kiro, Zed, and Shelley). Split out of classifyOnePath to keep -// that function within NilAway's per-function CFG-block limit. +// resolve a path whether or not it currently exists on disk (Kiro, Zed, +// Shelley, and Vibe). Split out of classifyOnePath to keep that function +// within NilAway's per-function CFG-block limit. func (e *Engine) classifyContainerPath( path string, pathExists bool, ) (parser.DiscoveredFile, bool) { - if df, ok := e.classifyOpenCodeFormatPath( - parser.AgentOpenCode, path, pathExists, - ); ok { - return df, true - } - if df, ok := e.classifyOpenCodeFormatPath( - parser.AgentKilo, path, pathExists, - ); ok { - return df, true - } - if df, ok := e.classifyOpenCodeFormatPath( - parser.AgentMiMoCode, path, pathExists, - ); ok { - return df, true - } if df, ok := e.classifyKiroSQLitePath(path); ok { return df, true } @@ -1382,138 +1366,6 @@ func (e *Engine) classifyAntigravityCLIBrainPath( return nil } -// classifyOpenCodeFormatPath classifies a path under an OpenCode-format -// root (OpenCode, its Kilo fork, or MiMoCode), which share an on-disk -// layout and differ only in the SQLite filename, the storage/ -// holding session JSON, and the agent label. MiMoCode stores sessions -// under storage/session_diff; the session subdir is taken from the -// resolved source rather than assumed. -// -// /storage///.json -// /storage/message//.json -// /storage/part//.json -func (e *Engine) classifyOpenCodeFormatPath( - agent parser.AgentType, path string, pathExists bool, -) (parser.DiscoveredFile, bool) { - sep := string(filepath.Separator) - dbName := openCodeFormatDBName(agent) - - for _, dir := range e.agentDirs[agent] { - if dir == "" { - continue - } - rel, ok := isUnder(dir, path) - if !ok { - continue - } - base := filepath.Base(rel) - if rel == dbName || strings.HasPrefix(base, dbName+"-") { - dbPath := filepath.Join(dir, dbName) - if info, err := os.Stat(dbPath); err == nil && - !info.IsDir() { - return parser.DiscoveredFile{ - Path: dbPath, - Agent: agent, - }, true - } - continue - } - src := resolveOpenCodeFormatSource(agent, dir) - if src.Mode != parser.OpenCodeSourceStorage { - continue - } - sessionSubdir := filepath.Base(src.SessionRoot) - parts := strings.Split(rel, sep) - switch { - case pathExists && - len(parts) == 4 && - parts[0] == "storage" && - parts[1] == sessionSubdir && - strings.HasSuffix(parts[3], ".json"): - return parser.DiscoveredFile{ - Path: path, - Agent: agent, - }, true - case len(parts) == 4 && - parts[0] == "storage" && - parts[1] == "message" && - strings.HasSuffix(parts[3], ".json"): - sessionPath := findOpenCodeFormatSourceFile( - agent, dir, parts[2], - ) - if sessionPath == "" { - continue - } - return parser.DiscoveredFile{ - Path: sessionPath, - Agent: agent, - }, true - case len(parts) == 4 && - parts[0] == "storage" && - parts[1] == "part" && - strings.HasSuffix(parts[3], ".json"): - sessionID := "" - if pathExists { - sessionID = readOpenCodeStorageSessionID(path) - } - if sessionID == "" { - sessionID = - findOpenCodeStorageSessionIDByMessageID( - dir, parts[2], - ) - } - if sessionID == "" { - continue - } - sessionPath := findOpenCodeFormatSourceFile( - agent, dir, sessionID, - ) - if sessionPath == "" { - continue - } - return parser.DiscoveredFile{ - Path: sessionPath, - Agent: agent, - }, true - case !pathExists && - len(parts) == 3 && - parts[0] == "storage" && - parts[1] == "message": - sessionPath := findOpenCodeFormatSourceFile( - agent, dir, parts[2], - ) - if sessionPath == "" { - continue - } - return parser.DiscoveredFile{ - Path: sessionPath, - Agent: agent, - }, true - case !pathExists && - len(parts) == 3 && - parts[0] == "storage" && - parts[1] == "part": - sessionID := findOpenCodeStorageSessionIDByMessageID( - dir, parts[2], - ) - if sessionID == "" { - continue - } - sessionPath := findOpenCodeFormatSourceFile( - agent, dir, sessionID, - ) - if sessionPath == "" { - continue - } - return parser.DiscoveredFile{ - Path: sessionPath, - Agent: agent, - }, true - } - } - return parser.DiscoveredFile{}, false -} - func (e *Engine) classifyKiroSQLitePath( path string, ) (parser.DiscoveredFile, bool) { @@ -1851,8 +1703,12 @@ func (e *Engine) resyncAllLocked( // restores those rows immediately after the sync pass. // A few permanent parse failures are tolerated since those // files were broken in the old DB too. - emptyDiscovery := stats.filesDiscovered == 0 && - stats.filesOK == 0 && + // OpenCode-format storage is a self-preserving container store that + // now flows through file discovery, so it is excluded here just as it + // is subtracted from oldFileSessions above. Otherwise its discovery + // would mask the disappearance of plain file-backed sessions whose + // directories went empty. + emptyDiscovery := stats.nonContainerDiscovered == 0 && oldFileSessions > 0 preservedOnly := stats.Synced == 0 && stats.TotalSessions > 0 && @@ -2501,6 +2357,13 @@ func (e *Engine) syncAllLocked( SessionsTotal: progressTotal, }) + nonContainerDiscovered := 0 + for _, f := range all { + if !isOpenCodeFormatStorageAgent(f.Agent) { + nonContainerDiscovered++ + } + } + tWorkers := time.Now() results := e.startWorkers(ctx, all) stats := e.collectAndBatch( @@ -2509,6 +2372,7 @@ func (e *Engine) syncAllLocked( for range providerFailures { stats.RecordFailed() } + stats.nonContainerDiscovered = nonContainerDiscovered if verbose { log.Printf( "file sync: %d synced, %d skipped in %s", @@ -2608,37 +2472,10 @@ func (e *Engine) syncAllLocked( return stats } - // Sync OpenCode-format sessions (DB-backed, not file-based). - // Uses full replace because these messages can change in place - // (streaming updates, tool result pairing). Kilo is a fork of - // OpenCode and shares the same SQLite-backed sync. - if scope.includesAny(e.agentDirs[parser.AgentOpenCode]) { - if e.syncOpenCodeFormatAgent( - ctx, parser.AgentOpenCode, "opencode", - writeMode, verbose, scope, &stats, advanceDBProgress, - ) { - stats.Aborted = true - return stats - } - } - if scope.includesAny(e.agentDirs[parser.AgentKilo]) { - if e.syncOpenCodeFormatAgent( - ctx, parser.AgentKilo, "kilo", - writeMode, verbose, scope, &stats, advanceDBProgress, - ) { - stats.Aborted = true - return stats - } - } - if scope.includesAny(e.agentDirs[parser.AgentMiMoCode]) { - if e.syncOpenCodeFormatAgent( - ctx, parser.AgentMiMoCode, "mimocode", - writeMode, verbose, scope, &stats, advanceDBProgress, - ) { - stats.Aborted = true - return stats - } - } + // OpenCode-format sessions (OpenCode and its Kilo and MiMoCode + // forks) are provider-authoritative: discovery and parsing flow + // through the provider facade in the file-sync phase above, so no + // dedicated DB-backed sync pass is needed here. // Sync Warp sessions (DB-backed, not file-based). tWarp := time.Now() @@ -3346,61 +3183,6 @@ func shelleyDBCompositeMtime(dbPath string) (int64, error) { return maxMtime, nil } -// openCodeFormatPendingSessionIDs returns the SQLite session IDs that -// need re-parsing for an OpenCode-format agent. Uses per-session -// time_updated to detect changes and skips IDs shadowed by a canonical -// storage transcript. -func (e *Engine) openCodeFormatPendingSessionIDs( - agent parser.AgentType, dir string, -) []string { - dbPath := filepath.Join(dir, openCodeFormatDBName(agent)) - if info, err := os.Stat(dbPath); err != nil || info.IsDir() { - return nil - } - - metas, err := listOpenCodeFormatSessionMeta(agent, dbPath) - if err != nil { - log.Printf("sync %s: %v", agent, err) - return nil - } - storageIDs := openCodeFormatStorageSessionIDs(agent, dir) - var changed []string - for _, m := range metas { - if _, ok := storageIDs[m.SessionID]; ok { - continue - } - _, storedMtime, ok := e.db.GetFileInfoByPath(m.VirtualPath) - if ok && storedMtime == m.FileMtime && - e.db.GetDataVersionByPath(m.VirtualPath) >= db.CurrentDataVersion() { - continue - } - changed = append(changed, m.SessionID) - } - return changed -} - -func (e *Engine) countOneOpenCodeFormatSessions( - agent parser.AgentType, dir string, -) int { - dbPath := filepath.Join(dir, openCodeFormatDBName(agent)) - if info, err := os.Stat(dbPath); err != nil || info.IsDir() { - return 0 - } - metas, err := listOpenCodeFormatSessionMeta(agent, dbPath) - if err != nil { - log.Printf("sync %s: %v", agent, err) - return 0 - } - storageIDs := openCodeFormatStorageSessionIDs(agent, dir) - count := 0 - for _, m := range metas { - if _, ok := storageIDs[m.SessionID]; !ok { - count++ - } - } - return count -} - func (e *Engine) countDBBackedProgressTotal( agent parser.AgentType, scope *rootSyncScope, ) int { @@ -3412,8 +3194,6 @@ func (e *Engine) countDBBackedProgressTotal( switch agent { case parser.AgentKiro: total += e.countOneKiroSQLiteSessions(dir) - case parser.AgentOpenCode, parser.AgentKilo, parser.AgentMiMoCode: - total += e.countOneOpenCodeFormatSessions(agent, dir) case parser.AgentWarp: total += e.countOneWarpSessions(dir) case parser.AgentForge: @@ -3434,9 +3214,6 @@ func (e *Engine) countDBBackedSessions( total := 0 for _, agent := range []parser.AgentType{ parser.AgentKiro, - parser.AgentOpenCode, - parser.AgentKilo, - parser.AgentMiMoCode, parser.AgentWarp, parser.AgentForge, parser.AgentPiebald, @@ -3606,125 +3383,6 @@ func (e *Engine) syncOneKiroSQLite( return pending } -// syncOpenCodeFormat syncs sessions from an OpenCode-format agent's -// SQLite database (OpenCode or its Kilo fork). Uses per-session -// time_updated to detect changes, so only modified sessions are fully -// parsed. Returns pending writes. -func (e *Engine) syncOpenCodeFormat( - ctx context.Context, agent parser.AgentType, scope *rootSyncScope, -) []pendingWrite { - var allPending []pendingWrite - for _, dir := range e.agentDirs[agent] { - if ctx.Err() != nil { - break - } - if dir == "" || !scope.includes(dir) { - continue - } - allPending = append( - allPending, e.syncOneOpenCodeFormat(ctx, agent, dir)..., - ) - } - return allPending -} - -// syncOneOpenCodeFormat handles a single OpenCode-format directory. -func (e *Engine) syncOneOpenCodeFormat( - ctx context.Context, agent parser.AgentType, dir string, -) []pendingWrite { - dbPath := filepath.Join(dir, openCodeFormatDBName(agent)) - changed := e.openCodeFormatPendingSessionIDs(agent, dir) - if len(changed) == 0 { - return nil - } - - var pending []pendingWrite - for _, sid := range changed { - if ctx.Err() != nil { - break - } - sess, msgs, err := parseOpenCodeFormatSession( - agent, dbPath, sid, e.machine, - ) - if err != nil { - log.Printf( - "%s session %s: %v", agent, sid, err, - ) - continue - } - if sess == nil { - continue - } - pending = append(pending, pendingWrite{ - sess: *sess, - msgs: msgs, - }) - } - - return pending -} - -// syncOpenCodeFormatAgent collects, writes, and records pending -// sessions for one OpenCode-format agent. It returns true when the -// context was cancelled so the caller can mark the sync aborted. -func (e *Engine) syncOpenCodeFormatAgent( - ctx context.Context, agent parser.AgentType, label string, - writeMode syncWriteMode, verbose bool, scope *rootSyncScope, - stats *SyncStats, - advanceDBProgress func(total int, pending []pendingWrite), -) bool { - start := time.Now() - pending := e.syncOpenCodeFormat(ctx, agent, scope) - if len(pending) > 0 { - stats.TotalSessions += len(pending) - tWrite := time.Now() - var written int - if writeMode == syncWriteBulk { - var failedWrites int - written, _, failedWrites = e.writeBatch( - pending, writeMode, true, - ) - for range failedWrites { - stats.RecordFailed() - } - } else { - resolveWorktreeProject := e.loadWorktreeProjectResolver() - for _, pw := range pending { - if ctx.Err() != nil { - break - } - switch err := e.writeSessionFullWithResolver( - pw, resolveWorktreeProject, - ); { - case err == nil: - written++ - case isIntentionalSessionSkip(err), - errors.Is(err, errSessionPreserved): - // Intentional skip, not a failure. - default: - stats.RecordFailed() - } - } - } - stats.RecordSynced(written) - if verbose { - log.Printf( - "%s write: %d sessions in %s", - label, len(pending), - time.Since(tWrite).Round(time.Millisecond), - ) - } - } - if verbose { - log.Printf( - "%s sync: %s", - label, time.Since(start).Round(time.Millisecond), - ) - } - advanceDBProgress(e.countDBBackedProgressTotal(agent, scope), pending) - return ctx.Err() != nil -} - // startWorkers fans out file processing across a worker pool // and returns a channel of results. When ctx is cancelled, // workers skip remaining jobs with a context error instead @@ -4033,10 +3691,6 @@ func (e *Engine) processFile( statPath := file.Path if dbPath, _, ok := parser.ParseKiroSQLiteVirtualPath(file.Path); ok { statPath = dbPath - } else if dbPath, _, ok := parser.ParseKiloSQLiteVirtualPath(file.Path); ok { - statPath = dbPath - } else if dbPath, _, ok := parser.ParseMiMoCodeSQLiteVirtualPath(file.Path); ok { - statPath = dbPath } else if dbPath, _, ok := parser.ParseZedSQLiteVirtualPath(file.Path); ok { statPath = dbPath } else if dbPath, _, ok := parser.ParseShelleyVirtualPath(file.Path); ok { @@ -4111,8 +3765,6 @@ func (e *Engine) processFile( res = e.processReasonix(file, info) case parser.AgentGemini: res = e.processGemini(file, info) - case parser.AgentOpenCode, parser.AgentKilo, parser.AgentMiMoCode: - res = e.processOpenCodeFormat(file.Agent, file, info) case parser.AgentVSCodeCopilot: res = e.processVSCodeCopilot(file, info) case parser.AgentVSCopilot: @@ -5524,139 +5176,6 @@ func (e *Engine) processCodex( } } -func (e *Engine) processOpenCodeFormat( - agent parser.AgentType, - file parser.DiscoveredFile, info os.FileInfo, -) processResult { - if dbPath, sessionID, ok := parseOpenCodeFormatSQLiteVirtualPath( - agent, file.Path, - ); ok { - sess, msgs, err := parseOpenCodeFormatSession( - agent, dbPath, sessionID, e.machine, - ) - if err != nil { - return processResult{err: err} - } - if sess == nil { - return processResult{} - } - return processResult{ - results: []parser.ParseResult{ - {Session: *sess, Messages: msgs}, - }, - } - } - if filepath.Base(file.Path) == openCodeFormatDBName(agent) { - metas, err := listOpenCodeFormatSessionMeta(agent, file.Path) - if err != nil { - return processResult{err: err} - } - storageIDs := openCodeFormatStorageSessionIDs( - agent, filepath.Dir(file.Path), - ) - var results []parser.ParseResult - var sessionErrs []sessionParseError - for _, meta := range metas { - if _, ok := storageIDs[meta.SessionID]; ok { - continue - } - _, storedMtime, ok := e.db.GetFileInfoByPath(meta.VirtualPath) - // parse-diff: !e.forceParse disables the stored-state skip. - if !e.forceParse && ok && storedMtime == meta.FileMtime && - e.db.GetDataVersionByPath(meta.VirtualPath) >= - db.CurrentDataVersion() { - continue - } - sess, msgs, err := parseOpenCodeFormatSession( - agent, file.Path, meta.SessionID, e.machine, - ) - if err != nil { - if e.forceParse { - sessionErrs = append(sessionErrs, sessionParseError{ - sessionID: meta.SessionID, - virtualPath: meta.VirtualPath, - err: err, - }) - } else { - log.Printf( - "%s sqlite watch session %s: %v", - agent, meta.SessionID, err, - ) - } - continue - } - if sess == nil { - continue - } - results = append(results, parser.ParseResult{ - Session: *sess, - Messages: msgs, - }) - } - return processResult{ - results: results, - sessionErrs: sessionErrs, - forceReplace: true, - } - } - if e.shouldSkipOpenCodeFormatByPath(agent, file.Path) { - return processResult{skip: true} - } - - sess, msgs, err := parseOpenCodeFormatFile( - agent, file.Path, e.machine, - ) - if err != nil { - return processResult{err: err} - } - if sess == nil { - return processResult{} - } - - hash, err := ComputeFileHash(file.Path) - if err == nil && sess.File.Hash == "" { - sess.File.Hash = hash - } - - sess.File.Inode, sess.File.Device = getFileIdentity(info) - - return processResult{ - results: []parser.ParseResult{ - {Session: *sess, Messages: msgs}, - }, - } -} - -func (e *Engine) shouldSkipOpenCodeFormatByPath( - agent parser.AgentType, path string, -) bool { - if e.forceParse { // parse-diff: always re-parse - return false - } - lookupPath := path - if e.pathRewriter != nil { - lookupPath = e.pathRewriter(path) - } - - _, storedMtime, ok := e.db.GetFileInfoByPath(lookupPath) - if !ok { - return false - } - - sourceMtime, err := openCodeFormatSourceMtime(agent, path) - if err != nil || sourceMtime == 0 { - return false - } - if storedMtime != sourceMtime { - return false - } - if e.db.GetDataVersionByPath(lookupPath) < - db.CurrentDataVersion() { - return false - } - return true -} - func (e *Engine) processCopilot( file parser.DiscoveredFile, info os.FileInfo, ) processResult { @@ -7963,88 +7482,12 @@ func isOpenCodeFormatSQLiteVirtualPath( if !isOpenCodeFormatStorageAgent(agent) { return false } - _, _, ok := parseOpenCodeFormatSQLiteVirtualPath(agent, path) + _, _, ok := parser.ParseVirtualSourcePathForBase( + path, openCodeFormatDBName(agent), + ) return ok } -func parseOpenCodeFormatSQLiteVirtualPath( - agent parser.AgentType, path string, -) (dbPath, sessionID string, ok bool) { - switch agent { - case parser.AgentKilo: - return parser.ParseKiloSQLiteVirtualPath(path) - case parser.AgentMiMoCode: - return parser.ParseMiMoCodeSQLiteVirtualPath(path) - default: - return parser.ParseOpenCodeSQLiteVirtualPath(path) - } -} - -func listOpenCodeFormatSessionMeta( - agent parser.AgentType, dbPath string, -) ([]parser.OpenCodeSessionMeta, error) { - switch agent { - case parser.AgentKilo: - return parser.ListKiloSessionMeta(dbPath) - case parser.AgentMiMoCode: - return parser.ListMiMoCodeSessionMeta(dbPath) - default: - return parser.ListOpenCodeSessionMeta(dbPath) - } -} - -func openCodeFormatStorageSessionIDs( - agent parser.AgentType, dir string, -) map[string]struct{} { - switch agent { - case parser.AgentKilo: - return parser.KiloStorageSessionIDs(dir) - case parser.AgentMiMoCode: - return parser.MiMoCodeStorageSessionIDs(dir) - default: - return parser.OpenCodeStorageSessionIDs(dir) - } -} - -func findOpenCodeFormatSourceFile( - agent parser.AgentType, dir, sessionID string, -) string { - switch agent { - case parser.AgentKilo: - return parser.FindKiloSourceFile(dir, sessionID) - case parser.AgentMiMoCode: - return parser.FindMiMoCodeSourceFile(dir, sessionID) - default: - return parser.FindOpenCodeSourceFile(dir, sessionID) - } -} - -func parseOpenCodeFormatSession( - agent parser.AgentType, dbPath, sessionID, machine string, -) (*parser.ParsedSession, []parser.ParsedMessage, error) { - switch agent { - case parser.AgentKilo: - return parser.ParseKiloSession(dbPath, sessionID, machine) - case parser.AgentMiMoCode: - return parser.ParseMiMoCodeSession(dbPath, sessionID, machine) - default: - return parser.ParseOpenCodeSession(dbPath, sessionID, machine) - } -} - -func parseOpenCodeFormatFile( - agent parser.AgentType, path, machine string, -) (*parser.ParsedSession, []parser.ParsedMessage, error) { - switch agent { - case parser.AgentKilo: - return parser.ParseKiloFile(path, machine) - case parser.AgentMiMoCode: - return parser.ParseMiMoCodeFile(path, machine) - default: - return parser.ParseOpenCodeFile(path, machine) - } -} - func derefString(s *string) string { if s == nil { return "" @@ -8727,14 +8170,10 @@ func (e *Engine) SyncSingleSessionContext( case parser.AgentPiebald: return e.syncSinglePiebald(sessionID) default: - err = e.syncSingleOpenCodeFormat( - sessionID, parser.AgentOpenCode, + return fmt.Errorf( + "cannot resync non-file-based session %s for agent %s", + sessionID, def.Type, ) - if errors.Is(err, errSessionPreserved) { - preserved = true - return nil - } - return err } } @@ -8753,15 +8192,10 @@ func (e *Engine) SyncSingleSessionContext( "source file not found for %s", sessionID, ) } - if isOpenCodeFormatStorageAgent(def.Type) && - isOpenCodeFormatSQLiteVirtualPath(def.Type, path) { - err = e.syncSingleOpenCodeFormat(sessionID, def.Type) - if errors.Is(err, errSessionPreserved) { - preserved = true - return nil - } - return err - } + // OpenCode-format agents (OpenCode, Kilo, MiMoCode) are + // provider-authoritative: their SQLite virtual paths and storage + // sessions resync through the generic processFile path below, which + // routes to the provider facade. if def.Type == parser.AgentKiro && isKiroSQLiteVirtualPath(path) { err = e.syncSingleKiroSQLite(sessionID) @@ -8983,61 +8417,6 @@ func (e *Engine) applyWorktreeMappingToSingleSession( return nil } -// syncSingleOpenCodeFormat re-syncs a single SQLite-backed session for -// an OpenCode-format agent (OpenCode or its Kilo fork). -func (e *Engine) syncSingleOpenCodeFormat( - sessionID string, agent parser.AgentType, -) error { - if !isOpenCodeFormatStorageAgent(agent) { - return fmt.Errorf("unknown OpenCode-format agent: %s", agent) - } - rawID := strings.TrimPrefix(sessionID, string(agent)+":") - dbName := openCodeFormatDBName(agent) - - var lastErr error - for _, dir := range e.agentDirs[agent] { - if dir == "" { - continue - } - dbPath := filepath.Join(dir, dbName) - if info, err := os.Stat(dbPath); err != nil || - info.IsDir() { - continue - } - sess, msgs, err := parseOpenCodeFormatSession( - agent, dbPath, rawID, e.machine, - ) - if err != nil { - lastErr = err - continue - } - if sess == nil { - continue - } - if err := e.writeSessionFull( - pendingWrite{sess: *sess, msgs: msgs}, - ); err != nil && - !isIntentionalSessionSkip(err) && - !errors.Is(err, errSessionPreserved) { - return fmt.Errorf("write session %s: %w", - sess.ID, err) - } else if errors.Is(err, errSessionPreserved) { - return err - } - return nil - } - - if len(e.agentDirs[agent]) == 0 { - return fmt.Errorf("%s dir not configured", agent) - } - if lastErr != nil { - return fmt.Errorf( - "%s session %s: %w", agent, sessionID, lastErr, - ) - } - return fmt.Errorf("%s session %s not found", agent, sessionID) -} - func (e *Engine) syncSingleKiroSQLite( sessionID string, ) error { @@ -9147,45 +8526,6 @@ func isKiroSQLiteVirtualPath(path string) bool { return ok } -func readOpenCodeStorageSessionID(path string) string { - raw, err := os.ReadFile(path) - if err != nil { - return "" - } - var data struct { - SessionID string `json:"sessionID"` - } - if err := json.Unmarshal(raw, &data); err != nil { - return "" - } - return data.SessionID -} - -func findOpenCodeStorageSessionIDByMessageID( - openCodeDir, messageID string, -) string { - messageRoot := filepath.Join( - openCodeDir, "storage", "message", - ) - entries, err := os.ReadDir(messageRoot) - if err != nil { - return "" - } - for _, entry := range entries { - if !entry.IsDir() { - continue - } - path := filepath.Join( - messageRoot, entry.Name(), messageID+".json", - ) - if info, err := os.Stat(path); err == nil && - !info.IsDir() { - return entry.Name() - } - } - return "" -} - func (e *Engine) warpPendingSessionIDs(dir string) []string { dbPath := parser.FindWarpDBPath(dir) if dbPath == "" { diff --git a/internal/sync/engine_integration_test.go b/internal/sync/engine_integration_test.go index 49a237b09..e791f2839 100644 --- a/internal/sync/engine_integration_test.go +++ b/internal/sync/engine_integration_test.go @@ -3840,10 +3840,24 @@ func TestSyncPathsOpenCodeStorageChildUpdateAdvancesSessionMtime( time.Unix(0, sessionMtime), ) require.NoError(t, err, "restore session mtime") - _, parsedMsgs, parseErr := parser.ParseOpenCodeFile( - sessionPath, "local", + ocProvider, ok := parser.NewProvider(parser.AgentOpenCode, parser.ProviderConfig{ + Roots: []string{env.opencodeDir}, + Machine: "local", + }) + require.True(t, ok, "opencode provider available") + ocSource, found, parseErr := ocProvider.FindSource( + context.Background(), + parser.FindSourceRequest{FullSessionID: "opencode:oc-storage-mtime"}, ) - require.NoError(t, parseErr, "ParseOpenCodeFile after rewrite") + require.NoError(t, parseErr, "find opencode source after rewrite") + require.True(t, found, "opencode source found after rewrite") + ocOutcome, parseErr := ocProvider.Parse(context.Background(), parser.ParseRequest{ + Source: ocSource, + Machine: "local", + }) + require.NoError(t, parseErr, "parse opencode source after rewrite") + require.Len(t, ocOutcome.Results, 1, "parsed results after rewrite") + parsedMsgs := ocOutcome.Results[0].Result.Messages require.Len(t, parsedMsgs, 1, "parsed messages after rewrite = %#v, want updated reply", parsedMsgs) require.Equal(t, "updated reply", parsedMsgs[0].Content, "parsed messages after rewrite = %#v, want updated reply", parsedMsgs) @@ -4110,8 +4124,8 @@ func TestOpenCodeHybridRootSyncsSQLiteSessions(t *testing.T) { // multi-root shadowing case: an early hybrid root with an // opencode.db that lacks the requested session must not shadow a // later pure-storage root that contains it. Without the -// session-existence gate in FindOpenCodeSourceFile, the engine -// would return a virtual SQLite path pointing at the wrong DB. +// session-existence gate in the OpenCode-format source lookup, the +// engine would return a virtual SQLite path pointing at the wrong DB. func TestFindSourceFileSkipsHybridRootMissingSession(t *testing.T) { hybridRoot := t.TempDir() storageRoot := t.TempDir() @@ -4547,7 +4561,12 @@ func TestSyncAllSinceOpenCodeStoragePicksUpUsagePartUpdate(t *testing.T) { assert.Equal(t, []string{"Gemini 3.5 Flash (High)"}, daily.Daily[0].ModelsUsed) } -func TestSyncAllOpenCodeStorageSkipsUnchangedSessions(t *testing.T) { +// TestSyncAllOpenCodeStorageReparsesUnchangedSessionsIdempotently covers a +// re-sync of an unchanged OpenCode storage session. OpenCode is +// provider-authoritative, so a full SyncAll re-parses the source through +// the provider facade rather than taking the legacy DB-mtime skip; the +// re-parse must be idempotent and keep the same content. +func TestSyncAllOpenCodeStorageReparsesUnchangedSessionsIdempotently(t *testing.T) { env := setupTestEnv(t) oc := createOpenCodeStorageFixture(t, env.opencodeDir) @@ -4572,8 +4591,11 @@ func TestSyncAllOpenCodeStorageSkipsUnchangedSessions(t *testing.T) { }) stats := env.engine.SyncAll(context.Background(), nil) - require.Equal(t, 1, stats.Skipped, "SyncAll stats = %+v, want 1 skipped and 0 synced", stats) - require.Equal(t, 0, stats.Synced, "SyncAll stats = %+v, want 1 skipped and 0 synced", stats) + require.Equal(t, 1, stats.TotalSessions, "SyncAll stats = %+v", stats) + require.Equal(t, 0, stats.Failed, "SyncAll stats = %+v", stats) + assertMessageContent( + t, env.db, "opencode:oc-skip-unchanged", "stable reply", + ) } func TestSyncAllOpenCodeStorageMissingMessagePreservesArchive(t *testing.T) { diff --git a/internal/sync/engine_test.go b/internal/sync/engine_test.go index 67f87fcc4..c505c3796 100644 --- a/internal/sync/engine_test.go +++ b/internal/sync/engine_test.go @@ -2855,6 +2855,10 @@ func TestEngine_ClassifyPathsOpenCodeRemovedMessageDir( assert.Equal(t, sessionPath, files[0].Path) } +// TestEngine_ClassifyPathsOpenCodeSQLiteWALFile covers a WAL-file change on +// a pure-SQLite OpenCode root. OpenCode is provider-authoritative, so the +// provider facade classifies the change into the per-session SQLite virtual +// paths it would re-parse rather than the raw opencode.db path. func TestEngine_ClassifyPathsOpenCodeSQLiteWALFile( t *testing.T, ) { @@ -2868,13 +2872,61 @@ func TestEngine_ClassifyPathsOpenCodeSQLiteWALFile( }) dbPath := filepath.Join(opencodeDir, "opencode.db") - require.NoError(t, os.WriteFile(dbPath, []byte("db"), 0o644), "WriteFile(%q)", dbPath) + seedOpenCodeSQLiteSession(t, dbPath, "ses_wal") walPath := filepath.Join(opencodeDir, "opencode.db-wal") require.NoError(t, os.WriteFile(walPath, []byte("wal"), 0o644), "WriteFile(%q)", walPath) files := engine.classifyPaths([]string{walPath}) require.Len(t, files, 1) - assert.Equal(t, dbPath, files[0].Path) + assert.Equal(t, + parser.OpenCodeSQLiteVirtualPath(dbPath, "ses_wal"), + files[0].Path, + ) + assert.Equal(t, parser.AgentOpenCode, files[0].Agent) +} + +// seedOpenCodeSQLiteSession creates a minimal OpenCode-shaped SQLite database +// with a single session row so changed-path classification can enumerate it. +func seedOpenCodeSQLiteSession(t *testing.T, dbPath, sessionID string) { + t.Helper() + d, err := sql.Open("sqlite3", dbPath) + require.NoError(t, err, "open opencode db") + t.Cleanup(func() { d.Close() }) + _, err = d.Exec(` + CREATE TABLE project (id TEXT PRIMARY KEY, worktree TEXT NOT NULL); + CREATE TABLE session ( + id TEXT PRIMARY KEY, + project_id TEXT NOT NULL, + parent_id TEXT, + title TEXT, + time_created INTEGER NOT NULL, + time_updated INTEGER NOT NULL + ); + CREATE TABLE message ( + id TEXT PRIMARY KEY, + session_id TEXT NOT NULL, + data TEXT NOT NULL, + time_created INTEGER NOT NULL + ); + CREATE TABLE part ( + id TEXT PRIMARY KEY, + session_id TEXT NOT NULL, + message_id TEXT NOT NULL, + data TEXT NOT NULL, + time_created INTEGER NOT NULL + ); + `) + require.NoError(t, err, "create opencode schema") + _, err = d.Exec( + "INSERT INTO project (id, worktree) VALUES ('prj_1', '/home/user/code/app')", + ) + require.NoError(t, err, "insert project") + _, err = d.Exec( + `INSERT INTO session (id, project_id, time_created, time_updated) + VALUES (?, 'prj_1', 1, 2)`, + sessionID, + ) + require.NoError(t, err, "insert session") } func TestEngine_ClassifyPathsOpenCodeRemovedMessageFile( @@ -2912,6 +2964,58 @@ func TestEngine_ClassifyPathsOpenCodeRemovedMessageFile( assert.Equal(t, sessionPath, files[0].Path) } +// TestEngine_ClassifyPathsOpenCodeFamilyRemovedSessionFile covers a removed +// storage session file for the provider-authoritative OpenCode-format agents. +// A delete event yields no reparse classification: there is no source to +// re-read, and the deletion is reconciled by the presence sweep rather than +// changed-path classification. +func TestEngine_ClassifyPathsOpenCodeFamilyRemovedSessionFile( + t *testing.T, +) { + for _, tc := range []struct { + name string + agent parser.AgentType + sessionSubdir string + }{ + {name: "opencode", agent: parser.AgentOpenCode, sessionSubdir: "session"}, + {name: "kilo", agent: parser.AgentKilo, sessionSubdir: "session"}, + {name: "mimocode", agent: parser.AgentMiMoCode, sessionSubdir: "session_diff"}, + } { + t.Run(tc.name, func(t *testing.T) { + db := openTestDB(t) + root := t.TempDir() + engine := NewEngine(db, EngineConfig{ + AgentDirs: map[parser.AgentType][]string{ + tc.agent: {root}, + }, + Machine: "local", + }) + + sessionPath := filepath.Join( + root, "storage", tc.sessionSubdir, "global", + "ses_removed.json", + ) + require.NoError( + t, os.MkdirAll(filepath.Dir(sessionPath), 0o755), + "MkdirAll(%q)", sessionPath, + ) + require.NoError( + t, + os.WriteFile( + sessionPath, + []byte(`{"id":"ses_removed","directory":"/tmp/proj","time":{"created":1,"updated":2}}`), + 0o644, + ), + "WriteFile(%q)", sessionPath, + ) + require.NoError(t, os.Remove(sessionPath), "Remove(%q)", sessionPath) + + files := engine.classifyPaths([]string{sessionPath}) + assert.Empty(t, files) + }) + } +} + func TestEngine_ClassifyPathsOpenCodeRemovedPartDir( t *testing.T, ) { diff --git a/internal/sync/parsediff.go b/internal/sync/parsediff.go index 672dba557..580a82c41 100644 --- a/internal/sync/parsediff.go +++ b/internal/sync/parsediff.go @@ -114,7 +114,7 @@ func (e *Engine) ParseDiff(ctx context.Context, opts ParseDiffOptions) (*ParseDi s := &storedSessions[i] storedByID[s.ID] = s if s.FilePath != nil && *s.FilePath != "" { - base := stripVirtualSourceSuffix(*s.FilePath) + base := parseDiffSourceKey(*s.FilePath) storedByPath[base] = append(storedByPath[base], s) } } @@ -340,6 +340,16 @@ func (e *Engine) parseDiffDatabaseSources( ) []parser.DiscoveredFile { var extra []parser.DiscoveredFile for _, def := range resolved { + // Provider-authoritative agents (no DiscoverFunc) already have + // their shared-SQLite sessions enumerated by + // parseDiffProviderSources, which applies the provider's + // storage-ID filter so a file-backed storage session is not also + // re-parsed from its stale db row. Synthesizing the raw db here + // would re-add those sessions through the legacy fan-out, double + // counting and bypassing the filter. + if def.DiscoverFunc == nil { + continue + } switch def.Type { case parser.AgentKiro: for _, dir := range e.agentDirs[def.Type] { @@ -400,13 +410,29 @@ func sortAndLimitParseDiffFiles( if limit > 0 && len(files) > limit { limited = true for _, f := range files[limit:] { - cutPaths[stripVirtualSourceSuffix(f.Path)] = true + cutPaths[parseDiffSourceKey(f.Path)] = true } files = files[:limit] } return files, cutPaths, limited } +func parseDiffSourceKey(path string) string { + if isOpenCodeFamilyProviderVirtualSource(path) { + return path + } + return stripVirtualSourceSuffix(path) +} + +func isOpenCodeFamilyProviderVirtualSource(path string) bool { + for _, base := range []string{"opencode.db", "kilo.db", "mimocode.db"} { + if _, _, ok := parser.ParseVirtualSourcePathForBase(path, base); ok { + return true + } + } + return false +} + // stripVirtualSourceSuffix maps a stored file_path to its on-disk base // file by removing the "#rawID" suffix that Kiro, Zed, OpenCode, Kilo, // MiMoCode, and Shelley SQLite-backed sessions append to their shared database @@ -426,13 +452,13 @@ func stripVirtualSourceSuffix(path string) string { if dbPath, _, ok := parser.ParseZedSQLiteVirtualPath(path); ok { return dbPath } - if dbPath, _, ok := parser.ParseOpenCodeSQLiteVirtualPath(path); ok { + if dbPath, _, ok := parser.ParseVirtualSourcePathForBase(path, "opencode.db"); ok { return dbPath } - if dbPath, _, ok := parser.ParseKiloSQLiteVirtualPath(path); ok { + if dbPath, _, ok := parser.ParseVirtualSourcePathForBase(path, "kilo.db"); ok { return dbPath } - if dbPath, _, ok := parser.ParseMiMoCodeSQLiteVirtualPath(path); ok { + if dbPath, _, ok := parser.ParseVirtualSourcePathForBase(path, "mimocode.db"); ok { return dbPath } if dbPath, _, ok := parser.ParseShelleyVirtualPath(path); ok { @@ -587,7 +613,7 @@ func (e *Engine) parseDiffCollectFile( resolver worktreeProjectResolver, presencePaths *[]string, ) error { - base := stripVirtualSourceSuffix(job.path) + base := parseDiffSourceKey(job.path) if job.err != nil { storedHere := storedByPath[base] @@ -935,11 +961,12 @@ func parseDiffSweepStored( case s.FilePath == nil || *s.FilePath == "": reason = "source missing" default: - base := stripVirtualSourceSuffix(*s.FilePath) + sourceKey := parseDiffSourceKey(*s.FilePath) + sourcePath := stripVirtualSourceSuffix(*s.FilePath) switch { - case cutPaths[base]: + case cutPaths[sourceKey]: reason = "not sampled (--limit)" - case !statExists(base): + case !statExists(sourcePath): reason = "source missing" default: reason = "not discovered" diff --git a/internal/sync/parsediff_compare_test.go b/internal/sync/parsediff_compare_test.go index ccd9e61c5..abe9cc22e 100644 --- a/internal/sync/parsediff_compare_test.go +++ b/internal/sync/parsediff_compare_test.go @@ -3,6 +3,7 @@ package sync import ( "context" "encoding/json" + "errors" "os" "path/filepath" "strings" @@ -1888,6 +1889,149 @@ func TestParseDiffPresenceSweepSkipsIncompleteProviderResults(t *testing.T) { assert.Empty(t, report.Sessions) } +func TestParseDiffProviderVirtualSQLiteErrorUsesExactSource(t *testing.T) { + dbPath := "/tmp/opencode.db" + firstPath := parser.OpenCodeSQLiteVirtualPath(dbPath, "ses_one") + secondPath := parser.OpenCodeSQLiteVirtualPath(dbPath, "ses_two") + first := &db.Session{ + ID: "opencode:ses_one", + Agent: string(parser.AgentOpenCode), + Machine: "devbox", + Project: "project", + FilePath: &firstPath, + DataVersion: db.CurrentDataVersion(), + } + second := &db.Session{ + ID: "opencode:ses_two", + Agent: string(parser.AgentOpenCode), + Machine: "devbox", + Project: "project", + FilePath: &secondPath, + DataVersion: db.CurrentDataVersion(), + } + storedByPath := map[string][]*db.Session{ + parseDiffSourceKey(firstPath): {first}, + parseDiffSourceKey(secondPath): {second}, + } + job := syncJob{ + path: firstPath, + processResult: processResult{ + err: errors.New("bad virtual session"), + }, + } + engine := &Engine{db: dbtest.OpenTestDB(t)} + report := &ParseDiffReport{FieldCounts: map[string]int{}} + visited := map[string]bool{} + var presencePaths []string + + err := engine.parseDiffCollectFile( + context.Background(), + report, + job, + map[string]parser.AgentType{firstPath: parser.AgentOpenCode}, + map[string]*db.Session{ + first.ID: first, + second.ID: second, + }, + storedByPath, + visited, + engine.loadWorktreeProjectResolver(), + &presencePaths, + ) + require.NoError(t, err) + + require.Len(t, report.Sessions, 1) + assert.Equal(t, first.ID, report.Sessions[0].SessionID) + assert.Equal(t, DiffParseError, report.Sessions[0].Class) + assert.True(t, visited[first.ID]) + assert.False(t, visited[second.ID]) + assert.Empty(t, presencePaths) + assert.Equal(t, ParseDiffTotals{ParseErrors: 1}, report.Totals) +} + +func TestParseDiffProviderVirtualSQLitePresenceUsesExactSource(t *testing.T) { + dbPath := "/tmp/opencode.db" + firstPath := parser.OpenCodeSQLiteVirtualPath(dbPath, "ses_one") + secondPath := parser.OpenCodeSQLiteVirtualPath(dbPath, "ses_two") + first := &db.Session{ + ID: "opencode:ses_one", + Agent: string(parser.AgentOpenCode), + Machine: "devbox", + Project: "project", + FilePath: &firstPath, + DataVersion: db.CurrentDataVersion(), + } + second := &db.Session{ + ID: "opencode:ses_two", + Agent: string(parser.AgentOpenCode), + Machine: "devbox", + Project: "project", + FilePath: &secondPath, + DataVersion: db.CurrentDataVersion(), + } + storedByPath := map[string][]*db.Session{ + parseDiffSourceKey(firstPath): {first}, + parseDiffSourceKey(secondPath): {second}, + } + job := syncJob{path: firstPath} + engine := &Engine{db: dbtest.OpenTestDB(t)} + report := &ParseDiffReport{FieldCounts: map[string]int{}} + visited := map[string]bool{} + var presencePaths []string + + err := engine.parseDiffCollectFile( + context.Background(), + report, + job, + map[string]parser.AgentType{firstPath: parser.AgentOpenCode}, + map[string]*db.Session{ + first.ID: first, + second.ID: second, + }, + storedByPath, + visited, + engine.loadWorktreeProjectResolver(), + &presencePaths, + ) + require.NoError(t, err) + engine.parseDiffPresenceSweep( + report, + presencePaths, + storedByPath, + visited, + ) + + require.Len(t, report.Sessions, 1) + assert.Equal(t, first.ID, report.Sessions[0].SessionID) + assert.Equal(t, DiffChanged, report.Sessions[0].Class) + assert.True(t, visited[first.ID]) + assert.False(t, visited[second.ID]) + assert.Equal(t, ParseDiffTotals{Changed: 1}, report.Totals) +} + +func TestParseDiffProviderVirtualSQLiteLimitUsesExactSource(t *testing.T) { + dbPath := "/tmp/opencode.db" + firstPath := parser.OpenCodeSQLiteVirtualPath(dbPath, "ses_one") + secondPath := parser.OpenCodeSQLiteVirtualPath(dbPath, "ses_two") + _, cutPaths, limited := sortAndLimitParseDiffFiles( + []parser.DiscoveredFile{ + {Path: firstPath, Agent: parser.AgentOpenCode}, + {Path: secondPath, Agent: parser.AgentOpenCode}, + }, + 1, + ) + + require.True(t, limited) + assert.Len(t, cutPaths, 1) + assert.False(t, cutPaths[dbPath]) + for path := range cutPaths { + assert.True(t, + path == firstPath || path == secondPath, + "cut path %q must be one exact virtual source", path, + ) + } +} + func TestParseDiffReportHasFailures(t *testing.T) { tests := []struct { name string diff --git a/internal/sync/progress.go b/internal/sync/progress.go index ca74669a6..17b351986 100644 --- a/internal/sync/progress.go +++ b/internal/sync/progress.go @@ -55,11 +55,17 @@ type SyncStats struct { Warnings []string `json:"warnings,omitempty"` Aborted bool `json:"aborted,omitempty"` - filesOK int // unexported: file-level success counter - filesDiscovered int // file-based total, excludes DB-backed agents - messagesIndexed int // unexported: progress message counter - parserExcludedFiles int // file-level intentional parser exclusions - parserExcludedIDs []string + filesOK int // unexported: file-level success counter + filesDiscovered int // file-based total, excludes DB-backed agents + // nonContainerDiscovered counts discovered files that are not part of + // a self-preserving container store (OpenCode-format storage and its + // SQLite virtual paths). The resync empty-discovery guard uses it so a + // container store's discovery does not mask the disappearance of plain + // file-backed sessions whose directories went empty. + nonContainerDiscovered int + messagesIndexed int // unexported: progress message counter + parserExcludedFiles int // file-level intentional parser exclusions + parserExcludedIDs []string } // RecordSkip increments the skipped session counter.