Skip to content

added migration order to respect FK dependencies - #26

Open
motriys98-alt wants to merge 1 commit into
taylorwilsdon:mainfrom
motriys98-alt:migration_priority
Open

added migration order to respect FK dependencies#26
motriys98-alt wants to merge 1 commit into
taylorwilsdon:mainfrom
motriys98-alt:migration_priority

Conversation

@motriys98-alt

Copy link
Copy Markdown

Problem

The migration script (migrator.py) migrates tables from SQLite to PostgreSQL in the order returned by sqlite_master, which does not respect foreign key (FK) dependencies.
This causes Foreign Key constraint violations for child tables that reference parent tables not yet migrated:

Child Table FK Constraint Missing Parent Key Rows Lost
chat_file chat_file_chat_id_fkey → chat(id) chat_id = '075ec634-...' 33+
chat_message chat_message_chat_id_fkey → chat(id) chat_id = 'a2aa8fcf-...' 243+
Total: 276+ rows of data lost due to FK violations during migration.

Root Cause

The migration loop uses the raw order from sqlite_master:

Old code — order from sqlite_master (arbitrary, ignores FK)

sqlite_cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
tables = sqlite_cursor.fetchall()

for (table_name,) in tables:
await process_table(table_name, ...)
In the original order, chat was migrated after chat_file and chat_message, causing FK violations.

Solution

  1. Add TABLE_MIGRATION_PRIORITY — explicit priority map for all 41 Open WebUI tables. Parent tables get lower numbers (migrated first), child tables get higher numbers.
  2. Add resolve_migration_order() — sorts tables by priority before migration.
  3. Update migration loop — uses priority-sorted order instead of raw sqlite_master order.

Key priority anchors:

Table Priority Role
chat 70 Parent of chat_file, chat_message
chat_file 110 Child — migrated after chat
chat_message 111 Child — migrated after chat

@FedorenkoKirill

Copy link
Copy Markdown

hit exactly this on a real OWUI migration — 3 tables (chat_file, knowledge_file, group_member) landed with 0 rows. Your hardcoded priority map is a straightforward fix; a retry pass over failed tables might be more resilient to future OWUI schema additions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants