Skip to content

Bucardo chokes on large tables — let's delay indexes! #5

@jessehanley

Description

@jessehanley

For databases with large tables, say >1TB with a handful of indexes on it, a little network hiccup can cause the whole onetimecopy to fail. We can drastically speed up the initial copy by building indexes AFTER the copy instead of during.

By doing this, we can take a ~4TB migration down from 3 days to 10-12 hours. This means someone could easily pull it off in a day on a Heroku dyno without worrying about the daily resets too.

Can we consider adding rebuild_index=1 to the initial sync?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions