feat!(backend): refactor multi-segment submission

[anna-parker]

resolves #4708, #4734

partially resolves #5392, #5185 (comment)

Breaking Changes

When users submit to multi-segmented organisms and want to group multiple segments under one metadata entry they are required to add an additional fastaId column with a comma-separated list of the fastaIds (fasta header IDs) of the respective sequences. If no fastaId column is supplied the submissionId will be used instead and the backend will assume that (as in the single-segmented case) there is a one-to-one mapping of metadata submissionId to fastaId.

This requires:

• changes to the edit-page, started in feat!(website, prepro, backend, config, integration):multi pathogen - refactor multi segment submission in backend and edit page and have prepro assign segments #5382
• requires Single sequence revise form does not surface backend error nicely when sequence discarded/missing #4850 to be fixed -> this is leading to failing integration tests

Future Steps

For simplicity in this PR prepro does not yet use a minimizer index to assign segments but parses the fasta header (as the backend did before). In future steps we should

1. do a migration to change the originalData unalignedSequences keys for older processed versions
2. update the processedData return type to include a mapping from fasta header ID to segment
3. have prepro use a minimizer index to assign segments

PR Checklist

• Update ingest for multi-segmented organism (CCHF)
• Use prepro updates from feat!(backend): remove fasta header validation from backend and refactor how fasta and metadata are merged #4821 and feat!(prepro, config): assign segment with nextclade sort #4783
• Update integration tests
• Fix backend tests
• Add backend test that multi-segmented can use id column instead of fastaId as well

🚀 Preview: https://multi-segment-submission.loculus.org

[anna-parker]

As discussed we should change to a space-separated internal separator.

We could also allow a different column for each segment (and only require that the prefix be called fastaId) -> lets maybe ask on microbioinfo which option would be easier for people

Update: consensus for this option reached: https://microbial-bioinfo.slack.com/archives/CB0HYT53M/p1760961465729399

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[anna-parker]

I will close this in favor of #5398