Skip to content

Make LightningIR compatible with transformers v5 - #169

Open
RaykKretzschmar wants to merge 7 commits into
mainfrom
transformers-v5
Open

Make LightningIR compatible with transformers v5#169
RaykKretzschmar wants to merge 7 commits into
mainfrom
transformers-v5

Conversation

@RaykKretzschmar

Copy link
Copy Markdown
Collaborator

This PR updates Lightning IR for compatibility with Transformers v5.

The main changes are:

  • Update dynamic config/model/tokenizer class factories for Transformers v5 behavior.
  • Preserve and resolve backbone_model_type more robustly when loading derived Lightning IR configs.
  • Adapt model-specific logic for v5, including SPLADE tied-weight metadata and SetEncoder attention mask handling.
  • Update tokenizer handling for newer tokenizer mappings and T5-style cross-encoder formatting.
  • Normalize dataset training targets to float32.
  • Add compatibility shims for external model comparison tests where upstream libraries still assume older Transformers internals.
  • Stabilize shared test configuration and update typing for the Python 3.10+ baseline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant