Fix state-dict round-trip for directly injected shared tuners

[CoralGarden52]

Description

The documented low-level adapter API does not round-trip shared tuner state when adapters are injected directly.

The low-level API supports inject_adapter_in_model(model, config) followed by get_peft_model_state_dict and set_peft_model_state_dict. Direct injection returns the original model with PEFT layers inserted, while shared tensors remain registered on the temporary BaseTuner. Several tuner-specific state-dict hooks still assume the base_model.* keys from the PeftModel wrapper.

This affects:

• TinyLoRA: tinylora_v is silently omitted from the saved state dict.
• UniLoRA: saving raises a KeyError for base_model.unilora_theta_d.
• VeRA and PVeRA: saving with the default save_projection=True raises a projection-missing ValueError.
• VBLoRA: saving raises a KeyError for base_model.vblora_vector_bank.
• FRoD: projection buffers are silently omitted and loading then raises a projection-missing ValueError.

Reproduction

With a small torch.nn.Linear model on the current main branch:

model = inject_adapter_in_model(config, model)
state_dict = get_peft_model_state_dict(model)
set_peft_model_state_dict(model, state_dict)

For TinyLoRA on cuda:0, setting the source shared vector to 0.7 produced:

saved keys = ['lin0.tinylora_A', 'lin0.tinylora_B', 'lin0.tinylora_P']
missing adapter key = 'lin0.tinylora_v.default.0'
source v mean = 0.7
loaded v mean = 0.0
max output diff = 0.8507426

The corresponding get_peft_model wrapper path saves the shared tensors correctly for all six tuners.

Expected behavior

Directly injected shared tuners should save all adapter-owned shared tensors, load them without adapter-related missing keys, and preserve the model output. The existing get_peft_model save/load behavior should remain unchanged.

This is separate from #3495, which concerns UniLoRA trainability during injection rather than state-dict serialization.

[CoralGarden52]

I would like to take this fix myself. Please review the proposed scope and direction when you have a moment. If everything looks good, please explicitly confirm that I may proceed. Once I get approval, I'll prepare a focused PR with the necessary tests and style checks. I will wait for approval before starting the implementation and submitting the PR. Thanks!

[BenjaminBossan]

IMO, inject_adapter_in_model should raise in such cases. We could add a new class level attribute on the models like uses_shared_data or so, which we enable for said PEFT models. When it's detected, we raise an error in inject_adapter_in_model.

I'm sure that we could also get it to work somehow, as evidenced by #3642, but that looks very hacky to me and I'd rather avoid it tbh. @githubnemo do you have an opinion on that?

[CoralGarden52]

Hi! @BenjaminBossan
Thanks for the suggestion. Following your recommendation, I opened a new draft PR implementing the class-level capability marker approach instead of remapping shared state-dict keys.

The new draft PR adds BaseTuner.uses_shared_state, enables it for TinyLoRA, UniLoRA, VeRA, PVeRA, VBLoRA, and FRoD, and makes inject_adapter_in_model raise before modifying the model for these tuners. get_peft_model remains supported, and the low-level API documentation now states this limitation.