[docs] peft (huggingface#44804)

* peft

* feedback

Author: stevhliu

docs/source/en/_toctree.yml

@@ -163,6 +163,8 @@
     - local: hpo_train
       title: Hyperparameter search
     title: Customization
+  - local: peft
+    title: Parameter-efficient fine-tuning
   - isExpanded: false
     sections:
     - local: accelerator_selection
@@ -195,8 +197,6 @@
     - local: model_memory_anatomy
       title: Model training anatomy
     title: Hardware
-  - local: peft
-    title: PEFT
   title: Training
 - isExpanded: false
   sections:

docs/source/en/main_classes/peft.md

@@ -11,13 +11,15 @@ rendered properly in your Markdown viewer.
 
 # PEFT
 
-The [`~integrations.PeftAdapterMixin`] provides functions from the [PEFT](https://huggingface.co/docs/peft/index) library for managing adapters with Transformers. This mixin currently supports LoRA, IA3, and AdaLora. Prefix tuning methods (prompt tuning, prompt learning) aren't supported because they can't be injected into a torch module.
+The [`~integrations.PeftAdapterMixin`] provides functions from the [PEFT](https://huggingface.co/docs/peft/index) library for managing adapters with Transformers. This mixin supports all non-prompt-learning PEFT methods (LoRA, IA3, AdaLoRA, and others). Prefix tuning methods (prompt tuning, prompt learning) aren't supported because they can't be injected into a torch module.
 
 [[autodoc]] integrations.PeftAdapterMixin
     - load_adapter
     - add_adapter
     - set_adapter
     - disable_adapters
     - enable_adapters
+    - enable_peft_hotswap
     - active_adapters
     - get_adapter_state_dict
+    - delete_adapter

docs/source/en/peft.md

@@ -9,98 +9,130 @@ specific language governing permissions and limitations under the License.
 rendered properly in your Markdown viewer.
 -->
 
-# PEFT
+# Parameter-efficient fine-tuning
 
-[[open-in-colab]]
+[Parameter-efficient fine-tuning (PEFT)](https://huggingface.co/docs/peft/index) methods only fine-tune a small number of extra model parameters (adapters) on top of a pretrained model. Because only adapter parameters are updated, the optimizer tracks far fewer gradients and states, reducing memory usage significantly. Adapters are lightweight, making them convenient to share, store, and load.
 
-[PEFT](https://huggingface.co/docs/peft/index), a library of parameter-efficient fine-tuning methods, enables training and storing large models on consumer GPUs. These methods only fine-tune a small number of extra model parameters, also known as adapters, on top of the pretrained model. A significant amount of memory is saved because the GPU doesn't need to store the optimizer states and gradients for the pretrained base model. Adapters are very lightweight, making it convenient to share, store, and load them.
+Transformers integrates directly with the PEFT library through [`~integrations.PeftAdapterMixin`], added to all [`PreTrainedModel`] classes. You can load, add, train, switch, and delete adapters without wrapping your model in a separate [`~peft.PeftModel`]. All non-prompt-learning PEFT methods are supported (LoRA, IA3, AdaLoRA). Prompt-based methods like prompt tuning and prefix tuning require using the [PEFT library](https://huggingface.co/docs/peft/index) directly.
 
-This guide provides a short introduction to the PEFT library and how to use it for training with Transformers. For more details, refer to the PEFT [documentation](https://huggingface.co/docs/peft/index).
+Install PEFT to get started. The integration requires `peft >= 0.18.0`.
 
-Install PEFT with the command below.
-
-<hfoptions id="install">
-<hfoption id="pip">
-
-```bash
+```shell
 pip install -U peft
 ```
 
-</hfoption>
-<hfoption id="source">
+## Add an adapter
 
-```bash
-pip install git+https://github.com/huggingface/peft.git
-```
+Create a PEFT config, like [`~peft.LoraConfig`] for example, and attach it to a model with [`~integrations.PeftAdapterMixin.add_adapter`].
 
-</hfoption>
-</hfoptions>
+```py
+from peft import LoraConfig, TaskType
+from transformers import AutoModelForCausalLM
 
-> [!TIP]
-> PEFT currently supports the LoRA, IA3, and AdaLoRA methods for Transformers. To use another PEFT method, such as prompt learning or prompt tuning, use the PEFT library directly.
+model = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b")
 
-[Low-Rank Adaptation (LoRA)](https://huggingface.co/docs/peft/conceptual_guides/adapter#low-rank-adaptation-lora) is a very common PEFT method that decomposes the weight matrix into two smaller trainable matrices. Start by defining a [LoraConfig](https://huggingface.co/docs/peft/package_reference/lora#peft.LoraConfig) object with the parameters shown below.
+lora_config = LoraConfig(
+    task_type=TaskType.CAUSAL_LM,
+    inference_mode=False,
+    r=8,
+    lora_alpha=32,
+    lora_dropout=0.1,
+)
 
-```py
-from peft import LoraConfig, TaskType, get_peft_model
-from transformers import AutoModelForCausalLM
+model.add_adapter(lora_config, adapter_name="my_adapter")
+```
+
+### Fully fine-tuning specific layers
 
-# create LoRA configuration object
+To train additional modules alongside an adapter (for example, the language model head), specify them in `modules_to_save`. `modules_to_save` specifies layers that are fully fine-tuned alongside the adapter, so *all* of their parameters are updated. This is useful when certain layers need updates, for example the language model head (`lm_head`), when adapting a causal LM for sequence classification.
+
+```py
 lora_config = LoraConfig(
-    task_type=TaskType.CAUSAL_LM, # type of task to train on
-    inference_mode=False, # set to False for training
-    r=8, # dimension of the smaller matrices
-    lora_alpha=32, # scaling factor
-    lora_dropout=0.1 # dropout of LoRA layers
+    modules_to_save=["lm_head"],
+    ...
 )
+model.add_adapter(lora_config)
 ```
 
-Add [LoraConfig](https://huggingface.co/docs/peft/package_reference/lora#peft.LoraConfig) to the model with [`~integrations.PeftAdapterMixin.add_adapter`]. The model is now ready to be passed to [`Trainer`] for training.
+### Choosing which layers to adapt
+
+For common architectures (Llama, Gemma, Qwen, etc.), PEFT has predefined default targets (like `q_proj` and `v_proj`), so you don't need to specify `target_modules`. If you want to target different layers, or the model doesn't have predefined targets, pass `target_modules` explicitly as a list of module names or a regex pattern.
 
 ```py
-model.add_adapter(lora_config, adapter_name="lora_1")
-trainer = Trainer(model=model, ...)
-trainer.train()
+lora_config = LoraConfig(
+    target_modules=["q_proj", "k_proj"],
+    ...
+)
+model.add_adapter(lora_config)
 ```
 
-To add an additional trainable adapter on top of a model with an existing adapter attached, specify the modules you want to train in [modules_to_save()](https://huggingface.co/docs/peft/package_reference/lora#peft.LoraConfig.modules_to_save).
+## Training
 
-For example, to train the `lm_head` module on top of a causal language model with a LoRA adapter attached, set `modules_to_save=["lm_head"]`. Add the adapter to the model as shown below, and then pass it to [`Trainer`].
+Pass the model with an attached adapter to [`Trainer`] and call [`~Trainer.train`]. [`Trainer`] only updates the adapter parameters (those with `requires_grad=True`) because the base model is frozen.
 
 ```py
-from transformers import AutoModelForCausalLM
-from peft import LoraConfig
+from transformers import Trainer, TrainingArguments
 
-model = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b")
+training_args = TrainingArguments(
+    output_dir="./output",
+    num_train_epochs=3,
+    per_device_train_batch_size=4,
+)
 
-lora_config = LoraConfig(
-    target_modules=["q_proj", "k_proj"],
-    modules_to_save=["lm_head"],
+trainer = Trainer(
+    model=model,
+    args=training_args,
+    train_dataset=dataset,
 )
 
-model.add_adapter(lora_config)
-trainer = Trainer(model=model, ...)
 trainer.train()
 ```
 
-Save your adapter with [`~PreTrainedModel.save_pretrained`] to reuse it.
+During training, [`Trainer`] checkpoints contain only the adapter weights (`adapter_model.safetensors`) and configuration (`adapter_config.json`), keeping checkpoints small. The base model isn't included.
+
+After training, save the final adapter with [`~PreTrainedModel.save_pretrained`].
+
+```py
+model.save_pretrained("./my_adapter")
+```
+
+### Resuming from a checkpoint
+
+[`Trainer`] automatically detects adapter checkpoints when resuming. [`Trainer`] scans the checkpoint directory for subdirectories containing adapter weights and reloads each adapter with the correct trainable state.
+
+```py
+trainer.train(resume_from_checkpoint="./output/checkpoint-1000")
+```
+
+### Distributed training
 
-## Load adapter
+PEFT adapters work with distributed training out of the box.
 
-To load an adapter with Transformers, the Hub repository or local directory must contain an `adapter_config.json` file and the adapter weights. Load the adapter with [`~PreTrainedModel.from_pretrained`] or with [`~integrations.PeftAdapterMixin.load_adapter`].
+For ZeRO-3, [`Trainer`] passes `exclude_frozen_parameters=True` when saving checkpoints with a PEFT model. Frozen base model weights are skipped. Only the trainable adapter parameters are saved, reducing checkpoint size and save time.
+
+For FSDP, [`Trainer`] updates the FSDP auto-wrap policy to correctly handle LoRA layers. For QLoRA (quantized base model + LoRA), [`Trainer`] also adjusts the mixed precision policy to match the quantization storage dtype.
+
+## Loading an adapter
+
+To load an adapter, the Hub repository or local directory must contain an `adapter_config.json` file and the adapter weights.
 
 <hfoptions id="load">
 <hfoption id="from_pretrained">
 
+[`~PreTrainedModel.from_pretrained`] automatically detects adapters. When it finds an `adapter_config.json`, it reads the `base_model_name_or_path` field to load the correct base model, then loads the adapter on top.
+
 ```py
 from transformers import AutoModelForCausalLM
 
+# Automatically loads the base model and attaches the adapter
 model = AutoModelForCausalLM.from_pretrained("klcsp/gemma7b-lora-alpaca-11-v1")
 ```
 
 </hfoption>
 <hfoption id="load_adapter">
 
+To load an adapter onto an existing model, use [`~integrations.PeftAdapterMixin.load_adapter`].
+
 ```py
 from transformers import AutoModelForCausalLM
 
@@ -111,9 +143,7 @@ model.load_adapter("klcsp/gemma7b-lora-alpaca-11-v1")
 </hfoption>
 </hfoptions>
 
-For very large models, it is helpful to load a quantized version of the model in 8 or 4-bit precision to save memory. Transformers supports quantization with its [bitsandbytes](https://huggingface.co/docs/bitsandbytes/index) integration. Specify in [`BitsAndBytesConfig`] whether you want to load a model in 8 or 4-bit precision.
-
-For multiple devices, add `device_map="auto"` to automatically distribute the model across your hardware.
+For large models, load a quantized version in 8-bit or 4-bit precision with [bitsandbytes](./quantization/bitsandbytes) to save memory. Add `device_map="auto"` to distribute the model across available hardware.
 
 ```py
 from transformers import AutoModelForCausalLM, BitsAndBytesConfig
@@ -125,75 +155,91 @@ model = AutoModelForCausalLM.from_pretrained(
 )
 ```
 
-## Set adapter
+## Managing multiple adapters
 
-[`~integrations.PeftAdapterMixin.add_adapter`] adds a new adapter to a model. To add a second adapter, the new adapter must be the same type as the first adapter. Use the `adapter_name` parameter to assign a name to the adapter.
+A model can hold multiple adapters at once. Add adapters with unique names, and switch between them as needed.
 
 ```py
-model.add_adapter(lora_config, adapter_name="lora_2")
+from peft import LoraConfig
+
+model.add_adapter(LoraConfig(r=8, lora_alpha=32), adapter_name="adapter_1")
+model.add_adapter(LoraConfig(r=16, lora_alpha=64), adapter_name="adapter_2")
 ```
 
-Once added, use [`~integrations.PeftAdapterMixin.set_adapter`] to force a model to use the specified adapter and disable the other adapters.
+Use [`~integrations.PeftAdapterMixin.set_adapter`] to activate a specific adapter. The other adapters are disabled but remain in memory.
 
 ```py
-model.set_adapter("lora_2")
+model.set_adapter("adapter_2")
 ```
 
-## Enable and disable adapter
-
-[`~integrations.PeftAdapterMixin.enable_adapters`] is a broader function that enables *all* adapters attached to a model, and [`~integrations.PeftAdapterMixin.disable_adapters`] disables *all* attached adapters.
+[`~integrations.PeftAdapterMixin.enable_adapters`] enables all attached adapters, and [`~integrations.PeftAdapterMixin.disable_adapters`] disables all of them.
 
 ```py
-model.add_adapter(lora_1)
-model.add_adapter(lora_2)
+# Disable all adapters for base model inference
+model.disable_adapters()
+
+# Re-enable all adapters
 model.enable_adapters()
+```
 
-# disable all adapters
-model.disable_adapters()
+Use [`~integrations.PeftAdapterMixin.active_adapters`] to see which adapters are currently active.
+
+```py
+model.active_adapters()
+# ["adapter_1"]
 ```
 
-## Hotswapping adapters
+Remove adapters you no longer need with [`~integrations.PeftAdapterMixin.delete_adapter`] to free memory.
 
-A common use case when serving multiple adapters is to load one adapter first, generate output, load another adapter, generate more outputs, load another adapter, etc. This can be inefficient, since each time a new adapter is loaded, new memory is reserved; moreover, if the model is compiled with `torch.compile`, it needs to be re-compiled each time a new adapter is used. When switching frequently, the compilation time may never be amortized.
+```py
+model.delete_adapter("adapter_1")
+```
 
-To better support this common workflow, you can "hotswap" a LoRA adapter, to avoid accumulating memory and, in some cases, recompilation. It requires an adapter to already be loaded, and the new adapter weights are swapped in-place for the existing adapter. Note that other PEFT methods are not supported yet, only LoRA.
+## Hotswapping adapters
+
+Loading a new adapter each time you serve a request allocates new memory. If the model is compiled with `torch.compile`, each new adapter triggers recompilation. Hotswapping replaces adapter weights in-place, avoiding both issues. Only LoRA adapters are supported.
 
-Pass `hotswap=True` when loading a LoRA adapter to enable this feature. It is important to indicate the name of the existing adapter (`"default"` is the default adapter name) to be swapped.
+Pass `hotswap=True` when loading a LoRA adapter to swap its weights into an existing adapter slot. Set `adapter_name` to the name of the adapter to replace (`"default"` is the default adapter name).
 
-```python
+```py
 model = AutoModel.from_pretrained(...)
-# load adapter 1 as normal
-model.load_adapter(file_name_adapter_1)
-# generate outputs with adapter 1
+# Load the first adapter normally
+model.load_adapter(adapter_path_1)
+# Generate outputs with adapter 1
 ...
-# now hotswap the 2nd adapter
-model.load_adapter(file_name_adapter_2, hotswap=True, adapter_name="default")
-# generate outputs with adapter 2
+# Hotswap the second adapter in-place
+model.load_adapter(adapter_path_2, hotswap=True, adapter_name="default")
+# Generate outputs with adapter 2
 ```
 
-For compiled models, it is often necessary to call [`~integrations.peft.PeftAdapterMixin.enable_peft_hotswap`] to avoid recompilation. Call this method *before* loading the first adapter, while `torch.compile` should be called *after* loading the first adapter.
+### torch.compile
+
+For compiled models, call [`~integrations.peft.PeftAdapterMixin.enable_peft_hotswap`] *before* loading the first adapter and before compiling.
 
-```python
+```py
 model = AutoModel.from_pretrained(...)
-max_rank = ...  # the highest rank among all LoRAs that you want to load
-# call *before* compiling and loading the LoRA adapter
+max_rank = ...  # highest rank among all LoRAs you'll load
 model.enable_peft_hotswap(target_rank=max_rank)
-model.load_adapter(file_name_1, adapter_name="default")
-# optionally compile the model now
+model.load_adapter(adapter_path_1, adapter_name="default")
 model = torch.compile(model, ...)
 output_1 = model(...)
-# now you can hotswap the 2nd adapter, use the same name as for the 1st
-model.load_adapter(file_name_2, adapter_name="default")
+
+# Hotswap without recompilation
+model.load_adapter(adapter_path_2, adapter_name="default")
 output_2 = model(...)
 ```
 
-The `target_rank=max_rank` argument is important for setting the maximum rank among all LoRA adapters that will be loaded. If you have one adapter with rank 8 and another with rank 16, pass `target_rank=16`. You should use a higher value if in doubt. By default, this value is 128.
+The `target_rank` argument sets the maximum rank among all LoRA adapters you'll load. If you have adapters with rank 8 and rank 16, pass `target_rank=16`. The default is 128.
 
-By default, hotswapping is disabled and requires you to pass `hotswap=True` to `load_adapter`. However, if you called `enable_peft_hotswap` first, hotswapping will be enabled by default. If you want to avoid using it, you need to pass `hotswap=False`.
+After calling `enable_peft_hotswap`, all subsequent `load_adapter` calls hotswap by default. Pass `hotswap=False` explicitly to disable hotswapping.
 
-However, there can be situations where recompilation is unavoidable. For example, if the hotswapped adapter targets more layers than the initial adapter, then recompilation is triggered. Try to load the adapter that targets the most layers first. Refer to the PEFT docs on [hotswapping](https://huggingface.co/docs/peft/main/en/package_reference/hotswap#peft.utils.hotswap.hotswap_adapter) for more details about the limitations of this feature.
+Recompilation may still occur if the hotswapped adapter targets more layers than the initial adapter. Load the adapter that targets the most layers first to avoid recompilation.
+
+> [!TIP]
+> Wrap your code in `with torch._dynamo.config.patch(error_on_recompile=True)` to detect unexpected recompilation. If you detect recompilation despite following the steps above, open an issue with [PEFT](https://github.com/huggingface/peft/issues) with a reproducible example.
 
-> [!Tip]
-> Move your code inside the `with torch._dynamo.config.patch(error_on_recompile=True)` context manager to detect if a model was recompiled. If you detect recompilation despite following all the steps above, please open an issue with [PEFT](https://github.com/huggingface/peft/issues) with a reproducible example.
+## Next steps
 
-For an example of how the use of `torch.compile` in combination with hotswapping can improve runtime, check out [this blogpost](https://huggingface.co/blog/lora-fast). Although that example uses Diffusers, similar improvements can be expected here.
+- The PEFT [documentation](https://huggingface.co/docs/peft/index) covers the full range of PEFT methods and options.
+- The PEFT [hotswapping reference](https://huggingface.co/docs/peft/main/en/package_reference/hotswap#peft.utils.hotswap.hotswap_adapter) details limitations and edge cases.
+- A [blog post](https://huggingface.co/blog/lora-fast) benchmarks how `torch.compile` with hotswapping improves runtime.