locuslab · Shubin-vadim · Oct 19, 2025
diff --git a/community/methods/LoRA/README.md b/community/methods/LoRA/README.md
@@ -0,0 +1,133 @@
+# LoRA (Low-Rank Adaptation) Integration
+
+## Overview
+
+This directory contains the implementation of LoRA (Low-Rank Adaptation) integration for the Open-Unlearning project. LoRA allows for efficient fine-tuning and unlearning by adding trainable low-rank matrices to the model while keeping the original parameters frozen.
+
+## Method Details
+
+### What is LoRA?
+
+LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that:
+- Adds trainable low-rank matrices to existing model layers
+- Keeps original model parameters frozen during training
+- Significantly reduces memory usage and training time
+- Maintains performance comparable to full fine-tuning
+
+### Technical Implementation
+
+The LoRA integration includes:
+
+1. **LoRA Model Wrapper** (`src/model/lora.py`)
+   - `LoRAModelForCausalLM` class for loading models with LoRA adapters
+   - Support for custom LoRA parameters (rank, alpha, dropout, target modules)
+   - Automatic device placement with `device_map: "auto"`
+
+2. **Model Integration** (`src/model/__init__.py`)
+   - Added LoRA support to the main `get_model()` function
+   - Automatic detection of `use_lora: true` in configurations
+   - Registration of `LoRAModelForCausalLM` in the model registry
+
+3. **Configuration Files**
+   - Model configurations with LoRA parameters
+   - Experiment configurations for fine-tuning and unlearning
+   - Automatic device placement configuration
+
+### LoRA Parameters
+
+Default LoRA configuration:
+```yaml
+lora_config:
+  target_modules: ["q_proj", "v_proj", "k_proj", "o_proj", "gate_proj", "down_proj", "up_proj", "lm_head"]
+  lora_alpha: 128
+  lora_dropout: 0.05
+  r: 128
+  bias: "none"
+  task_type: "CAUSAL_LM"
+```
+
+### Supported Models
+
+- `Qwen2.5-3B-Instruct-lora`
+- `Llama-2-7b-hf-lora`
+- `Llama-2-7b-chat-hf-lora`
+
+## Hyperparameters and Strategy
+
+### Fine-tuning with LoRA
+- **Learning Rate**: `2e-4` (higher than standard `1e-5`)
+- **Training Epochs**: `3` (fewer than standard `5-10`)
+- **Warmup**: `0.1` epochs (shorter than standard `1.0`)
+- **Batch Size**: `4` with gradient accumulation `4`
+
+### Unlearning with LoRA
+- **Learning Rate**: `1e-4` (higher than standard `1e-5`)
+- **Training Epochs**: `5` (fewer than standard `10`)
+- **Warmup**: `0.1` epochs (shorter than standard `1.0`)
+- **Batch Size**: `4` with gradient accumulation `4`
+
+### Strategy for Selecting Best Model
+
+1. **Memory Efficiency**: LoRA trains only ~1% of model parameters
+2. **Faster Convergence**: Higher learning rates work well with LoRA
+3. **Modularity**: Easy to switch between different LoRA configurations
+4. **Device Optimization**: Automatic device placement for optimal GPU/CPU usage
+
+## Benefits
+
+1. **Memory Efficiency**: Only train a small number of parameters (typically <1% of the original model)
+2. **Faster Training**: Reduced computational requirements
+3. **Modularity**: Easy to switch between different LoRA adapters
+4. **Storage**: Smaller checkpoint sizes
+5. **No Authentication Required**: Works without HuggingFace tokens
+6. **Automatic Device Placement**: Uses `device_map: "auto"` for optimal performance
+
+## Usage
+
+### Fine-tuning with LoRA
+```bash
+# TOFU dataset
+python src/train.py --config-name=train @experiment=finetune/tofu/lora
+
+# MUSE dataset
+python src/train.py --config-name=train @experiment=finetune/muse/lora
+```
+
+### Unlearning with LoRA
+```bash
+# TOFU dataset
+python src/train.py --config-name=unlearn @experiment=unlearn/tofu/lora
+
+# MUSE dataset
+python src/train.py --config-name=unlearn @experiment=unlearn/muse/lora
+
+# WMDP dataset
+python src/train.py --config-name=unlearn @experiment=unlearn/wmdp/lora
+```
+
+### Custom Model Selection
+```bash
+python src/train.py --config-name=train @experiment=finetune/tofu/lora model=Llama-2-7b-hf-lora
+```
+
+## Dependencies
+
+- `peft==0.17.1` - Parameter-Efficient Fine-Tuning library
+- Standard HuggingFace ecosystem (transformers, torch, etc.)
+
+## Environment Variables
+
+- `HF_HOME`: Cache directory for HuggingFace models (optional)
+
+## References
+
+- [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)
+- [Parameter-Efficient Fine-Tuning (PEFT) Library](https://github.com/huggingface/peft)
+
+## Implementation Notes
+
+- LoRA adapters are applied to attention layers (`q_proj`, `v_proj`, `k_proj`, `o_proj`) and MLP layers (`gate_proj`, `down_proj`, `up_proj`)
+- The `lm_head` layer is also adapted for better performance
+- Default rank `r=128` provides a good balance between performance and efficiency
+- `lora_alpha=128` scales the LoRA contributions appropriately
+- `device_map: "auto"` automatically places model layers across available devices
diff --git a/community/methods/LoRA/run.sh b/community/methods/LoRA/run.sh
@@ -0,0 +1,123 @@
+#!/bin/bash
+
+# LoRA Integration Experiments for Open-Unlearning
+# This script demonstrates how to run fine-tuning and unlearning experiments with LoRA
+
+set -e
+
+echo "🚀 Starting LoRA Integration Experiments"
+echo "========================================"
+
+# Set default values
+MODEL="Qwen2.5-3B-Instruct-lora"
+EXPERIMENT_TYPE="finetune"
+DATASET="tofu"
+
+# Parse command line arguments
+while [[ $# -gt 0 ]]; do
+    case $1 in
+        --model)
+            MODEL="$2"
+            shift 2
+            ;;
+        --type)
+            EXPERIMENT_TYPE="$2"
+            shift 2
+            ;;
+        --dataset)
+            DATASET="$2"
+            shift 2
+            ;;
+        --help)
+            echo "Usage: $0 [--model MODEL] [--type TYPE] [--dataset DATASET]"
+            echo ""
+            echo "Options:"
+            echo "  --model MODEL     LoRA model to use (default: Qwen2.5-3B-Instruct-lora)"
+            echo "                    Available: Qwen2.5-3B-Instruct-lora, Llama-2-7b-hf-lora, Llama-2-7b-chat-hf-lora"
+            echo "  --type TYPE       Experiment type: finetune or unlearn (default: finetune)"
+            echo "  --dataset DATASET Dataset to use: tofu, muse, or wmdp (default: tofu)"
+            echo "  --help            Show this help message"
+            echo ""
+            echo "Examples:"
+            echo "  $0                                    # Fine-tune Qwen2.5-3B-Instruct with LoRA on TOFU"
+            echo "  $0 --type unlearn                      # Unlearn with Qwen2.5-3B-Instruct LoRA on TOFU"
+            echo "  $0 --model Llama-2-7b-hf-lora         # Fine-tune Llama-2-7b-hf with LoRA"
+            echo "  $0 --dataset muse --type unlearn      # Unlearn with Qwen2.5-3B-Instruct LoRA on MUSE"
+            echo "  $0 --dataset wmdp --type unlearn      # Unlearn with Qwen2.5-3B-Instruct LoRA on WMDP"
+            exit 0
+            ;;
+        *)
+            echo "Unknown option: $1"
+            echo "Use --help for usage information"
+            exit 1
+            ;;
+    esac
+done
+
+echo "Configuration:"
+echo "  Model: $MODEL"
+echo "  Type: $EXPERIMENT_TYPE"
+echo "  Dataset: $DATASET"
+echo ""
+
+# Validate inputs
+if [[ "$EXPERIMENT_TYPE" != "finetune" && "$EXPERIMENT_TYPE" != "unlearn" ]]; then
+    echo "❌ Error: Experiment type must be 'finetune' or 'unlearn'"
+    exit 1
+fi
+
+if [[ "$DATASET" != "tofu" && "$DATASET" != "muse" && "$DATASET" != "wmdp" ]]; then
+    echo "❌ Error: Dataset must be 'tofu', 'muse', or 'wmdp'"
+    exit 1
+fi
+
+# Check if model configuration exists
+MODEL_CONFIG="configs/model/${MODEL}.yaml"
+if [[ ! -f "$MODEL_CONFIG" ]]; then
+    echo "❌ Error: Model configuration not found: $MODEL_CONFIG"
+    echo "Available LoRA models:"
+    ls configs/model/*-lora.yaml 2>/dev/null | sed 's/configs\/model\///g' | sed 's/\.yaml//g' | sed 's/^/  - /'
+    exit 1
+fi
+
+# Check if experiment configuration exists
+if [[ "$DATASET" == "wmdp" && "$EXPERIMENT_TYPE" == "finetune" ]]; then
+    echo "❌ Error: WMDP dataset only supports unlearning, not fine-tuning"
+    echo "Use --type unlearn for WMDP dataset"
+    exit 1
+fi
+
+EXPERIMENT_CONFIG="configs/experiment/${EXPERIMENT_TYPE}/${DATASET}/lora.yaml"
+if [[ ! -f "$EXPERIMENT_CONFIG" ]]; then
+    echo "❌ Error: Experiment configuration not found: $EXPERIMENT_CONFIG"
+    echo "Available experiment configurations:"
+    find configs/experiment -name "lora.yaml" | sed 's/^/  - /'
+    exit 1
+fi
+
+echo "✅ All configurations found"
+echo ""
+
+# Set up experiment command
+if [[ "$EXPERIMENT_TYPE" == "finetune" ]]; then
+    TRAIN_CONFIG="train"
+else
+    TRAIN_CONFIG="unlearn"
+fi
+
+# Build the command
+CMD="python src/train.py --config-name=${TRAIN_CONFIG} @experiment=${EXPERIMENT_TYPE}/${DATASET}/lora model=${MODEL}"
+
+echo "Running command:"
+echo "  $CMD"
+echo ""
+
+# Run the experiment
+echo "🏃 Starting experiment..."
+eval $CMD
+
+echo ""
+echo "✅ Experiment completed!"
+echo ""
+echo "Results should be saved in the output directory."
+echo "Check the logs for detailed information about the training process."
diff --git a/configs/experiment/finetune/muse/lora.yaml b/configs/experiment/finetune/muse/lora.yaml
@@ -0,0 +1,43 @@
+# @package _global_
+
+defaults:
+  - override /model: Qwen2.5-3B-Instruct-lora
+  - override /trainer: finetune
+  - override /data/[email protected]: MUSE_train
+  - override /eval: muse
+  - override /data: finetune
+
+mode: finetune
+data_split: News
+data_sub_set: full   # full or retain
+
+data:
+  train:
+    MUSE_train:
+      args:
+        hf_args:
+          path: tamarsonha/MUSE-${data_split}-Train
+          split: ${data_sub_set}
+# you can find fine-tuned models on https://huggingface.co/tamarsonha
+
+trainer:
+  args:
+    learning_rate: 2e-4  # Higher learning rate for LoRA
+    weight_decay: 0.01
+    warmup_epochs: 0.1  # Shorter warmup for LoRA
+    num_train_epochs: 3  # Fewer epochs for LoRA
+    per_device_train_batch_size: 4
+    per_device_eval_batch_size: 4
+    gradient_accumulation_steps: 4
+    logging_steps: 10
+    save_steps: 500
+    eval_steps: 500
+    evaluation_strategy: steps
+    save_strategy: steps
+    load_best_model_at_end: false  # Disable to avoid metric issues
+    save_total_limit: 2
+    remove_unused_columns: false
+    dataloader_pin_memory: false
+    seed: 42
+
+task_name: muse_news_full_lora
diff --git a/configs/experiment/finetune/tofu/lora.yaml b/configs/experiment/finetune/tofu/lora.yaml
@@ -0,0 +1,42 @@
+# @package _global_
+
+defaults:
+  - override /model: Qwen2.5-3B-Instruct-lora
+  - override /trainer: finetune
+  - override /data/[email protected]: TOFU_QA_full
+  - override /eval: tofu
+
+mode: finetune
+
+trainer:
+  args:
+    learning_rate: 2e-4  # Higher learning rate for LoRA
+    weight_decay: 0.01
+    warmup_epochs: 0.1
+    num_train_epochs: 3
+    per_device_train_batch_size: 4
+    per_device_eval_batch_size: 4
+    gradient_accumulation_steps: 4
+    logging_steps: 10
+    save_steps: 500
+    eval_steps: 500
+    evaluation_strategy: steps
+    save_strategy: steps
+    load_best_model_at_end: false  # Disable to avoid metric issues
+    save_total_limit: 2
+    remove_unused_columns: false
+    dataloader_pin_memory: false
+    seed: 42
+
+forget_split: forget10
+holdout_split: holdout10
+retain_logs_path: null
+
+eval:
+  tofu:
+    forget_split: ${forget_split}
+    holdout_split: ${holdout_split}
+    retain_logs_path: ${retain_logs_path}
+    overwrite: true
+
+task_name: tofu_Qwen2.5-3B-Instruct_lora_finetune