PyTorch CUDA Device Alignment Fix for ACE-Step on RX 9070 XT

[EmmanuelWorrell]

2026-08-16 11:47:34

🚨 PyTorch CUDA Device Alignment Fix for ACE-Step on RX 9070 XT

Problem:
Encountered "Expected all tensors to be on the same device" error when using ACE-Step with mixed CPU/CUDA tensors.

Solution:
I've created PowerShell scripts to automatically fix device alignment issues in the ACE-Step framework. These scripts patch both the generation handler and HuggingFace model cache files.

# PyTorch CUDA Device Alignment Fix for ACE-Step

## Problem

When running ACE-Step on CUDA-enabled hardware, you might encounter this error:

```python
RuntimeError: Expected all tensors to be on the same device, but got mat2 is on cpu, different from other tensors on cuda:0

This occurs because some model components (especially dynamically added ones) are not properly moved to the GPU device.

Solution

Here are two PowerShell scripts that automatically fix the device alignment issue:

1. 01_Patch_GenerationHandler.ps1
2. 02_Patch_HFModelCache.ps1

Installation & Usage

# Save the scripts to your ACE-Step project directory
cd C:\ACE-Step-1.5\
mkdir patches
cd patches

# Copy the scripts (you'll need to host them yourself)
New-Item -ItemType File -Path "01_Patch_GenerationHandler.ps1" -Force
New-Item -ItemType File -Path "02_Patch_HFModelCache.ps1" -Force

# Set execution policy (run once)
Set-ExecutionPolicy -Scope CurrentUser RemoteSigned

# Apply the patches sequentially
powershell -ExecutionPolicy Bypass -File ..\01_Patch_GenerationHandler.ps1
powershell -ExecutionPolicy Bypass -File ..\02_Patch_HFModelCache.ps1

How It Works

Each script:

1. Automatically locates the target file
2. Creates a backup before making changes
3. Patches the code to ensure proper device alignment
4. Provides instructions for reverting changes

Verification

After applying both patches:

• Run your ACE-Step application again
• Check for .BAK backup files created alongside the modified files
• Look for the absence of CUDA/CPU tensor device mismatch errors

Reverting Changes

If you need to revert the changes:

# Use these commands to restore the original files
Copy-Item ..\01_Patch_GenerationHandler.BAK -Destination ..\01_Patch_GenerationHandler.ps1
Copy-Item ..\*.BAK -Destination ..\*. reverted files

Additional Notes

• These scripts are designed for ACE-Step v1.5
• For permanent fixes, clone the original ACE-Step repository
• Scripts will overwrite existing files - always check backups first
• Maintained compatibility with both CUDA and ROCm environments

📜 License

MIT

📝 Notes

• The scripts are designed to work with ACE-Step v1.5
• These are temporary fixes - consider reporting to the ACE-Step team for a permanent solution
• Always test the patched files with your specific ACE-Step version

## Permanent Fix Recommendation

For a more sustainable solution, I recommend:

```markdown
# Permanent Fix: Cloning the ACE-Step Repository

```bash
# Clone the official ACE-Step repository
git clone https://github.com/yourusername/ACE-Step.git

# Apply the patches directly in the original files
# Copy these replacements into your PowerShell editor:

## 1. GenerationHandler Fix

```powershell
# Add device alignment helper after imports
$insertIdx = 0
for ($i = 0; $i -lt $lines.Count; $i++) {
    if ($lines[$i] -match "^import |^from ") { $insertIdx = $i + 1 }
}

$helper = @"
function _align_model_device($model, $device = "cuda") {
    $target_dev = CUDA.FloatTensor(0)
    foreach ($parameter in $model.parameters()) {
        if ($parameter.device() -ne $target_dev) {
            $parameter.data = $parameter.data.to($target_dev)
            if ($parameter.grad_is_none() -ne $true) {
                $parameter.grad = $parameter.grad.to($target_dev)
            }
        }
    }
    return $model.to($target_dev)
}
"@

📌 Conclusion

These scripts provide a quick fix for the device alignment issue in ACE-Step. For long-term solutions, I recommend implementing the permanent fix in the original repository or reporting this issue to the ACE-Step maintainers for a proper fix in future releases.

Would you like me to explain any specific part of the implementation or add any additional information to this GitHub post?

README

PyTorch CUDA Device Alignment Patch for ACE-Step

This repository provides PowerShell scripts to fix device mismatch errors in the ACE-Step PyTorch implementation.

🚨 Problem

Encounter the following error when running ACE-Step with mixed CPU/CUDA tensors:

RuntimeError: Expected all tensors to be on the same device, but got mat2 is on cpu, different from other tensors on cuda:0

💡 Solution

The provided scripts identify and fix the device mismatch by:

1. Ensuring all model parameters are on the correct device
2. Verifying tensor consistency before operations
3. Creating reversible backups

📦 Scripts Included

1. 01_Patch_GenerationHandler.ps1

   • Fixes device alignment in the generation handler

2. 02_Patch_HFModelCache.ps1

   • Patches HuggingFace transformer cache files

⚙️ Prerequisites

1. Windows OS
2. PowerShell 3.0 or newer
3. Python environment with PyTorch installed
4. ACE-Step framework installed

📌 Installation & Usage

# Set execution policy (run once)
Set-ExecutionPolicy -Scope CurrentUser RemoteSigned

# Save this README in your project directory
cd C:\ACE-Step-1.5\
mkdir patches
cd patches

# Copy scripts to your project
Copy-Item "C:\Path\To\This\Repository\*.ps1" -Destination "C:\ACE-Step-1.5\patches\"

# Apply the patches sequentially
powershell -ExecutionPolicy Bypass -File .\01_Patch_GenerationHandler.ps1
powershell -ExecutionPolicy Bypass -File .\02_Patch_HFModelCache.ps1

🔍 How It Works

Each script:

1. Locates the target file automatically
2. Creates a backup with .BAK extension
3. Patches the device alignment code
4. Provides instructions for reverting changes

📜 License

MIT

📝 Notes

• These scripts are designed for ACE-Step v1.5
• For permanent fixes, clone the original ACE-Step repository
• Scripts will overwrite existing files - always check backups first
• Compatibility with ROCm/CUDA is maintained

Would you like me to explain any specific part of the implementation or add any additional information?
ACE-Step Inference Fix SafePatch & BAK Revert.zip
readme.md