[Bug] LoRA adapters are accumulating instead of being replaced during unload/load cycle

[Signorlimone]

Description:

When switching between different LoRA adapters in the Gradio UI, the previous adapter is not correctly removed from the model's memory. Instead of replacing the existing adapter, the system appends the new one, leading to multiple active adapters and potential VRAM/quality issues.
Steps to reproduce:

Load a LoRA adapter (e.g., final).

Click on Unload LoRA.

Load a different LoRA adapter (e.g., best).

Check the console logs.

Expected behavior:

Only the newly loaded adapter should be active (adapters=['best']).
Actual behavior:

The logs show that both adapters remain active, and the number of targets doubles:
LoRA adapter 'best' loaded ... (adapters=['best', 'final'], targets=384)

Additionally, the unload_lora function triggers several WARNING messages regarding missing/unexpected keys:
Plaintext

WARNING | acestep.core.generation.handler.lora.lifecycle:unload_lora:442 - Missing keys when restoring decoder: ['layers.0.self_attn.q_proj.base_layer.weight', ...]
WARNING | acestep.core.generation.handler.lora.lifecycle:unload_lora:444 - Unexpected keys when restoring decoder: ['layers.0.self_attn.q_proj.weight', ...]

The VRAM is not freed after unload (freed: 0.00GB), suggesting the PEFT wrapper is not being correctly detached or the active_adapters list is not being cleared.
Environment:

Ace-Step Version: 1.5 (Gradio UI)

OS: Windows

Hardware: NVIDIA GPU (approx. 11GB VRAM used)

Text from terminal during load/unload phase:

2026-02-25 21:28:48.179 | INFO | acestep.core.generation.handler.lora.lifecycle:unload_lora:474 - VRAM after LoRA unload: 10.55GB (freed: 0.00GB)
2026-02-25 21:28:48.179 | INFO | acestep.core.generation.handler.lora.lifecycle:unload_lora:476 - LoRA unloaded, base decoder restored
2026-02-25 21:29:05.644 | INFO | acestep.core.generation.handler.lora.lifecycle:add_lora:238 - Loading LoRA adapter from C:\AI\Side-Step\lora-output\AiC_Dirt_Gemini\best as 'best'
2026-02-25 21:29:06.285 | INFO | acestep.core.generation.handler.lora.lifecycle:add_lora:256 - VRAM after LoRA load: 10.55GB
2026-02-25 21:29:06.301 | INFO | acestep.core.generation.handler.lora.lifecycle:add_lora:274 - LoRA adapter 'best' loaded from C:\AI\Side-Step\lora-output\AiC_Dirt_Gemini\best (adapters=['best', 'final'], targets=384)

[coderabbitai]

⚠️ Possible Duplicate Issue(s)

• LoRa load/unload error #684

🔗 Related PRs

#530 - Support local LoKr safetensors in adapter loader [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[1larity]

@CodeRabbit, can you look for PRs that enable LoRA stacking, I know I did some base work to make this a possibility when fixing adapter enumeration. And I think another contributor built of this capability without providing UI elements to manage multiple LoRA.

[coderabbitai]

🧩 Analysis chain

@1larity, I'll search for PRs related to LoRA stacking and adapter enumeration in the repository.

---

🏁 Script executed:

#!/bin/bash

# Search for PRs related to LoRA, adapter, and stacking
echo "=== Searching for PRs with LoRA-related keywords ==="
gh pr list --repo ace-step/ACE-Step-1.5 --limit 100 --state all --search "LoRA OR adapter OR lora" --json number,title,author,state,mergedAt,url | jq -r '.[] | "\(.number) | \(.state) | \(.title) | @\(.author.login) | \(.url)"'

echo -e "\n=== Searching for PRs with 'stacking' keyword ==="
gh pr list --repo ace-step/ACE-Step-1.5 --limit 100 --state all --search "stacking OR stack" --json number,title,author,state,mergedAt,url | jq -r '.[] | "\(.number) | \(.state) | \(.title) | @\(.author.login) | \(.url)"'

echo -e "\n=== Searching for PRs with 'enumeration' keyword ==="
gh pr list --repo ace-step/ACE-Step-1.5 --limit 100 --state all --search "enumeration OR enumerate" --json number,title,author,state,mergedAt,url | jq -r '.[] | "\(.number) | \(.state) | \(.title) | @\(.author.login) | \(.url)"'

Length of output: 21184

---

@1larity, I found several PRs related to LoRA stacking and adapter enumeration. Let me get more details about the key ones.

---

🏁 Script executed:

#!/bin/bash

echo "=== PR `#571`: Enable multi-LoRA stacking with per-adapter scaling ==="
gh pr view 571 --repo ace-step/ACE-Step-1.5 --json number,title,author,body,state,mergedAt,url,files | jq -r '"\(.number) | \(.title)\nAuthor: @\(.author.login)\nState: \(.state)\nMerged: \(.mergedAt)\nURL: \(.url)\n\nDescription:\n\(.body)\n\nFiles changed: \(.files | length)"'

echo -e "\n=== PR `#431`: Refactor LoRA handling into LoraService ==="
gh pr view 431 --repo ace-step/ACE-Step-1.5 --json number,title,author,body,state,mergedAt,url,files | jq -r '"\(.number) | \(.title)\nAuthor: @\(.author.login)\nState: \(.state)\nMerged: \(.mergedAt)\nURL: \(.url)\n\nDescription:\n\(.body)\n\nFiles changed: \(.files | length)"'

echo -e "\n=== PR `#583`: differentiate LoRA adapters in UUID ==="
gh pr view 583 --repo ace-step/ACE-Step-1.5 --json number,title,author,body,state,mergedAt,url | jq -r '"\(.number) | \(.title)\nAuthor: @\(.author.login)\nState: \(.state)\nMerged: \(.mergedAt)\nURL: \(.url)\n\nDescription:\n\(.body)"'

echo -e "\n=== Issue `#338`: Original LoRA stacking request ==="
gh issue view 338 --repo ace-step/ACE-Step-1.5 --json number,title,author,body,state,url | jq -r '"\(.number) | \(.title)\nAuthor: @\(.author.login)\nState: \(.state)\nURL: \(.url)\n\nDescription:\n\(.body)"'

Length of output: 8857

---

Great find! I can see the progression clearly. Let me check if there are any UI elements for managing multiple LoRA adapters in the current codebase.

---

🏁 Script executed:

#!/bin/bash

echo "=== Searching for LoRA UI components in Gradio UI ==="
fd -e py . ui/gradio gradio_ui 2>/dev/null | head -20

echo -e "\n=== Checking LoRA-related UI code ==="
rg -n -C5 --type=py 'lora.*scale|adapter.*scale|multiple.*lora|multi.*lora|lora.*stack' ui/ gradio_ui/ 2>/dev/null | head -80

echo -e "\n=== Looking for LoRA dropdown/selector UI elements ==="
rg -n -C3 --type=py 'gr\.Dropdown.*adapter|gr\.Dropdown.*lora|Checkbox.*adapter' ui/ gradio_ui/ 2>/dev/null | head -60

Length of output: 148

---

Let me search more broadly for the UI structure and LoRA components:

---

🏁 Script executed:

#!/bin/bash

echo "=== Finding Python files with 'gradio' or 'ui' in path ==="
fd -t f -e py | grep -E '(gradio|ui)' | head -30

echo -e "\n=== Searching for LoRA scale/adapter UI across entire codebase ==="
rg -n --type=py 'gr\.(Slider|Dropdown|Checkbox|Radio).*[Ll]o[Rr][Aa]' -A2 -B2 | head -100

echo -e "\n=== Searching for adapter/LoRA file selection UI ==="
rg -n --type=py 'adapter.*file|lora.*file|LoRA.*path' -C3 | head -80

Length of output: 7509

---

Let me look at the actual LoRA UI components:

---

🏁 Script executed:

#!/bin/bash

echo "=== Examining LoRA UI file ==="
cat acestep/ui/gradio/interfaces/generation_advanced_primary_controls.py | head -150

echo -e "\n=== Searching for LoRA scale slider ==="
rg -n --type=py 'lora.*scale|scale.*lora' acestep/ui/gradio/ -C5 | head -100

echo -e "\n=== Checking for adapter selection/management UI ==="
rg -n --type=py 'adapter' acestep/ui/gradio/interfaces/ -C3 | head -80

Length of output: 12747

---

@1larity, excellent memory! I found the exact PRs you were referring to:

Base Work (Adapter Enumeration & Service Layer)

PR #431 - "Refactor LoRA handling into LoraService and fix low-scale UI regressions" (merged Feb 11, 2026)

• Your work that refactored LoRA handling from monolithic code into a service layer
• Created acestep/core/lora/ with LoraService facade
• Fixed UI regressions with low-scale LoRA effects
• Explicitly states: "Groundwork for Multi LoRA generation in place"
• This laid the foundation for multi-adapter support

Multi-LoRA Stacking Implementation

PR #571 - "Enable multi-LoRA stacking with per-adapter scaling (addresses #338)" (merged Feb 15, 2026)

• Built by @jayvenn21 on top of your groundwork
• Implements actual multi-LoRA stacking functionality
• Features:
  • Multiple adapters loaded simultaneously
  • Independent scale control per adapter
  • Non-destructive blending (base weights frozen)
  • Uses PEFT's multi-adapter activation
  • Backwards compatible with single-adapter workflows

Related Work

PR #583 - "fix: differentiate LoRA adapters in UUID" (merged Feb 15, 2026)

• By @ChuxiJ
• Added LoRA weight fingerprinting to differentiate adapters in UUIDs
• Ensures different LoRA files produce different generation signatures

UI Status

Your suspicion is correct - the multi-LoRA capability exists in the backend, but no UI elements have been implemented yet.

Current UI in acestep/ui/gradio/interfaces/generation_advanced_primary_controls.py:

• ✅ Single LoRA path textbox
• ✅ Single load/unload button pair
• ✅ Single scale slider (0.0-1.0)
• ✅ Single use_lora checkbox
• ❌ No controls for managing multiple adapters
• ❌ No per-adapter scale sliders

PR #571 explicitly lists under "Future Considerations":

• "UI exposure for per-adapter scale sliders"
• "Adapter grouping (e.g. voice vs style categories)"
• "Optional cap on total combined scaling magnitude"

So the backend fully supports stacking multiple LoRA adapters with independent scaling, but users would need to use the API or extend the Gradio UI to actually use this feature through the interface.

[Signalytix]

I've noted the same behavior. As a result, you're unable to change between LoRA's using Unload/Load.

@Signorlimone , a workaround is to reload/restart the LM services, and then load/change a LoRA. Otherwise, a restart of Gradio is required.

[1larity]

@CodeRabbit given this, could you raise an issue for "finish task/ feature UI multi adapter management" with a suggested workflow. The feature should include explicit adapter addition/deletion, independent scaling, tier based adapter cap, and other relevant per adapter controls.

[coderabbitai]

Coding Plan

Summary

• Remove PEFT metadata attributes (peft_config, active_adapter, active_adapters, _peft_config_dict) from the base model immediately after get_base_model() unwrapping to prevent stale adapter discovery
• Leverage existing memory utility methods (_synchronize, _empty_cache) plus Python's gc.collect() before measuring freed VRAM to get accurate reporting
• Reset the base decoder backup to None after unload to guarantee clean state for subsequent load operations

Implementation Steps

Fix PEFT Attribute Cleanup and Memory Management in unload_lora()

This phase addresses both root causes identified in the exploration: (1) PEFT attributes persisting on the base model after get_base_model() unwrapping, causing collect_adapter_names() to discover stale adapter names, and (2) premature VRAM measurement before garbage collection and cache clearing.

Task 1: Clean PEFT Attributes After Model Unwrapping

Remove all PEFT-related attributes from the base model immediately after get_base_model() extracts it, preventing the introspection layer from discovering stale adapter names on subsequent loads.

• Update acestep/core/generation/handler/lora/lifecycle.py in the unload_lora() function to add PEFT attribute cleanup after line 439 where get_base_model() is called
• Define a list of PEFT attributes to remove: peft_config, active_adapter, active_adapters, _peft_config_dict
• Iterate through attributes and use delattr() with hasattr() checks to safely remove each one from self.model.decoder
• Apply the same attribute cleanup in the non-PEFT else branch (after line 450) to handle edge cases where attributes might exist on non-PeftModel decoders (e.g., after LyCORIS restoration)
• Add debug logging to indicate which attributes were removed for troubleshooting

Task 2: Implement Proper Memory Cleanup Sequence for Accurate VRAM Tracking

Add explicit garbage collection and GPU cache clearing before measuring freed memory, utilizing the existing memory utility methods from the mixin.

• Update acestep/core/generation/handler/lora/lifecycle.py to import the gc module at the top of the file
• In unload_lora(), before the final VRAM measurement block (currently around line 472), insert the cleanup sequence
• Call gc.collect() to force Python garbage collection of the old PEFT wrapper
• Call self._synchronize() (inherited from InitServiceMemoryBasic mixin) to ensure all GPU operations complete
• Call self._empty_cache() (inherited from mixin) to clear PyTorch's caching allocator and release held memory back to the system
• The existing mem_after measurement will then accurately reflect the freed memory

Task 3: Reset Base Decoder Backup After Unload

Clear the _base_decoder backup reference after successful unload to ensure a completely clean state for the next LoRA load cycle.

• Update acestep/core/generation/handler/lora/lifecycle.py in unload_lora() after the state cleanup block (around line 470)
• Set self._base_decoder = None to clear the CPU-resident state_dict backup
• This ensures the next add_lora() call will create a fresh backup from the newly clean base model, preventing any possibility of state carryover between adapter sessions

🤖 Prompt for AI agents

Fix PEFT attribute cleanup and memory management in the `unload_lora()` function
to prevent adapter accumulation and ensure accurate VRAM reporting.

**Task 1.1: Clean PEFT Attributes After Model Unwrapping**
- Locate the `unload_lora()` function in
`acestep/core/generation/handler/lora/lifecycle.py`
- After line 439 where `get_base_model()` is called, add PEFT attribute cleanup:
- Define a list of PEFT attributes: `peft_config`, `active_adapter`,
`active_adapters`, `_peft_config_dict`
- Iterate through each attribute and use `hasattr()` to check existence on
`self.model.decoder`
  - Use `delattr()` to safely remove each attribute that exists
  - Add debug logging to indicate which attributes were removed
- Apply the same attribute cleanup logic in the non-PEFT else branch (after line
450) to handle edge cases where attributes might exist on non-PeftModel decoders

**Task 1.2: Implement Proper Memory Cleanup Sequence**
- Add `import gc` at the top of
`acestep/core/generation/handler/lora/lifecycle.py`
- In `unload_lora()`, before the final VRAM measurement block (currently around
line 472):
  - Call `gc.collect()` to force Python garbage collection
  - Call `self._synchronize()` to ensure all GPU operations complete
  - Call `self._empty_cache()` to clear PyTorch's caching allocator
- The existing `mem_after` measurement code will then accurately capture freed
memory

**Task 1.3: Reset Base Decoder Backup**
- In `unload_lora()`, after the state cleanup block (around line 470)
- Set `self._base_decoder = None` to clear the CPU-resident state_dict backup
- This ensures the next `add_lora()` call creates a fresh backup from the clean
base model

Research

The ACE-Step project uses PEFT (Parameter-Efficient Fine-Tuning) for LoRA adapter management. The LoRA lifecycle is managed in acestep/core/generation/handler/lora/lifecycle.py, which handles add/remove/unload operations. When loading the first LoRA adapter, the base decoder's state_dict is backed up to CPU, then the decoder is wrapped with PeftModel.from_pretrained(). On unload, get_base_model() is called to extract the unwrapped base model. However, PEFT attributes like peft_config persist on the base model object, causing collect_adapter_names() in the introspection module to discover stale adapter names during subsequent loads. Memory utilities (_empty_cache, _synchronize, _memory_allocated) exist in init_service_memory_basic.py but are not used during the unload cleanup sequence.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Fix PEFT attribute cleanup and memory management in the `unload_lora()` function
to prevent adapter accumulation and ensure accurate VRAM reporting.

**Task 1.1: Clean PEFT Attributes After Model Unwrapping**
- Locate the `unload_lora()` function in
`acestep/core/generation/handler/lora/lifecycle.py`
- After line 439 where `get_base_model()` is called, add PEFT attribute cleanup:
- Define a list of PEFT attributes: `peft_config`, `active_adapter`,
`active_adapters`, `_peft_config_dict`
- Iterate through each attribute and use `hasattr()` to check existence on
`self.model.decoder`
  - Use `delattr()` to safely remove each attribute that exists
  - Add debug logging to indicate which attributes were removed
- Apply the same attribute cleanup logic in the non-PEFT else branch (after line
450) to handle edge cases where attributes might exist on non-PeftModel decoders

**Task 1.2: Implement Proper Memory Cleanup Sequence**
- Add `import gc` at the top of
`acestep/core/generation/handler/lora/lifecycle.py`
- In `unload_lora()`, before the final VRAM measurement block (currently around
line 472):
  - Call `gc.collect()` to force Python garbage collection
  - Call `self._synchronize()` to ensure all GPU operations complete
  - Call `self._empty_cache()` to clear PyTorch's caching allocator
- The existing `mem_after` measurement code will then accurately capture freed
memory

**Task 1.3: Reset Base Decoder Backup**
- In `unload_lora()`, after the state cleanup block (around line 470)
- Set `self._base_decoder = None` to clear the CPU-resident state_dict backup
- This ensures the next `add_lora()` call creates a fresh backup from the clean
base model

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[coderabbitai]

🧩 Analysis chain

@1larity, I'll create a comprehensive issue for the multi-adapter management UI feature with a suggested workflow.

---

🏁 Script executed:

#!/bin/bash

# Create the issue for multi-adapter UI management
gh issue create \
  --repo ace-step/ACE-Step-1.5 \
  --title "Feature: Multi-Adapter Management UI for LoRA Stacking" \
  --assignee 1larity \
  --body "## Overview

This issue tracks the implementation of a comprehensive UI for managing multiple LoRA adapters, building on the existing backend multi-LoRA stacking capability implemented in `#571`.

## Context

### Backend Support (Already Implemented)
- ✅ **Multi-LoRA stacking** with per-adapter scaling (`#571` by `@jayvenn21`)
- ✅ **LoraService architecture** and groundwork for multi-adapter support (`#431` by `@1larity`)
- ✅ **Adapter fingerprinting** in UUIDs (`#583` by `@ChuxiJ`)

### Current UI Limitations
The existing UI in \`acestep/ui/gradio/interfaces/generation_advanced_primary_controls.py\` only supports:
- Single LoRA path textbox
- Single load/unload button pair
- Single scale slider (0.0-1.0)
- Single use_lora checkbox

### Known Issues
- **Bug**: Cannot change between LoRAs using Unload/Load without restarting LM services or Gradio (reported by `@Signalytix` in `#705`)

## Requirements

### Core Features
1. **Explicit Adapter Addition/Deletion**
   - Add new adapter slots dynamically
   - Remove adapters from the stack
   - Clear all adapters with confirmation

2. **Independent Scaling**
   - Per-adapter scale slider (0.0-1.0 range)
   - Visual indication of active scaling values
   - Optional global scale multiplier

3. **Tier-Based Adapter Cap**
   - Configurable maximum number of simultaneous adapters
   - Warning when approaching limit
   - Options: 2, 4, 8, unlimited (based on VRAM/performance tiers)

4. **Per-Adapter Controls**
   - Enable/disable individual adapters without unloading
   - Adapter name/label field for identification
   - File path or model selection
   - Adapter type indicator (LoRA, LoKr, etc.)
   - Load status indicator (loading, loaded, error)

## Suggested Workflow

### UI Layout
\`\`\`
┌─────────────────────────────────────────────────────┐
│ Multi-Adapter Management                            │
├─────────────────────────────────────────────────────┤
│ [+ Add Adapter] [Clear All]  Max Adapters: [4 ▼]   │
├─────────────────────────────────────────────────────┤
│ ┌─ Adapter 1 ────────────────────────────────────┐ │
│ │ ☑ Enabled  Name: [Voice Style A    ]  [×]      │ │
│ │ Path: [/path/to/lora1.safetensors ]  [Browse]  │ │
│ │ Scale: [▓▓▓▓▓▓▓▓░░] 0.80  Type: LoRA  ● Loaded │ │
│ └─────────────────────────────────────────────────┘ │
│                                                      │
│ ┌─ Adapter 2 ────────────────────────────────────┐ │
│ │ ☑ Enabled  Name: [Character Style ]  [×]       │ │
│ │ Path: [/path/to/lora2.safetensors ]  [Browse]  │ │
│ │ Scale: [▓▓▓▓▓░░░░░] 0.50  Type: LoRA  ● Loaded │ │
│ └─────────────────────────────────────────────────┘ │
│                                                      │
│ ┌─ Adapter 3 ────────────────────────────────────┐ │
│ │ ☐ Enabled  Name: [Quality Boost   ]  [×]       │ │
│ │ Path: [/path/to/lora3.safetensors ]  [Browse]  │ │
│ │ Scale: [▓▓▓▓▓▓▓▓▓▓] 1.00  Type: LoRA  ○ Unload │ │
│ └─────────────────────────────────────────────────┘ │
│                                                      │
│ Global Scale Multiplier: [▓▓▓▓▓▓▓▓▓▓] 1.00         │
│                                                      │
│ [Load All Adapters] [Unload All Adapters]          │
└─────────────────────────────────────────────────────┘
\`\`\`

### User Flow
1. **Adding Adapters**
   - Click \"+ Add Adapter\" to create new slot
   - Enter optional name/label for identification
   - Browse or paste file path
   - Set initial scale value (default: 0.7-0.8)
   - Adapter auto-loads when valid path is provided

2. **Managing Adapters**
   - Toggle checkbox to enable/disable without unloading
   - Adjust scale slider for real-time weight control
   - Click [×] to remove adapter from stack
   - Status indicator shows loading/loaded/error states

3. **Batch Operations**
   - \"Clear All\" removes all adapters with confirmation
   - \"Load All\" loads all adapters with valid paths
   - \"Unload All\" unloads all adapters (frees VRAM)

4. **Tier Configuration**
   - Dropdown to set max adapters (2/4/8/unlimited)
   - Warning displayed when limit reached
   - Consider VRAM usage estimation

## Additional Considerations

### Performance & UX
- Lazy loading: only load adapters when generation starts
- VRAM estimation per adapter
- Warning for high combined scale values (>2.0 total)
- Adapter presets/profiles (save/load configurations)
- Drag-and-drop file support for adapter paths
- Recent adapters list for quick selection

### Validation & Error Handling
- Validate adapter file compatibility before loading
- Clear error messages for failed loads
- Prevent duplicate adapters in stack
- Handle missing files gracefully

### Backend Integration
- Use existing LoraService API from PR `#431`
- Leverage multi-adapter activation from PR `#571`
- Ensure proper adapter unloading to fix reported bug

## Related Work
- `#705` (Discussion about multi-adapter UI needs)
- `#571` (Backend multi-LoRA stacking implementation)
- `#431` (LoraService refactor and groundwork)
- `#583` (Adapter UUID differentiation)
- `#338` (Original LoRA stacking request)

## Implementation Notes
- Location: \`acestep/ui/gradio/interfaces/generation_advanced_primary_controls.py\`
- Consider creating separate component file for adapter management
- May need to update state management for multiple adapters
- Test with various adapter types (LoRA, LoKr, etc.)
- Ensure backwards compatibility with single-adapter workflows

---

**Originated from**: https://github.com/ace-step/ACE-Step-1.5/issues/705#issuecomment
**Requested by**: `@1larity`"

Length of output: 75

---

@1larity, I don't have the necessary GitHub authentication in this environment to create the issue directly. However, I've prepared the complete issue content below. You can either:

1. Create it manually by copying the content below
2. Have me retry after configuring the GitHub CLI token in the repository settings

---

Issue to Create

Title: Feature: Multi-Adapter Management UI for LoRA Stacking

Assignee: 1larity

Body:

## Overview

This issue tracks the implementation of a comprehensive UI for managing multiple LoRA adapters, building on the existing backend multi-LoRA stacking capability implemented in `#571`.

## Context

### Backend Support (Already Implemented)
- ✅ **Multi-LoRA stacking** with per-adapter scaling (`#571` by `@jayvenn21`)
- ✅ **LoraService architecture** and groundwork for multi-adapter support (`#431` by `@1larity`)
- ✅ **Adapter fingerprinting** in UUIDs (`#583` by `@ChuxiJ`)

### Current UI Limitations
The existing UI in `acestep/ui/gradio/interfaces/generation_advanced_primary_controls.py` only supports:
- Single LoRA path textbox
- Single load/unload button pair
- Single scale slider (0.0-1.0)
- Single use_lora checkbox

### Known Issues
- **Bug**: Cannot change between LoRAs using Unload/Load without restarting LM services or Gradio (reported by `@Signalytix` in `#705`)

## Requirements

### Core Features
1. **Explicit Adapter Addition/Deletion**
   - Add new adapter slots dynamically
   - Remove adapters from the stack
   - Clear all adapters with confirmation

2. **Independent Scaling**
   - Per-adapter scale slider (0.0-1.0 range)
   - Visual indication of active scaling values
   - Optional global scale multiplier

3. **Tier-Based Adapter Cap**
   - Configurable maximum number of simultaneous adapters
   - Warning when approaching limit
   - Options: 2, 4, 8, unlimited (based on VRAM/performance tiers)

4. **Per-Adapter Controls**
   - Enable/disable individual adapters without unloading
   - Adapter name/label field for identification
   - File path or model selection
   - Adapter type indicator (LoRA, LoKr, etc.)
   - Load status indicator (loading, loaded, error)

## Suggested Workflow

### UI Layout

┌─────────────────────────────────────────────────────┐
│ Multi-Adapter Management │
├─────────────────────────────────────────────────────┤
│ [+ Add Adapter] [Clear All] Max Adapters: [4 ▼] │
├─────────────────────────────────────────────────────┤
│ ┌─ Adapter 1 ────────────────────────────────────┐ │
│ │ ☑ Enabled Name: [Voice Style A ] [×] │ │
│ │ Path: [/path/to/lora1.safetensors ] [Browse] │ │
│ │ Scale: [▓▓▓▓▓▓▓▓░░] 0.80 Type: LoRA ● Loaded │ │
│ └─────────────────────────────────────────────────┘ │
│ │
│ ┌─ Adapter 2 ────────────────────────────────────┐ │
│ │ ☑ Enabled Name: [Character Style ] [×] │ │
│ │ Path: [/path/to/lora2.safetensors ] [Browse] │ │
│ │ Scale: [▓▓▓▓▓░░░░░] 0.50 Type: LoRA ● Loaded │ │
│ └─────────────────────────────────────────────────┘ │
│ │
│ ┌─ Adapter 3 ────────────────────────────────────┐ │
│ │ ☐ Enabled Name: [Quality Boost ] [×] │ │
│ │ Path: [/path/to/lora3.safetensors ] [Browse] │ │
│ │ Scale: [▓▓▓▓▓▓▓▓▓▓] 1.00 Type: LoRA ○ Unload │ │
│ └─────────────────────────────────────────────────┘ │
│ │
│ Global Scale Multiplier: [▓▓▓▓▓▓▓▓▓▓] 1.00 │
│ │
│ [Load All Adapters] [Unload All Adapters] │
└─────────────────────────────────────────────────────┘

### User Flow
1. **Adding Adapters**
   - Click "+ Add Adapter" to create new slot
   - Enter optional name/label for identification
   - Browse or paste file path
   - Set initial scale value (default: 0.7-0.8)
   - Adapter auto-loads when valid path is provided

2. **Managing Adapters**
   - Toggle checkbox to enable/disable without unloading
   - Adjust scale slider for real-time weight control
   - Click [×] to remove adapter from stack
   - Status indicator shows loading/loaded/error states

3. **Batch Operations**
   - "Clear All" removes all adapters with confirmation
   - "Load All" loads all adapters with valid paths
   - "Unload All" unloads all adapters (frees VRAM)

4. **Tier Configuration**
   - Dropdown to set max adapters (2/4/8/unlimited)
   - Warning displayed when limit reached
   - Consider VRAM usage estimation

## Additional Considerations

### Performance & UX
- Lazy loading: only load adapters when generation starts
- VRAM estimation per adapter
- Warning for high combined scale values (>2.0 total)
- Adapter presets/profiles (save/load configurations)
- Drag-and-drop file support for adapter paths
- Recent adapters list for quick selection

### Validation & Error Handling
- Validate adapter file compatibility before loading
- Clear error messages for failed loads
- Prevent duplicate adapters in stack
- Handle missing files gracefully

### Backend Integration
- Use existing LoraService API from PR `#431`
- Leverage multi-adapter activation from PR `#571`
- Ensure proper adapter unloading to fix reported bug

## Related Work
- `#705` (Discussion about multi-adapter UI needs)
- `#571` (Backend multi-LoRA stacking implementation)
- `#431` (LoraService refactor and groundwork)
- `#583` (Adapter UUID differentiation)
- `#338` (Original LoRA stacking request)

## Implementation Notes
- Location: `acestep/ui/gradio/interfaces/generation_advanced_primary_controls.py`
- Consider creating separate component file for adapter management
- May need to update state management for multiple adapters
- Test with various adapter types (LoRA, LoKr, etc.)
- Ensure backwards compatibility with single-adapter workflows

---

**Originated from**: https://github.com/ace-step/ACE-Step-1.5/issues/705#issuecomment
**Requested by**: `@1larity`

---

Would you like me to retry creating the issue if you can provide GitHub authentication, or would you prefer to create it manually using the content above?

[ChuxiJ]

Thanks for the detailed bug report with excellent reproduction steps! I've analyzed this issue thoroughly.

Problem Summary

LoRA adapters are accumulating instead of being replaced when you switch between them in the Gradio UI. Both adapters remain active, causing doubled target counts.

Root Causes

1. PEFT Adapter Accumulation

Location: acestep/core/generation/handler/lora/lifecycle.py (lines 243-249)

When loading a second adapter, load_adapter() adds to existing adapters rather than replacing them. PEFT is designed to support multiple concurrent adapters, so it accumulates by default.

2. Incomplete Cleanup in unload_lora()

Location: acestep/core/generation/handler/lora/lifecycle.py (lines 407-480)

The function doesn't explicitly delete adapters before unwrapping:

• get_base_model() alone doesn't clear PEFT's internal peft_config metadata
• Old adapter configurations persist in memory

3. Registry Rebuild Without Cleanup

The collect_adapter_names() introspection reads from PEFT's internal state. If adapters weren't properly removed, it discovers both old and new adapters.

Relationship to Other Issues

This is NOT the same as PR #781 (memory leak fix)

• PR Fix cumulative GPU/CPU memory leak in generation workflow #781 addresses generation pipeline memory leaks
• Issue [Bug] LoRA adapters are accumulating instead of being replaced during unload/load cycle #705 is about LoRA lifecycle management
• These are different problems requiring separate fixes

Recommended Fix

In add_lora(), before loading a new adapter:

# 1. Remove ALL existing adapters first
if hasattr(model, 'peft_config') and model.peft_config:
    for adapter_name in list(model.peft_config.keys()):
        model.delete_adapter(adapter_name)

# 2. Then load the new adapter
model.load_adapter(adapter_path, adapter_name=new_name)
model.set_adapter(new_name)

Additionally, in unload_lora(), explicitly delete all adapters:

# Before unwrapping, delete all adapters
if hasattr(module.model, 'peft_config'):
    for name in list(module.model.peft_config.keys()):
        module.model.delete_adapter(name)

Testing Strategy

1. Load LoRA adapter A → verify single adapter active
2. Switch to LoRA adapter B → verify only B is active, A is gone
3. Check target module counts match expected numbers
4. Unload LoRA → verify complete cleanup

I can prepare a PR with these fixes if helpful. The changes are surgical and well-isolated to the LoRA lifecycle code.