You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Chat GPT suggested that I change the LoRA pre-set (to help resolve a problem). Training with its suggestion seems to be 10x faster than the shipped defaults. I have no idea what it will sound like , so maybe it's terrible. Hell I barely know what half the settings do in the LoRA panel.
It looks like its going to do 1000 epoch in about 28 hours, which would take over a week with regular defaults.
rank (r): 128
alpha: 256
dropout: 0.05
lr: 1e-4
I left the rest as they come apart from save every 50 epochs, but it did suggest I might like to try also changing:-
batch_size: 1 (or 2 if stable)
grad_accum: 8 (effective batch 8/16)
epochs: 800
save_every: 50
seed: fixed (e.g. 42)
shift: default for your base (don’t tune first pass)
Two-stage schedule (better)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Chat GPT suggested that I change the LoRA pre-set (to help resolve a problem). Training with its suggestion seems to be 10x faster than the shipped defaults. I have no idea what it will sound like , so maybe it's terrible. Hell I barely know what half the settings do in the LoRA panel.
It looks like its going to do 1000 epoch in about 28 hours, which would take over a week with regular defaults.
rank (r): 128
alpha: 256
dropout: 0.05
lr: 1e-4
I left the rest as they come apart from save every 50 epochs, but it did suggest I might like to try also changing:-
batch_size: 1 (or 2 if stable)
grad_accum: 8 (effective batch 8/16)
epochs: 800
save_every: 50
seed: fixed (e.g. 42)
shift: default for your base (don’t tune first pass)
Two-stage schedule (better)
Epochs 1-500: lr=1e-4
Epochs 501-800: lr=5e-5
Any thoughts from the model experts?
All reactions