My fine-tuning run kept crashing until I set the gradient accumulation steps to 4

It was failing on a 12GB VRAM card after about 20 minutes every single time. Anyone know other tricks for memory issues with smaller models?

2 comments

2 Comments

Try lowering your batch size to one.

Honestly, lowering the batch size just makes things slower for me.