What is LoRA (Low-Rank Adaptation)?

LoRA freezes the original model weights and trains tiny low-rank matrices alongside them, cutting fine-tuning cost and memory by orders of magnitude. The resulting adapters are megabytes instead of gigabytes and can be swapped at runtime — the dominant technique for customizing open-weight models and image models alike.