Can I combine multiple LoRAs?

Usually, with mixed results. You apply them together at weighted strengths, but they can fight each other, most often when they were trained on overlapping concepts.

Does LoRA work for LLMs, or just image models?

Both. LoRA was originally introduced for large language models and later adopted widely in the image-generation community; the technique itself is architecture-agnostic.

TRAINING TECHNIQUE

What Is a LoRA (Low-Rank Adaptation)?

LoRA (Low-Rank Adaptation) is a fine-tuning technique that freezes a pretrained model's original weights and trains a small set of new weights alongside them. It gets most of what full fine-tuning gets you for a fraction of the compute and storage.

The problem it solves

Full fine-tuning updates every weight in the model. For a modern model that means holding gradients for billions of parameters in GPU memory, and it produces a full new copy of the model for each fine-tune. Without a data-center budget, that stops you before you start.

How LoRA gets around it

Rather than change a weight matrix directly, LoRA freezes it and adds a pair of much smaller matrices next to it. Their product stands in for the update, at a rank far below the original matrix. Only those small matrices train. The original weights stay fixed. A low-rank product holds far fewer numbers than the full matrix, so training fits in a fraction of the memory and time.

At inference you can merge the LoRA into the base weights for zero added latency, or keep it separate and apply it on the fly. Keeping it separate lets one base model switch between many LoRAs, each a small file, with no full model copy per style or skill.

What it's good for

LoRA is strong at teaching a model one style, character, concept, or narrow skill from a small dataset. It is the standard way to add custom styles and characters to image models, and it is common for light LLM fine-tunes: a set tone, a domain vocabulary, a fixed output format.

It is a poor fit for broad new knowledge or a real change in what the model can do. That still needs full fine-tuning, or for new facts, retrieval (see what RAG is) instead of any weight update.

FAQ

Can I combine multiple LoRAs?: Usually, with mixed results. You apply them together at weighted strengths, but they can fight each other, most often when they were trained on overlapping concepts.
Does LoRA work for LLMs, or just image models?: Both. LoRA was originally introduced for large language models and later adopted widely in the image-generation community; the technique itself is architecture-agnostic.

Sources

Hu et al., 'LoRA: Low-Rank Adaptation of Large Language Models' (2021)

Last updated 2026-09-06

What Is a LoRA (Low-Rank Adaptation)?

The problem it solves

How LoRA gets around it

What it's good for

FAQ

Sources

Related