LoRA: Low-Rank Adaptation of Large Language Models

Abstract

This paper introduces and evaluates the idea described in “LoRA: Low-Rank Adaptation of Large Language Models”, and reports empirical results that helped shape subsequent work in finetuning, efficiency.