#79- LoRA and QLoRA.
Life with AI
English - April 11, 2024 11:32 - 13 minutes - 18.9 MB - ★★★★★ - 1 ratingTechnology Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed
Previous Episode: #78- RAFT: Why just to use RAG if you can also fine tune?
Next Episode: #80- Layer pruning and Mixture of Depths.
Hey guys, this is the first episode in a series of episodes about PEFT, Parameter Efficient Fine Tuning. In this episode I talk about LoRA and QLoRA, two widely used methods that allowed us to fine tune LLMs way faster and in a single GPU without losing performance.
Video sobre QLoRA: https://www.youtube.com/watch?v=6l8GZDPbFn8
LoRA paper: https://arxiv.org/pdf/2106.09685.pdf
QLoRA paper: https://arxiv.org/pdf/2305.14314.pdf
Instagram do podcast: https://www.instagram.com/podcast.lifewithai
Linkedin do podcast: https://www.linkedin.com/company/life-with-ai