#lora-finetuning News & Analysis

3 articles tagged with #lora-finetuning. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBullisharXiv – CS AI · May 47/10

🧠

RadLite: Multi-Task LoRA Fine-Tuning of Small Language Models for CPU-Deployable Radiology AI

Researchers demonstrate that small language models (3-4B parameters) can achieve strong multi-task radiology performance through LoRA fine-tuning, enabling deployment on consumer-grade CPUs without GPUs. The RadLite system, trained on 162K samples across 9 radiology tasks, shows dramatic performance improvements over zero-shot baselines and can be quantized to 1.8-2.4GB for practical clinical deployment.

AIBullisharXiv – CS AI · Jun 196/10

🧠

Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices

Researchers introduce memory optimization techniques for fine-tuning Large Language Models using LoRA on resource-constrained devices, achieving up to 28× peak memory reduction through quantization, efficient checkpointing, and token approximation methods. The work enables private model personalization on consumer hardware without compromising model quality.

🧠 Llama

AINeutralarXiv – CS AI · May 296/10

🧠

How LoRA Remembers? A Parametric Memory Law for LLM Finetuning

Researchers introduce the Parametric Memory Law, a power law framework quantifying how Large Language Models store information through Low-Rank Adaptation (LoRA) finetuning. The study reveals a deterministic phase transition at the token level and proposes MemFT, an optimization strategy that improves memory fidelity by dynamically redistributing training resources toward undertrained tokens.