Evaluating Fine-Tuned LLM Model For Medical Transcription With Small Low-Resource Languages Validated Dataset

Researchers successfully fine-tuned LLaMA 3.1-8B for medical transcription in Finnish, a low-resource language, achieving strong semantic similarity despite low n-gram overlap. The study used simulated clinical conversations from students and demonstrates the feasibility of privacy-oriented domain-specific language models for clinical documentation in underrepresented languages.

• →Fine-tuning LLaMA 3.1-8B on a small Finnish medical dataset achieved BLEU = 0.1214, ROUGE-L = 0.4982, and BERTScore F1 = 0.8230.
• →The model showed low n-gram overlap but strong semantic similarity with reference transcripts, indicating effective understanding of medical context.
• →This research addresses physician burnout by potentially reducing administrative burden in electronic health records for Finnish healthcare.
• →The study validates that domain-specific fine-tuning can work effectively even with small datasets in low-resource languages.
• →Results support the development of privacy-oriented medical AI systems that can operate locally without sending sensitive data to external services.