How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactions

Researchers demonstrate that smaller language models (270M-8B parameters) can match or nearly match the performance of larger models for merchant information extraction in financial transactions through strategic fine-tuning techniques. The study identifies Qwen 3.5 4B as achieving 96.60% F1 score with half the parameters of the baseline LLaMA 3.1-8B model, offering significant cost and latency improvements for production deployment.

This research addresses a critical challenge in financial technology: deploying efficient machine learning models for transaction processing without sacrificing accuracy. The study systematically evaluates 24 model variants to identify the optimal balance between performance and computational efficiency, directly tackling the operational constraints that limit widespread adoption of AI in fintech infrastructure.

The findings reveal a significant scaling efficiency gain. Qwen 3.5 4B achieves 96.60% F1 score—only 0.35 points below the 8B baseline—while using roughly half the parameters. Even more impressively, the 0.8B Qwen model reaches 94.75% F1, matching models 2.5-4x larger. These results suggest that model size is not the primary determinant of performance for structured extraction tasks, challenging conventional assumptions about parameter count requirements.

For the fintech and AI infrastructure sectors, this has substantial implications. Reducing model size from 8B to 4B or smaller dramatically decreases memory requirements, inference latency, and operational costs—critical factors for financial institutions processing millions of transactions daily. The reliable transfer of benchmark performance to production environments (average 0.8-point F1 change) validates that laboratory results translate to real-world deployments.

The research also demonstrates that explicit reasoning supervision through chain-of-thought fine-tuning provides minimal benefit for structured extraction, enabling simpler, more efficient training pipelines. This discovery could reshape how practitioners approach fine-tuning strategies for similar tasks. Going forward, practitioners should prioritize empirical deployment testing and explore whether these efficiency gains apply to adjacent financial processing tasks like fraud detection or compliance categorization.

• →Qwen 3.5 4B achieves 96.60% F1 score with half the parameters of the 8B baseline, enabling cost-effective production deployment.
• →Sub-billion parameter models can match larger models on structured extraction tasks, challenging conventional scaling assumptions.
• →LoRA fine-tuning with reduced rank (8 vs 32) maintains performance while further improving efficiency.
• →Chain-of-thought reasoning provides minimal benefits for structured extraction, simplifying training approaches.
• →Benchmark performance reliably transfers to production with average 0.8-point F1 degradation across sub-8B models.