Fashion Florence: Fine-Tuning Florence-2 for Structured Fashion Attribute Extraction

Researchers have fine-tuned Florence-2, a vision-language model, to extract structured fashion attributes from clothing images with 94.6% category accuracy. The resulting model, Fashion Florence, outperforms GPT-4o-mini and Gemini 2.5 Flash on fashion-specific tasks while running efficiently at 0.77B parameters, demonstrating specialized AI models can exceed general-purpose alternatives in narrow domains.

Fashion Florence represents a pragmatic application of parameter-efficient fine-tuning to solve a real e-commerce problem. By using LoRA (Low-Rank Adaptation) on Florence-2, the researchers achieved superior performance on fashion attribute extraction compared to significantly larger foundation models, suggesting that domain-specific optimization outweighs raw model scale for specialized classification tasks. The 94.6% category accuracy and 63% material accuracy metrics, paired with 99.8% valid JSON output reliability, indicate production-ready performance for retail systems.

This work builds on the broader trend of fine-tuning open-source vision-language models rather than relying exclusively on proprietary APIs. The iMaterialist Fashion dataset provided sufficient training data (3,688 examples) to achieve meaningful improvements through strategic label engineering, collapsing 228 labels into interpretable schema. Fashion Florence's integration into Loom, an open-source recommendation system, demonstrates immediate commercial applicability.

For e-commerce developers and recommendation system builders, this model offers a cost-efficient alternative to API-dependent solutions like GPT-4o-mini. The zero marginal inference cost on a single GPU contrasts sharply with per-request pricing of proprietary models, enabling scalable deployment across large product catalogs. The 99.8% JSON validity rate eliminates downstream parsing errors that plague vision-language output in production.

Future developments should focus on expanding the schema to handle emerging fashion categories and regional style variations. Potential improvements include multi-image attribute consistency checks and confidence scoring for uncertain classifications, advancing toward fully autonomous fashion catalog curation.

• →Fashion Florence achieves 94.6% category accuracy on fashion attribute extraction, outperforming GPT-4o-mini (89.3%) and Gemini 2.5 Flash (87.4%).
• →LoRA fine-tuning on Florence-2 enables efficient domain-specific optimization using only 3,688 training examples from iMaterialist Fashion dataset.
• →The model produces valid JSON output in 99.8% of cases, ensuring reliable integration with downstream e-commerce and recommendation systems.
• →Running at 0.77B parameters on a single GPU with zero marginal inference cost offers significant advantages over proprietary API-dependent alternatives.
• →Deployment as a Hugging Face Space and integration into Loom demonstrates immediate practical applicability for open-source fashion recommendation systems.