GATech at AbjadMed: Bidirectional Encoders vs. Causal Decoders: Insights from 82-Class Arabic Medical Classification

GATech researchers compared bidirectional encoders versus causal decoders for Arabic medical text classification across 82 categories, finding that specialized bidirectional encoders like AraBERTv2 significantly outperform large language models. The study demonstrates that causal decoders optimized for next-token prediction produce sequence-biased embeddings less effective for precise categorization tasks.