🧠 AI🟢 BullishImportance 7/10

JoyAI-LLM Flash: Advancing Mid-Scale LLMs with Token Efficiency

arXiv – CS AI|Aichen Cai, Anmeng Zhang, Anyu Li, Bo Zhang, Bohua Cai, Chang Li, Changjian Jiang, Changkai Lu, Chao Xue, Chaocai Liang, Cheng Zhang, Dongkai Liu, Fei Wang, Guoqiang Huang, Haijian Ke, Han Lin, Hao Wang, Ji Miao, Jiacheng Zhang, Jialong Shi, Jifeng Zhu, Jingjing Qian, Junhui Luo, Junwu Xiong, Lam So, Liang Huang, Ming Ke, Mingyang Li, Panfeng Shi, Peng Hao, Qi Wang, Qian Lai, Qiaoqiao Yuan, Qingyu Yin, Qiong Cao, Qixiang Wang, Rongcheng Bian, Rongduo Han, Shaoqiang Zheng, Shi Hu, Shi Suo, Shijie Ren, Shijin Zhang, Shiying Fan, Shuai Xie, Tianyi Zhang, Wei Liu, Wentao Tan, Xianghan Meng, Xiaodong He, Xing Pan, Xiran Wang, Xuyang Peng, Ya Zhang, Yang Liu, Yangyang Duan, Yanxu Chen, Yicheng Gong, Yidan Huang, Yifei Liu, Yinhao Bai, Yongqiang Liu, Yuesong Zhang, Yuqi Zhang, Zerui Xie, Zhenfang Wang, Zhennan Shen, Zheyuan Liu, Zhuwei Zeng|April 6, 2026 at 04:00 AM

🤖AI Summary

JoyAI-LLM Flash is a new efficient Mixture-of-Experts language model with 48B parameters that activates only 2.7B per forward pass, trained on 20 trillion tokens. The model introduces FiberPO, a novel reinforcement learning algorithm, and achieves higher sparsity ratios than comparable industry models while being released open-source on Hugging Face.

Key Takeaways

→JoyAI-LLM Flash achieves high efficiency with 48B total parameters but only 2.7B active parameters per forward pass.
→The model was pretrained on 20 trillion tokens and optimized through supervised fine-tuning, DPO, and reinforcement learning.
→FiberPO algorithm decomposes trust-region maintenance into global and local components for improved policy optimization.
→The model balances thinking and non-thinking cognitive modes to improve token efficiency.
→Both base and post-trained model variants are released open-source on Hugging Face.

Mentioned in AI

Companies

Hugging Face→