Bittensor’s Subnet 3 Trains 72B AI Model on Decentralized Network

Bittensor's Subnet 3 successfully trained Covenant-72B, a 72 billion parameter AI model on a decentralized network, outperforming LLaMA-2-70B with a 67.1 MMLU score versus 65.6. The achievement utilized SparseLoCo technology to reduce communication overhead by 146x and featured blockchain-based contribution tracking, driving TAO token up 14% to $236.

• →Covenant-72B achieved 67.1 on MMLU zero-shot testing, surpassing LLaMA-2-70B's 65.6 score under identical conditions.
• →SparseLoCo technology reduced communication overhead by 146x through sparsification, 2-bit quantization, and error feedback.
• →Gauntlet system scored node contributions via loss evaluation and OpenSkill ranking, with all data recorded on blockchain.
• →TAO token rose 14% to $236 following the announcement of the successful AI model training.
• →The achievement demonstrates the viability of training large AI models on decentralized networks.