CryptoBullishCrypto Briefing · 2d ago7/10
⛓️Sei has announced its Giga upgrade roadmap, targeting 200,000 transactions per second (TPS) and 400 millisecond finality, positioning itself as a high-performance blockchain solution for DeFi and high-frequency trading. This upgrade represents a significant scaling advancement that could reshape how blockchain networks handle demanding applications requiring speed and throughput.
AIBullisharXiv – CS AI · 3d ago7/10
🧠Researchers propose a new runtime layer architecture for serving multi-agent LLM systems, positioned between application frameworks and inference engines. The approach enables unified policy management for cross-cutting concerns like caching and fairness, with CacheSage demonstrating 13-37% improvements in cache hit rates and 12-29% reductions in time-to-first-token latency.
AIBullisharXiv – CS AI · 3d ago7/10
🧠Researchers present a systematic study of Attention-FFN Disaggregation (AFD), a technique that separates attention and expert layers across different GPU groups to optimize inference serving for Mixture-of-Experts language models. The framework demonstrates that AFD enables 4k tokens/s throughput on DeepSeek-V3.2 under strict latency constraints where traditional disaggregation approaches fail, providing design principles for scaling LLM infrastructure.
AIBullisharXiv – CS AI · 4d ago7/10
🧠Researchers introduce HiSpec, a hierarchical speculative decoding framework that accelerates large language model inference by using early-exit models for intermediate verification, achieving up to 2.01× throughput improvements without sacrificing accuracy.
CryptoBullishU.Today · May 227/10
⛓️Zcash has deployed its NU7 testnet upgrade, achieving a 75% reduction in block times that triples overall network speed. This significant performance enhancement addresses scalability concerns and positions the privacy-focused blockchain to compete more effectively with faster layer-1 networks.
AIBullisharXiv – CS AI · May 127/10
🧠SynerDiff is a new continuous batching system for diffusion model inference that addresses resource contention issues between UNet and VAE components. The system achieves 1.6× throughput improvement and up to 78.7% latency reduction through intra-level and inter-level optimization strategies, enabling faster AI-generated content services.
AIBullisharXiv – CS AI · May 117/10
🧠Researchers introduce TAPER, an admission controller for managing parallel branch execution in LLM serving systems. The system dynamically regulates how many concurrent decoding branches are allowed per request step, balancing throughput gains against degradation to co-batched requests, achieving 1.77x improvement in goodput over conservative baselines.
AIBullisharXiv – CS AI · Mar 267/10
🧠Researchers developed ODMA, a new memory allocation strategy that improves Large Language Model serving performance on memory-constrained accelerators by up to 27%. The technique addresses bandwidth limitations in LPDDR systems through adaptive bucket partitioning and dynamic generation-length prediction.
AINeutralarXiv – CS AI · Mar 127/10
🧠Researchers conducted comprehensive benchmarks of LLM inference on AMD Instinct MI325X GPUs, testing models from 235B to 1 trillion parameters. The study reveals that architecture-aware optimization is critical, with different model types requiring specific configurations for optimal performance on AMD hardware.
🧠 Llama
AIBullishMarkTechPost · Mar 117/10
🧠NVIDIA has released Nemotron 3 Super, a 120 billion parameter open-source AI model designed for multi-agent applications. The hybrid Mamba-Attention MoE model delivers 5x higher throughput and bridges the gap between proprietary frontier models and transparent open-source alternatives.
🏢 Nvidia
AIBullisharXiv – CS AI · Mar 67/10
🧠Researchers introduce AMV-L, a new memory management framework for long-running LLM systems that uses utility-based lifecycle management instead of traditional time-based retention. The system improves throughput by 3.1x and reduces latency by up to 4.7x while maintaining retrieval quality by controlling memory working-set size rather than just retention time.
AIBullisharXiv – CS AI · Mar 47/102
🧠Researchers propose SUN (Shared Use of Next-token Prediction), a novel approach for multi-LLM serving that enables cross-model sharing of decode execution by decomposing transformers into separate prefill and decode modules. The system achieves up to 2.0x throughput improvement per GPU while maintaining accuracy comparable to full fine-tuning, with a quantized version (QSUN) providing additional 45% speedup.
AIBullisharXiv – CS AI · Mar 47/103
🧠Nightjar is a new adaptive speculative decoding framework for large language models that dynamically adjusts to system load conditions. It achieves 27.29% higher throughput and up to 20.18% lower latency by intelligently enabling or disabling speculation based on workload demands.
CryptoNeutralCoinDesk · Apr 306/10
⛓️Sui is a Layer-1 blockchain featuring object-based architecture and parallel execution capabilities designed to deliver high throughput for consumer-focused Web3 applications. The platform differentiates itself through technical innovations that address scalability constraints common to earlier blockchain generations.
AIBullisharXiv – CS AI · Mar 36/104
🧠OrbitFlow is a new KV cache management system for long-context LLM serving that uses adaptive memory allocation and fine-grained optimization to improve performance. The system achieves up to 66% better SLO attainment and 3.3x higher throughput by dynamically managing GPU memory usage during token generation.
AIBullisharXiv – CS AI · Mar 26/1017
🧠Researchers developed a data-driven pipeline to optimize GPU efficiency for distributed LLM adapter serving, achieving sub-5% throughput estimation error while running 90x faster than full benchmarking. The system uses a Digital Twin, machine learning models, and greedy placement algorithms to minimize GPU requirements while serving hundreds of adapters concurrently.
AINeutralGoogle Research Blog · Feb 113/107
🧠This appears to be a research article focused on algorithmic optimization for scheduling systems with time-varying capacity constraints. The work addresses theoretical approaches to maximizing throughput in dynamic environments where system capacity changes over time.
