AIBullisharXiv – CS AI · Mar 267/10
🧠Researchers present Memory Sparse Attention (MSA), a new AI framework that enables language models to process up to 100 million tokens with linear complexity and less than 9% performance degradation. The technology addresses current limitations in long-term memory processing and can run 100M-token inference on just 2 GPUs, potentially revolutionizing applications like large-corpus analysis and long-history reasoning.
AIBullisharXiv – CS AI · Jun 257/10
🧠Researchers introduce ATMA, a novel hybrid attention architecture that solves the long-context problem in language models by combining polar attention with gated-delta compression memory. The system maintains 90%+ retrieval accuracy at 64K tokens (32x training length) while improving perplexity monotonically, addressing fundamental limitations of softmax attention that degrades with longer sequences.
🏢 Perplexity
AIBullisharXiv – CS AI · Jun 237/10
🧠SpotAttention is a lightweight machine learning technique that reduces computational costs for large language models processing long text sequences. By learning to identify only the most relevant tokens to attend to, it achieves 3.9x faster decoding speeds while maintaining accuracy at context lengths eight times longer than training, addressing a critical efficiency bottleneck in modern LLMs.
AIBearisharXiv – CS AI · Jun 237/10
🧠Researchers introduce RoboMME-Interference, a benchmark testing how robot memory systems perform across multiple sessions with irrelevant distractions. Testing current memory-augmented AI models reveals significant performance degradation as unrelated sessions accumulate, highlighting a critical gap in long-context robustness for real-world robot deployment.
AIBullisharXiv – CS AI · Jun 197/10
🧠DeepSeek released V4, a new series of efficient mixture-of-experts language models supporting one-million-token context windows. The models achieve significant computational improvements over predecessors while maintaining state-of-the-art performance, with V4-Pro requiring only 27% of the inference compute of DeepSeek-V3.2.
🏢 Hugging Face
AIBullisharXiv – CS AI · Jun 107/10
🧠Researchers introduce Engram, an open-source memory engine for LLM agents that achieves 83.6% accuracy on long-context tasks using only 9.6k tokens versus 79k for full-history baselines, demonstrating that selective retrieval outperforms exhaustive context replay while reducing computational costs by 8x.
AIBearisharXiv – CS AI · Jun 57/10
🧠Researchers discovered that lexical density—the rate at which new information appears in text—significantly limits LLM effective context windows, causing near-perfect models to drop below 60% accuracy on information-dense contexts. This finding reveals that input length and needle position, traditionally blamed for context degradation, overlook a critical third factor that directly impacts real-world LLM performance on compact, information-rich data.
AIBullisharXiv – CS AI · Jun 57/10
🧠Researchers propose Cross-Layer Sparse Attention (CLSA), a novel architecture that optimizes long-context LLM inference by sharing both key-value caches and routing indices across decoder layers. The method achieves up to 7.6x decoding speedup and 17.1x throughput improvement at 128K context while maintaining accuracy, addressing the efficiency-quality tradeoff that has constrained existing sparse attention approaches.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers demonstrate that long-context capacity in language models directly enhances reasoning performance, even on short tasks. The study shows models with stronger long-context abilities consistently achieve higher accuracy on reasoning benchmarks after fine-tuning, suggesting long-context modeling is foundational for advanced reasoning rather than merely useful for processing lengthy inputs.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers introduce SoLoPO, a framework that improves how large language models handle long-context information by decoupling preference optimization into short-context training and short-to-long reward alignment. The approach addresses fundamental limitations in LLM long-context capabilities while improving training efficiency and computational requirements.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce WaveFilter, a training-free framework that uses wavelet transforms to optimize Key-Value cache filtering in Diffusion Large Language Models, addressing computational bottlenecks in long-context processing. The technique enables sparse KV caching to maintain generation quality while reducing inference latency, offering plug-and-play compatibility with existing LLM architectures.
AIBullisharXiv – CS AI · Jun 27/10
🧠BudgetDraft is a new training method for sparse-KV speculative decoding that enables faster language model inference under memory constraints. By training drafters to handle multiple KV cache budgets simultaneously, the technique achieves up to 6.55x speedup on mid-to-long context inference while maintaining acceptance rates and reducing GPU memory usage.
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers propose OBCache, a novel KV cache pruning framework that optimizes memory efficiency for long-context LLM inference by measuring token importance based on actual impact to attention outputs rather than heuristic attention weights. The method, grounded in Optimal Brain Damage theory, demonstrates consistent accuracy improvements over existing eviction strategies on LLaMA and Qwen models.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce MAGIC-Video, a training-free framework that enables multimodal AI systems to process and reason about ultra-long videos spanning days or weeks by combining a structured memory graph with narrative chains. The system outperforms existing baselines on multiple benchmarks, addressing a critical limitation where current LLMs can only handle tens of minutes of video despite having million-token context windows.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers propose Kaczmarz Linear Attention (KLA), an improved algorithm for long-context language modeling that replaces empirically-learned coefficients with mathematically-derived key-norm-normalized step sizes. KLA outperforms existing linear attention baselines like Gated DeltaNet while maintaining computational efficiency and enabling stable processing of up to 65K token contexts.
🏢 Perplexity
AIBullisharXiv – CS AI · May 127/10
🧠Researchers present a biologically-inspired memory architecture for LLM agents that addresses persistent memory management across long interaction horizons. The system incorporates six cognitive mechanisms including sleep-phase consolidation and interference-based forgetting, achieving 97.2% retention precision with 58% storage reduction on a VSCode dataset and matching retrieval accuracy on streaming evaluations.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce Priming, a method that converts pre-trained Transformers into efficient Hybrid State-Space models through knowledge transfer rather than training from scratch. The technique recovers downstream performance using less than 0.5% of original pre-training tokens and enables the first large-scale comparison of SSM architectures, with Hybrid GKA 32B achieving 3.8-point reasoning improvements while delivering 2.3x faster decoding.
🧠 Llama
AIBullisharXiv – CS AI · May 127/10
🧠Researchers propose RDKV, a novel compression technique that jointly optimizes eviction and quantization of the Key-Value cache in large language models to reduce memory bottlenecks during inference. The method achieves 4.5x decode speedup and 1.9x peak memory reduction on 128K context lengths while maintaining 97.81% accuracy, addressing a critical performance constraint in LLM deployment.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce Key-Value Means (KVM), a novel attention mechanism that bridges traditional transformers and linear RNNs by supporting both fixed-size and growing state with linear time complexity. The approach achieves competitive long-context performance while reducing KV-cache memory requirements and enabling flexible prefill time complexity between O(N) and O(N²).
🏢 Hugging Face
AIBullisharXiv – CS AI · May 117/10
🧠Researchers introduce LaProx, a novel KV Cache eviction strategy for long-context LLM inference that reformulates the problem from head-wise weight averaging to output-aware layer-wise matrix multiplication. The method achieves 2× accuracy loss reduction under extreme compression while maintaining performance with just 5% of the original KV cache.
AIBullisharXiv – CS AI · May 117/10
🧠Researchers introduce MISA, an optimization technique that reduces computational costs in DeepSeek's sparse attention mechanism for large language models by treating indexer heads as a mixture-of-experts system. The method achieves 3.82x speedup on GPU inference while maintaining performance across benchmarks, addressing a key bottleneck in long-context LLM processing.
🏢 Nvidia
AIBullisharXiv – CS AI · May 77/10
🧠Researchers introduce Lossless Context Management (LCM), a deterministic architecture for LLM memory that outperforms Claude Code on long-context tasks up to 1M tokens. LCM combines recursive context compression with engine-managed task partitioning, representing an evolution of recursive language models that prioritizes reliability and state retrievability over flexibility.
🧠 Claude🧠 Opus
AIBullisharXiv – CS AI · Apr 207/10
🧠OjaKV introduces a novel framework for compressing key-value caches in large language models through online low-rank projection, addressing a critical memory bottleneck in long-context inference. The method combines selective full-rank storage for important tokens with adaptive compression for intermediate tokens, maintaining accuracy while reducing memory consumption without requiring model fine-tuning.
🧠 Llama
AIBullisharXiv – CS AI · Apr 207/10
🧠Researchers introduce CoMeT (Collaborative Memory Transformer), a novel architecture that enables large language models to process arbitrarily long sequences with constant memory usage and linear time complexity. The system uses a dual-memory approach with FIFO queues and gated updates, demonstrating remarkable performance on long-context tasks including 1M token sequences and real-world applications.
AIBullisharXiv – CS AI · Apr 147/10
🧠IceCache is a new memory management technique for large language models that reduces KV cache memory consumption by 75% while maintaining 99% accuracy on long-sequence tasks. The method combines semantic token clustering with PagedAttention to intelligently offload cache data between GPU and CPU, addressing a critical bottleneck in LLM inference on resource-constrained hardware.
