AIBullisharXiv – CS AI · Jun 257/10
🧠Researchers introduce Wan-Streamer, a unified foundation model that handles real-time audio-visual interaction through a single Transformer architecture, eliminating the need for separate modules and achieving approximately 200ms model-side latency. The system enables sub-second duplex communication by integrating perception, reasoning, generation, and response timing within one end-to-end model.
AIBullisharXiv – CS AI · Jun 237/10
🧠SpotAttention is a lightweight machine learning technique that reduces computational costs for large language models processing long text sequences. By learning to identify only the most relevant tokens to attend to, it achieves 3.9x faster decoding speeds while maintaining accuracy at context lengths eight times longer than training, addressing a critical efficiency bottleneck in modern LLMs.
AIBullisharXiv – CS AI · Jun 237/10
🧠MemoryVAM introduces an episodic memory mechanism for video-world-model policies that enables robots to perform long-horizon manipulation tasks by retaining and leveraging historical context. The system achieves significant performance improvements on benchmark tasks and real robot experiments, addressing a fundamental limitation where short observation windows make complex manipulation non-Markovian.
AINeutralarXiv – CS AI · Jun 237/10
🧠Researchers demonstrate that attention sinks, representation collapse, and norm stratification—previously thought to be transformer-specific problems—are universal behaviors of content-based routing systems with mismatched metrics. The study reveals this collapse pattern occurs across diverse architectures including softmax attention, graph attention, state-space models, and recurrent mixers, suggesting the issue stems from fundamental routing mechanics rather than transformer design.
AIBullishMIT Technology Review · Jun 197/10
🧠AI startup Subquadratic emerged from stealth claiming to have solved a mathematical bottleneck limiting large language model performance. The breakthrough addresses computational constraints that have hindered LLM efficiency and scalability, potentially accelerating AI development across the industry.
AIBullishMIT Technology Review · Jun 197/10
🧠Miami-based AI startup Subquadratic emerged from stealth claiming to have solved a decade-old mathematical bottleneck constraining large language model performance. The breakthrough could accelerate LLM capabilities and efficiency, though initial skepticism prompted the team to provide technical evidence.
AIBullisharXiv – CS AI · Jun 197/10
🧠Emyx, a 140M-parameter conditional flow matching model, achieves superior protein generation performance while requiring 4x less training compute than existing systems like RFdiffusion3. The model demonstrates that enzyme design generators can operate efficiently without inheriting expensive architectures from structure prediction systems, outperforming larger competitors on strict geometric accuracy and structural diversity benchmarks.
AIBullisharXiv – CS AI · Jun 97/10
🧠Researchers present RTPurbo, a method that transforms standard full-attention language models into efficient sparse models within just hundreds of training steps. By leveraging the observation that LLMs are intrinsically sparse, the approach achieves up to 9.36× speedup during prefill and 2.01× during decode at 1M context length while maintaining near-lossless accuracy.
AIBullisharXiv – CS AI · Jun 57/10
🧠Researchers propose Cross-Layer Sparse Attention (CLSA), a novel architecture that optimizes long-context LLM inference by sharing both key-value caches and routing indices across decoder layers. The method achieves up to 7.6x decoding speedup and 17.1x throughput improvement at 128K context while maintaining accuracy, addressing the efficiency-quality tradeoff that has constrained existing sparse attention approaches.
AIBullisharXiv – CS AI · Jun 57/10
🧠Researchers introduce SpanNorm, a novel normalization technique for deep Transformer architectures that combines the training stability of PreNorm with the performance benefits of PostNorm. The method uses spanning residual connections and PostNorm-style computation to prevent gradient instability and representation collapse, demonstrating improvements in both dense and Mixture-of-Experts model configurations.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers have identified critical performance gaps in open-source Relational Foundation Models (RFMs) compared to commercial alternatives by analyzing the Relational Transformer architecture. Their findings—that sparse label coverage and insufficient real-world training data limit current models—led to OpenRFM, which achieves 30% performance improvements and outperforms the commercial KumoRFMv1 baseline.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce SubFit, a post-training compression method for Large Language Models that operates at the submodule level rather than full-layer granularity, achieving superior perplexity-accuracy trade-offs. The approach selects non-contiguous Attention and FeedForward submodules with individual fitted residual bypasses, delivering 84.6% downstream accuracy retention at 25% sparsity compared to 81.6% for existing methods.
🏢 Perplexity
AIBullisharXiv – CS AI · Jun 27/10
🧠FastSLM introduces a Hierarchical Temporal Abstractor (HTA) that compresses long-form speech into just 1.67 tokens per second—a 97% reduction—while maintaining competitive performance on speech understanding benchmarks. This architecture solves a critical scaling bottleneck for multimodal AI models by preserving acoustic detail despite extreme compression, enabling efficient deployment of speech-capable language models.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce Prototype Transformer (ProtoT), a new language model architecture that replaces standard self-attention with a linear-cost prototype-based module to improve interpretability. The approach enables models to automatically learn and represent named concepts, addressing long-standing concerns about opacity in large language models while maintaining competitive performance on standard benchmarks.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers propose a render-free framework for 3D-aware video diffusion models that uses compressed mesh tokens instead of 2D rendered guidance to control human motion in generated videos. By processing 3D geometric information directly alongside video tokens, the approach demonstrates improved performance on motion control tasks while reducing artifacts associated with traditional 2D guidance methods.
AIBullisharXiv – CS AI · Jun 27/10
🧠LayerRoute is a lightweight adapter that enables language models to dynamically skip transformer blocks based on input type, achieving 12.91% computational efficiency gains with minimal training overhead. By combining per-layer routers with LoRA fine-tuning, the system learns to skip 15.25% of computations for tool calls while maintaining full capacity for complex reasoning tasks, demonstrating significant potential for optimizing agentic AI systems.
🏢 Perplexity
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers demonstrate that the 'reversal curse' — an autoregressive language model's inability to deduce inverse relationships from forward training data — can be mitigated through a simple data regularization technique called Identity Bridge. By adding self-referential training examples (e.g., 'Alice's name is Alice'), a 1B parameter model achieves 50% success on reversal tasks compared to near-zero baseline performance, suggesting LLMs can learn higher-level logical rules rather than merely memorizing facts.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce DeLask, a novel decoding framework that reduces hallucinations in Large Language Models by dynamically skipping decoder layers prone to generating false information. The method uses gradient-based analysis to identify problematic layers and partially aggregates their hidden states, demonstrating consistent improvements across diverse LLMs without requiring model retraining.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce MIND (Data Manifold-aware Image diffusioN moDel), a novel diffusion-based image generation framework that combines discrete patch tokenization with continuous diffusion modeling. The approach achieves significant performance improvements, reducing FID scores to 2.06 on ImageNet-256×256 with guidance using only 130M parameters, substantially outperforming larger baseline models.
AIBullisharXiv – CS AI · Jun 17/10
🧠RayDer introduces a unified transformer architecture that consolidates camera estimation, scene reconstruction, and rendering into a single model for self-supervised novel view synthesis from real-world video. The system achieves clean power-law scaling with data and compute while maintaining competitive performance with supervised approaches, addressing a key scalability challenge in 3D vision.
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers propose OBCache, a novel KV cache pruning framework that optimizes memory efficiency for long-context LLM inference by measuring token importance based on actual impact to attention outputs rather than heuristic attention weights. The method, grounded in Optimal Brain Damage theory, demonstrates consistent accuracy improvements over existing eviction strategies on LLaMA and Qwen models.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers present PeRQ, a post-training quantization method that uses permutations to optimize block rotations for neural network compression. The approach recovers up to 90% of full-vector rotation performance when quantizing large language models to INT4, significantly outperforming existing block rotation methods.
🏢 Perplexity🧠 Llama
AIBullisharXiv – CS AI · May 297/10
🧠Researchers have identified "keystone neurons" in large language models—a tiny subset of neurons that remain highly activated across diverse tasks and are critical for model performance. By fine-tuning only these neurons rather than updating all parameters, they achieved comparable or better task performance while preserving other capabilities, offering a more efficient approach to model adaptation.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce TimeRCD, a foundation model for time series anomaly detection that uses a novel Relative Context Discrepancy approach instead of traditional reconstruction methods. The model achieves superior zero-shot performance by detecting discrepancies between adjacent time windows, addressing fundamental limitations in existing anomaly detection systems that produce high false positive and negative rates.
AIBearisharXiv – CS AI · May 287/10
🧠Researchers demonstrate that large language model refusal behavior can be detected and exploited through intermediate layer activations before final output generation. A new attack method called Mechanistic AutoDAN leverages this discovery to achieve competitive jailbreak success rates while reducing computational time by up to 72%, raising concerns about LLM safety mechanisms.
