AIBullisharXiv – CS AI · Jun 97/10
🧠WhiFlash introduces a novel speculative decoding method that combines autoregressive and diffusion-based drafting models through token-level routing, achieving up to 69.6% throughput improvements over existing approaches. The system uses lightweight controllers to dynamically switch between drafting paradigms based on per-token conditions, addressing a key bottleneck in LLM inference efficiency.
AIBullisharXiv – CS AI · Jun 97/10
🧠Researchers introduce TLDR, a patch-based autoregressive framework that compresses audio tokens to accelerate text-to-speech synthesis. The method achieves 1.8x inference speedup and reduces KV-cache memory by 75% without replacing existing model modules, addressing a key efficiency bottleneck in codec-based speech language models.
AIBullisharXiv – CS AI · Jun 87/10
🧠Researchers introduce On-Policy Diffusion Language Models (OPDLM), a technique that converts autoregressive language models into diffusion models using 15-7,000x fewer training tokens. The method addresses fundamental efficiency problems by eliminating train-inference mismatches and preserving knowledge from the original model through on-policy distillation.
AIBullisharXiv – CS AI · Jun 87/10
🧠Researchers have developed dots.tts, a 2-billion parameter text-to-speech model that achieves state-of-the-art performance through innovations in continuous speech modeling, full-history conditioning, and self-corrective training. The model demonstrates exceptional multilingual capabilities and enables low-latency speech generation, with code and weights released open-source under Apache 2.0 license.
AIBullisharXiv – CS AI · May 297/10
🧠MENTOR is a novel autoregressive framework for multimodal-conditioned image generation that achieves strong visual control and prompt-following performance through efficient two-stage training without relying on auxiliary adapters or cross-attention modules. The method demonstrates superior performance on the DreamBench++ benchmark compared to diffusion-based approaches while requiring fewer training resources.
AIBullisharXiv – CS AI · May 117/10
🧠Researchers have developed CASCADE, a novel speculative decoding technique that accelerates autoregressive image generation by up to 3.6x through identifying and exploiting redundancies in neural network representations. The method addresses a critical bottleneck in image synthesis by reducing draft token rejection rates without requiring model retraining, advancing the efficiency of text-to-image AI systems.
AIBullisharXiv – CS AI · May 77/10
🧠Researchers demonstrate that masked fine-tuning—a demasking objective borrowed from diffusion models—significantly improves knowledge injection in autoregressive LLMs without requiring expensive paraphrase augmentation and while remaining resistant to the reversal curse. This technique closes the performance gap between autoregressive and diffusion language models, with applications extending to math tasks and large-scale knowledge-intensive benchmarks.
AIBearisharXiv – CS AI · Apr 147/10
🧠Researchers demonstrate critical vulnerabilities in watermarking techniques designed for autoregressive image generators, showing that watermarks can be removed or forged with access to only a single watermarked image and no knowledge of model secrets. These findings undermine the reliability of watermarking as a defense against synthetic content in training datasets and enable attackers to manipulate authentic images to falsely appear as AI-generated content.
AIBullisharXiv – CS AI · Mar 97/10
🧠Researchers have developed CanvasMAR, a new masked autoregressive video prediction model that generates high-quality videos with fewer sampling steps by using a "canvas" approach that provides global structure early in the generation process. The model demonstrates superior performance on major benchmarks including BAIR, UCF-101, and Kinetics-600, rivaling advanced diffusion-based methods.
AIBullisharXiv – CS AI · Mar 46/102
🧠Researchers introduce CoWVLA (Chain-of-World VLA), a new Vision-Language-Action model paradigm that combines world-model temporal reasoning with latent motion representation for embodied AI. The approach outperforms existing methods in robotic simulation benchmarks while maintaining computational efficiency through a unified autoregressive decoder that models both keyframes and action sequences.
AIBullisharXiv – CS AI · Jun 236/10
🧠Researchers present Gazer, a training-free framework that uses multimodal large language models to identify and correct semantic errors in autoregressive visual models during image and video generation. The approach operates through diagnostic and correction stages that analyze intermediate generation states and adjust trajectories without requiring additional model training.
AINeutralarXiv – CS AI · Jun 116/10
🧠Researchers propose Autoregressive Direct Preference Optimization (ADPO), a refined theoretical framework for aligning large language models with human preferences. The innovation explicitly incorporates autoregressive assumptions before applying the Bradley-Terry model, resulting in a mathematically elegant loss function and introducing two distinct length measures—token length and feedback length—for optimizing LLM preference alignment.
AINeutralarXiv – CS AI · Jun 96/10
🧠BioVid introduces an autoregressive video generation framework that learns temporal structure from behavioral data rather than using fixed frame counts. The system uses a specialized tokenizer and transformer architecture to naturally determine when behavioral sequences end, matching real-world action duration distributions significantly better than existing methods.
AIBullisharXiv – CS AI · Jun 96/10
🧠Researchers introduce DySink, a novel framework for autoregressive long video generation that dynamically selects relevant historical frames instead of using static early-frame anchors. The method addresses the problem of outdated context degrading video quality and introduces a sink anomaly gate to prevent content collapse, demonstrating improvements in temporal consistency for minute-long videos.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers develop theoretical bounds for KV cache compression in language models, discovering that context sensitivity decays polynomially rather than exponentially. Their findings enable more efficient memory-aware cache policies that reduce memory requirements while maintaining model performance, with practical implications for deploying larger models on resource-constrained systems.
AINeutralarXiv – CS AI · Jun 56/10
🧠Researchers introduce Double Preconditioning (DoPr), a new optimization technique that improves neural network performance during real-world deployment by combining gradient-wise and activation-wise preconditioning. The method addresses test-time feedback—the gap between training metrics and actual task performance in autoregressive models—without requiring improvements in traditional validation loss metrics.
AIBullisharXiv – CS AI · Jun 46/10
🧠Researchers propose a lightweight autoregressive framework for graph generation that achieves near log-linear complexity by using structure-guided topological ordering, addressing scalability limitations in current diffusion and autoregressive models. The two-phase training strategy reduces overfitting and promotes novel graph generation while maintaining validity, with applications spanning molecular discovery, circuit design, and cybersecurity.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers introduce Channel-wise Vector Quantization (CVQ), a novel image tokenization method that quantizes individual channels rather than spatial patches, paired with a Channel-wise Autoregressive (CAR) generation model that produces images by progressively refining visual details. The approach achieves 100% codebook utilization and demonstrates strong performance on text-to-image generation benchmarks, suggesting a fundamentally different approach to visual AI tasks.
AINeutralarXiv – CS AI · May 286/10
🧠A new arXiv paper challenges the widespread claim that Transformers are Turing-complete, arguing that existing proofs conflate two distinct computational settings. The research clarifies that real-world LLM deployment operates under fixed-system constraints where context management critically determines actual computational power, rather than the idealized scaling-family setting used in most theoretical proofs.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers present a unified mathematical framework for Test-Time Adaptation (TTA) in autoregressive generative models, decomposing entropy minimization into token-level policy gradient and entropy losses. Validated on Whisper ASR across 20+ domains, the approach demonstrates consistent performance improvements and reconciles previously disparate adaptation methods under a single theoretical foundation.
AINeutralarXiv – CS AI · May 126/10
🧠MeshFIM introduces a Fill-in-the-Middle autoregressive framework that enables local editing of low-poly meshes without regenerating entire structures. The technology allows targeted mesh region refinement while preserving surrounding geometry, addressing a critical limitation in current mesh generation workflows through specialized techniques including boundary enforcement, topological preservation, and a gated geometry encoder.
AIBullisharXiv – CS AI · May 116/10
🧠Researchers introduce REPR-ALIGN, a method that converts autoregressive language models into diffusion language models by aligning their internal representations rather than retraining from scratch. The approach achieves up to 4x training acceleration and demonstrates that semantic structures learned through next-token prediction can transfer across different generation orders.
AINeutralarXiv – CS AI · Apr 206/10
🧠Researchers demonstrate that reward-weighted classifier-free guidance (RCFG) can dynamically adjust autoregressive model outputs to optimize arbitrary reward functions at test time without retraining. Applied to molecular generation, this approach enables real-time optimization of competing objectives and accelerates reinforcement learning convergence when used as a teacher for policy distillation.
AINeutralarXiv – CS AI · Apr 146/10
🧠Researchers evaluated eight large Masked Diffusion Language Models (up to 100B parameters) and found they still underperform comparable autoregressive models despite promises of parallel token generation. The study reveals MDLMs exhibit task-dependent decoding behavior and propose a Generate-then-Edit paradigm to improve performance while maintaining parallel processing efficiency.
AIBullisharXiv – CS AI · Apr 136/10
🧠Researchers introduce WAND, a framework that reduces computational and memory costs of autoregressive text-to-speech models by replacing full self-attention with windowed attention combined with knowledge distillation. The approach achieves up to 66.2% KV cache memory reduction while maintaining speech quality, addressing a critical scalability bottleneck in modern AR-TTS systems.
