9 articles tagged with #neural-compression. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce TLDR, a patch-based autoregressive framework that compresses audio tokens to accelerate text-to-speech synthesis. The method achieves 1.8x inference speedup and reduces KV-cache memory by 75% without replacing existing model modules, addressing a key efficiency bottleneck in codec-based speech language models.
Researchers propose a novel framework combining importance-aware news compression and process reward models to improve LLM-based time series forecasting across finance, energy, and cryptocurrency markets. The method addresses practical limitations of existing approaches by intelligently filtering news articles within context windows and guiding iterative retrieval, achieving better accuracy with fewer refinement iterations.
Researchers demonstrate that 2-bit quantization of large reasoning models causes instability leading to longer inference traces rather than speedup, but introduce lightweight recovery techniques (FP16 planning and loop rescue) that restore accuracy from 17-65% to 74-87% while maintaining computational efficiency.
Researchers demonstrate that multifidelity simulation-based inference can extract cosmological information from weak lensing fields using fewer than 100 high-fidelity N-body simulations, achieving an order-of-magnitude reduction in computational cost. By pre-training neural models on fast, low-fidelity simulations and fine-tuning on expensive high-fidelity runs, the method enables field-level cosmological inference that captures substantially more information than traditional two-point statistics.
Researchers introduce Whisper-GPT, a hybrid language model that combines continuous audio representations (spectrograms) with discrete acoustic tokens to improve speech and music generation. This approach addresses context length limitations in traditional token-based models while maintaining high-fidelity audio synthesis capabilities.
Researchers propose Kernel Affine Hull Machines (KAHM) as a lightweight alternative to transformer-based neural encoders for semantic search in frozen representation spaces. The method achieves 8.53x faster query encoding while maintaining competitive retrieval performance, offering practical efficiency gains for production deployment scenarios.
Researchers introduce LEAP, a new technique for pruning large language models that uses learnable per-weight masks to achieve better accuracy than existing layer-wise methods, particularly at aggressive sparsity levels. The approach replaces earlier intractable parameterization methods with a Bernoulli-via-Gumbel-sigmoid relaxation, demonstrating 2.59 points average improvement over ADMM across multiple LLM families.
Researchers introduce ProbScale, a framework that combines neural scaling laws with probing analysis to identify parameter-efficient subnetworks in Small Language Models. The method achieves 5-10x parameter reduction while maintaining 95-98% performance on downstream tasks, addressing deployment challenges for resource-constrained environments.
Researchers introduce STARS, a data-free knowledge distillation method that improves the transfer of learning from artificial neural networks (ANNs) to spiking neural networks (SNNs) without access to original training data. The technique combines batch normalization matching with relational consistency and threshold-aware regularization, achieving significant accuracy improvements across standard benchmarks.
