17 articles tagged with #optimization-algorithms. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce YAQA, a new quantization algorithm that improves model compression by directly optimizing end-to-end error rather than layer-by-layer error. The method achieves 30% error reduction compared to existing approaches like GPTQ and even outperforms quantization-aware training, with theoretical guarantees backing its performance.
Researchers introduce StoSignSGD, a novel optimization algorithm that fixes convergence issues in SignSGD by injecting structural stochasticity while maintaining unbiased updates. The algorithm demonstrates 1.44x to 2.14x speedup in low-precision FP8 LLM pretraining where AdamW fails, and outperforms existing optimizers in mathematical reasoning fine-tuning tasks.
Researchers present a new approach to General Matrix Multiplication (GEMM) using Space Filling Curves that automatically optimizes data movement across memory hierarchies without requiring platform-specific tuning. The method achieves up to 5.5x speedups over vendor libraries and demonstrates significant performance gains in LLM inference and distributed computing applications.
Researchers propose a constraint-enhanced physical search principle demonstrating that exploration efficiency improves by matching temporal correlations in exploration patterns to spatial correlations generated by physical constraints, rather than maximizing randomness or anti-correlation.
Researchers propose MC-PSO and MC-APSO, novel parallel neural network architectures that combine multi-column radial basis function networks with particle swarm optimization algorithms. These methods outperform existing approaches in accuracy, recall, and computational efficiency on benchmark datasets by distributing training across spatial subsets.
Researchers developed an integrated algorithmic platform combining Building Information Modeling, sensor data, and multi-objective optimization to design energy-efficient buildings. Testing on a mid-rise office building achieved a 29.3% reduction in annual energy consumption while limiting lifecycle cost increases to 3.7%, demonstrating practical scalability for green building design.
Researchers introduce a new class of asynchronous adaptive first-order optimization methods that improve upon existing algorithms through momentum and inexact normalization variants. The methods achieve O(1/√t) convergence rates in stochastic non-convex settings and demonstrate practical relevance for large-scale heterogeneous machine learning systems.
Researchers propose FOAM, an adaptive algorithm that addresses the computational bottleneck in Shampoo optimization by dynamically controlling damping factors and eigendecomposition frequency to mitigate errors from stale preconditioner updates. The method reduces wall-clock training time while maintaining convergence stability, offering a practical solution to the efficiency-fidelity trade-off in large-scale machine learning optimization.
Researchers demonstrate how selection hyper-heuristics can automatically adjust learning periods to optimize pseudo-Boolean problem solving, eliminating manual parameter tuning. The Random Gradient hyper-heuristic achieves optimal neighbourhood size selection in nearly all iterations while maintaining theoretically optimal performance on the LeadingOnes benchmark.
Researchers propose Coherent Coordinate Descent (CoCD), a deterministic zeroth-order optimization method that improves sample efficiency for scenarios where backpropagation is unavailable. The approach reframes stale gradients as computational assets and demonstrates that larger finite-difference step sizes create implicit landscape smoothing, achieving superior convergence stability compared to existing randomized methods across neural network architectures.
Researchers introduce ReWA, a novel sparse optimization method combining reparameterization, weight decay, and adaptive learning rates to address instability issues in ℓp regularization. Experiments on CIFAR-10 and ImageNet demonstrate that ReWA achieves superior sparsity compared to ℓ1 regularization while maintaining test accuracy, offering a practical alternative for neural network compression.
Researchers introduce HEART, a novel framework for efficient multi-model federated learning across vehicle-edge-cloud architectures that addresses training latency and resource allocation challenges in IoV systems. The solution combines hybrid synchronous-asynchronous aggregation with optimized task scheduling using particle swarm optimization and genetic algorithms.
Researchers introduced FrontierOR, a benchmark that tests whether leading LLMs can design efficient optimization algorithms for real-world large-scale problems. The evaluation of seven models reveals significant limitations: even frontier models outperform Gurobi (a standard solver) in only 31% of cases, highlighting a substantial gap between LLM capabilities in formulation and practical algorithmic optimization.
Researchers present Optimal FALQON, an enhanced quantum optimization algorithm that adaptively tunes layer-wise parameters to improve performance on noisy quantum devices. Testing on 3-regular graphs demonstrates significant improvements in convergence speed and solution quality compared to standard approaches, with implications for practical quantum computing applications.
Researchers propose Posterior Sampling-based Policy Optimization (PSPO), a novel approach to offline reinforcement learning that addresses the critical challenge of balancing model generalization with robustness against exploitation errors. By formulating dynamics modeling as Bayesian inference, PSPO enables safer learning from out-of-distribution data while maintaining theoretical convergence guarantees.
Researchers present a comprehensive mathematical framework unifying generalized Euler logarithms with applications to machine learning optimization. The work establishes theoretical foundations for deformed exponential functions and introduces new algorithms—Generalized Exponentiated Gradient and Mirror Descent schemes—alongside an Euler-based loss function for neural networks that integrates with natural gradient descent.
Researchers propose DREE, a novel lifelong learning framework for neural vehicle routing problem solvers that handles continually drifting task patterns with limited training resources per task. The approach addresses a gap in existing methods by managing catastrophic forgetting while learning sequential tasks in real-world logistics scenarios where problem patterns shift over time.
