19 articles tagged with #theoretical-cs. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers present a theoretical framework for space-efficient language generation that characterizes the tradeoff between memory constraints and learning accuracy. Using polynomial space, a streaming algorithm can identify most strings in a target language while missing at most O(k^(2s-2)) strings, with a matching lower bound proving this gap is near-optimal.
Researchers formalize test-time training (TTT) as a theoretical framework for sampling from complex probability distributions, proving that the Jerrum-Sinclair random walk approach is query-optimal with a quadratic lower bound. The work bridges generative AI sampling efficiency with classical algorithmic theory, establishing foundational principles for adapting language models during inference.
A theoretical study proves that quantization fundamentally limits dense top-k retrieval systems, requiring embedding dimension and precision to scale logarithmically with corpus size, contradicting prior corpus-independent bounds that assumed infinite precision. This finding has direct implications for practical vector databases and dense retrieval systems where quantization is standard practice.
A theoretical computer science paper formalizes decision-making under information constraints as action-sufficient compression, where systems need only preserve distinctions relevant to choosing optimal actions rather than reconstructing full state information. The framework applies rate-distortion theory to support states with regret-based distortion, offering a mathematical foundation for robust single-cycle arbitration.
A new arXiv paper challenges the premise that AI shutdown problems are inherently difficult to solve, arguing that existing theoretical arguments lack rigor. The authors contend that efforts to address shutdown safety concerns have imposed unnecessary performance constraints on AI models without establishing that the problem is genuinely intractable.
Researchers have developed LegoNE, a framework that enables large language models to automatically discover and formally verify polynomial-time algorithms for computing Nash equilibria in games. The system rediscovered existing optimal algorithms and discovered a new three-player algorithm that provably improves upon previous best-known guarantees, demonstrating that LLMs can innovate beyond established human design paradigms when augmented with formal verification tools.
Researchers present a framework for exact unlearning in reinforcement learning that enables efficient removal of user data upon request, with computational costs only a ρ√ln T fraction of full retraining. The work establishes both an algorithm achieving near-optimal regret bounds for tabular MDPs and matching lower bounds, advancing the theoretical foundation for privacy-preserving machine learning systems.
Researchers present a unified mathematical framework for certifying locality in scalable multi-agent reinforcement learning (MARL) systems by decomposing the state-transition matrix into environment and policy sensitivity components. The approach uses spectral radius analysis to weaken prior Dobrushin bounds and applies temperature-scaled softmax policies to control locality, enabling exponentially decaying truncation bias in networked agent systems.
Researchers introduce MINTS (Minimalist Thompson Sampling), a Bayesian framework that simplifies sequential decision-making under uncertainty by placing priors only on optimal parameters while eliminating unnecessary variables through profile likelihood. The approach achieves near-optimal regret bounds for multi-armed bandits and automatically adapts to structural constraints, matching classical performance benchmarks.
Researchers have developed an algebraic (semantic) theory of anti-unification that extends abstraction and generalization from syntactic term-based systems to arbitrary algebras. This theoretical computer science advancement moves anti-unification beyond equational theories and establishes foundational properties compatible with homomorphisms and isomorphisms, with computability analysis for finite algebras.
Researchers present optimal algorithms for sparse contextual bandits that achieve sample complexity of Õ((s/ε² + |A|/ε)log|Π|/δ), closing a gap from prior work that had exponential dependence on action set size. The results apply to multiclass classification and combinatorial semi-bandits through information-theoretic and algorithmic approaches.
This theoretical computer science paper investigates language generation under bounded memory constraints, extending classical learning theory to a practical setting where algorithms cannot retain complete historical information. The research characterizes when language generation remains possible with various memory limitations and reveals that bounded memory affects different learning tasks—generation, density optimization, and identification—in fundamentally different ways.
Researchers resolve a gap in online fair division theory by proving that proportionality up to one good (PROP1) cannot be approximated by standard greedy algorithms against adaptive adversaries, but can be achieved through randomized allocation or learning-augmented approaches with predictions.
Researchers have demonstrated that Stochastic Gradient Descent with Momentum (SGDM), a fundamental optimization algorithm in machine learning, maintains strong generalization properties through algorithmic stability analysis. The study resolves a longstanding conjecture that momentum, while accelerating training, might harm generalization performance, providing tight stability bounds applicable to both Polyak's and Nesterov's momentum schemes.
Researchers resolve an open problem in multi-armed bandit theory by characterizing how best-action oracle queries improve learning algorithms in the realistic bandit-feedback model. They prove that benefits depend critically on reward structure: correlated stochastic rewards cannot achieve the theoretical gains seen in full-feedback settings, while i.i.d. stochastic rewards maintain near-optimal improvements with logarithmic precision.
This theoretical computer science paper establishes formal conditions for efficient personalized alignment in large language models, proving that user diversity—specifically whether user-specific parameters span latent reward directions—is both necessary and sufficient for optimal statistical efficiency. The research provides rigorous mathematical foundations for adapting AI systems to heterogeneous user preferences.
A theoretical paper demonstrates that principals using standard scoring rules to oversee strategic AI agents face an inherent impossibility: achieving both honest reporting and accurate calibration simultaneously. The research identifies step-function approval thresholds as the only mechanism that preserves calibration while maintaining incentive compatibility, with specific equivalence properties under the Brier score.
Researchers introduce a resilience framework for bi-criteria combinatorial optimization under noisy conditions, extending bandit feedback algorithms from single-objective to multi-objective settings. The framework achieves sublinear regret bounds without requiring structural assumptions like linearity or submodularity, with potential applications to constrained optimization problems in machine learning and algorithmic decision-making.
Researchers introduce a spectral filtering method for learning complex-valued linear dynamical systems with sector-bounded spectrum, achieving dimension-free regret bounds for sequence prediction. The approach uses Slepian basis functions and demonstrates that learning efficiency depends on an effective dimension independent of state space size, with applications to signal processing and quantum systems.
