4 articles tagged with #embedding-optimization. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce Leviathan, a Transformer architecture that decouples input embeddings from output projections using learned embedding vectorization (LEV), achieving 9% perplexity reduction at 1.2B parameters with minimal overhead. The approach concentrates improvements on rare tokens while requiring 2.1x fewer training tokens to match baseline performance.
Researchers propose Null-Text Test-Time Alignment (Null-TTA), a novel method for adapting text-to-image diffusion models during inference by optimizing the unconditional embedding in classifier-free guidance rather than manipulating latent variables. This approach maintains semantic coherence while achieving superior alignment to target rewards without reward hacking, establishing a new paradigm for test-time model adaptation.
Researchers propose simplicial embeddings, a lightweight geometric technique that constrains neural network representations to discrete, sparse structures, improving sample efficiency in reinforcement learning agents. When integrated into popular actor-critic algorithms like PPO and FastTD3, the method enhances performance and learning speed across diverse control tasks without sacrificing computational speed.
Researchers propose Self-Conditioned Positional HNSW (SCP-HNSW), a method to improve retrieval-augmented generation (RAG) systems by reducing redundant overlapping chunks in document retrieval. The approach adds positional codes to embeddings and implements a two-pass query procedure, validated through 770 text-evidence reviews and 70 OCR audits showing varying quality levels across different document types.
