Think in Sentences: Explicit Sentence Boundaries Enhance Language Model's Capabilities

Researchers demonstrate that inserting sentence boundary delimiters in LLM inputs significantly enhances model performance across reasoning tasks, with improvements up to 12.5% on specific benchmarks. This technique leverages the natural sentence-level structure of human language to enable better processing during inference, tested across model scales from 7B to 600B parameters.

This research addresses a fundamental gap in how large language models process information. While previous work has explored dummy token insertion to improve LLM capabilities, this study recognizes that human language naturally organizes around sentence boundaries—a structural property that models should exploit during reasoning. By inserting delimiters at sentence edges, researchers create explicit cognitive boundaries that align with how language models learned from human-generated text.

The approach stems from understanding that LLMs acquire linguistic knowledge through exposure to naturally-structured documents where sentences serve as semantic units. Rather than treating tokens uniformly, this method acknowledges architectural properties of language itself. The researchers validate their hypothesis using both in-context learning and supervised fine-tuning across model scales, demonstrating scalability from 7B to Deepseek-V3's 600B parameters—indicating the technique's robustness across different architectures.

The performance gains are substantial: 7.7% improvement on GSM8k (mathematics reasoning) and 12.5% on DROP (reading comprehension) represent meaningful advances in critical benchmarks. Internal representation analysis reveals that fine-tuned models develop sentence-aware processing, suggesting the modification creates interpretable cognitive structures rather than superficial performance gains.

For the AI development community, this work validates a cognitive-inspired enhancement paradigm that's simple to implement yet empirically effective. The technique requires minimal computational overhead compared to alternative approaches like scaling or architectural redesigns. Future research should explore whether sentence awareness transfers across domains and whether similar structural insights apply to other linguistic units beyond sentences.

• →Explicit sentence boundary delimiters improve LLM reasoning performance by up to 12.5% on benchmark tasks without requiring model retraining.
• →The technique works across model scales from 7B to 600B parameters, indicating broad applicability across different architectures.
• →Fine-tuned models develop measurable sentence-awareness in their internal representations, suggesting structural rather than superficial improvements.
• →This approach leverages the natural sentence-level structure of human language that models originally learned from, aligning training data properties with inference behavior.
• →The method combines simplicity with effectiveness, offering a low-overhead alternative to other capability enhancement techniques like scaling or architectural modification.