#prompting-strategies News & Analysis

3 articles tagged with #prompting-strategies. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBearisharXiv – CS AI · Jun 56/10

🧠

Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

Researchers conducted the first systematic evaluation of Large Language Models' ability to generate correct TLA+ formal specifications from natural language, testing 30 LLMs across 2,730 runs. Results show LLMs achieve only 8.6% semantic correctness despite 26.6% syntactic correctness, indicating current models cannot reliably produce formal specifications without expert oversight.

AINeutralarXiv – CS AI · May 96/10

🧠

Evaluating Prompting and Execution-Based Methods for Deterministic Computation in LLMs

Researchers systematically evaluated multiple prompting strategies for LLMs on deterministic computation tasks, finding that standard methods like Chain-of-Thought achieve only moderate accuracy while Program-of-Thought (PoT) and specialized models achieve perfect accuracy by delegating computation to external tools. The study demonstrates that LLMs simulate reasoning patterns rather than reliably performing exact symbolic computation, suggesting hybrid approaches combining LLMs with external executors provide more reliable solutions for deterministic tasks.

AINeutralarXiv – CS AI · Apr 136/10

🧠

Temperature-Dependent Performance of Prompting Strategies in Extended Reasoning Large Language Models

Researchers systematically evaluated how sampling temperature and prompting strategies affect extended reasoning performance in large language models, finding that zero-shot prompting peaks at moderate temperatures (T=0.4-0.7) while chain-of-thought performs better at extremes. The study reveals that extended reasoning benefits grow substantially with higher temperatures, suggesting that T=0 is suboptimal for reasoning tasks.

🧠 Grok