Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generation

A benchmark study compares Token-Oriented Object Notation (TOON) with JSON for structured data serialization in LLMs, finding that while TOON reduces token usage, plain JSON shows better accuracy overall. The research reveals that TOON's efficiency benefits may only emerge at scale where syntax savings offset the initial prompt overhead.

• →TOON shows promising accuracy-to-token consumption ratio for in-domain generation tasks compared to JSON.
• →Plain JSON generation demonstrates the best one-shot and final accuracy, even outperforming constrained decoding methods.
• →Constrained decoding offers lowest token usage but at the cost of decreased accuracy and significant degradation in some models.
• →For simple structures, constrained decoding actually outperformed TOON in token efficiency.
• →TOON's true efficiency potential likely follows a non-linear scaling curve, becoming beneficial only beyond a specific complexity threshold.