Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures

Researchers investigated lower bounds for language modeling using semantic structures, finding that binary vector representations of semantic structure can be dramatically reduced in dimensionality while maintaining effectiveness. The study establishes that prediction quality bounds require analysis of signal-noise distributions rather than single scores alone.

• →Binary vector representations of semantic structure can be dramatically reduced in dimensionality without losing main advantages
• →Lower bounds on prediction quality cannot be established via single scores but need signal-noise distribution analysis
• →The research builds on negative results to establish empirical bounds for semantic-bootstrapping language models
• →A hybrid system combining sequential-neural and hierarchical-symbolic components could generate interpretable text
• →Incremental tagger quality requirements were evaluated for achieving better-than-baseline performance