CONE: Embeddings for Complex Numerical Data Preserving Unit and Variable Semantics

Researchers introduce CONE, a hybrid transformer encoder model that improves numerical reasoning in AI by creating embeddings that preserve the semantics of numbers, ranges, and units. The model achieves 87.28% F1 score on DROP dataset, representing a 9.37% improvement over existing state-of-the-art models across web, medical, finance, and government domains.

• →CONE addresses a key limitation in large language models by improving their ability to understand and process numerical data effectively.
• →The model uses a novel composite embedding algorithm that integrates numerical values with their units and attribute names to capture semantic meaning.
• →Testing across diverse domains shows significant performance improvements, with up to 25% gain in Recall@10 metrics.
• →The research demonstrates that treating numbers as simple text tokens is inadequate for optimal AI performance on numerical tasks.
• →CONE's approach could enhance AI applications in finance, healthcare, and other data-intensive sectors requiring numerical reasoning.