Knowledge Graph and Hypergraph Transformers with Repository-Attention and Journey-Based Role Transport

Researchers present a new transformer architecture that jointly trains on natural language and structured data by maintaining separate knowledge and language representations. The model uses a key-value repository system with journey-based role transport to enable cross-attention between linguistic context and structured knowledge graphs.