AINeutralarXiv – CS AI · 10h ago6/10
🧠
What Does a Chemical Language Model Know About Molecules?
Researchers used sparse autoencoders to mechanistically analyze MolFormer, a chemical language model, revealing that it learns meaningful molecular semantics beyond surface-level syntax. Early layers track molecular grammar through position-encoding, while deeper layers capture pharmacologically relevant atomic features, with non-canonical SMILES notations causing more disruption than invalid ones due to cascading positional errors.