Weight Space Representation Learning via Neural Field Adaptation

Researchers have developed a new approach using multiplicative LoRA (Low-Rank Adaptation) weights for neural field representation learning, achieving improved quality in reconstruction, generation, and analysis tasks. The method constrains optimization space through pre-trained base models, creating structured weight representations that outperform existing weight-space methods when used with latent diffusion models.

• →Multiplicative LoRA weights can serve as effective representations for neural fields by constraining optimization space through pre-trained base models.
• →The approach demonstrates high representation quality with distinctive and semantic structure across 2D and 3D data tasks.
• →When integrated with latent diffusion models, multiplicative LoRA weights enable superior generation quality compared to existing weight-space methods.
• →The research spans reconstruction, generation, and analysis applications, showing broad applicability of the technique.
• →The method induces structure in weight space through low-rank adaptation, offering a novel perspective on neural network representations.