MeshFIM: Local Low-Poly Mesh Editing via Fill-in-the-Middle Autoregressive Generation

MeshFIM introduces a Fill-in-the-Middle autoregressive framework that enables local editing of low-poly meshes without regenerating entire structures. The technology allows targeted mesh region refinement while preserving surrounding geometry, addressing a critical limitation in current mesh generation workflows through specialized techniques including boundary enforcement, topological preservation, and a gated geometry encoder.

MeshFIM represents a meaningful advancement in 3D mesh generation by solving a fundamental inefficiency in autoregressive model workflows. Traditional AR models for mesh generation operate on an all-or-nothing basis—any dissatisfaction with a local region forces complete regeneration, destroying satisfactory structures and consuming unnecessary computation. This new framework enables surgical precision in mesh editing by conditionally regenerating target regions while preserving context, a capability highly relevant for iterative 3D design workflows.

The technical approach addresses mesh-specific challenges that generalist language models cannot handle. The five design components—boundary vertex markers, context positional embeddings, expanded context width, context augmentation, and a gated subtraction geometry encoder—work synergistically to enforce exact attachment along boundaries, maintain topological order, and prevent generation overflow. The gated subtraction mechanism is particularly innovative, leveraging differences between reference surfaces and existing meshes to focus generation effort precisely on missing regions.

Industry implications extend across 3D content creation, CAD systems, and game development pipelines. Interactive brush-based editing and automatic defect repair capabilities significantly reduce friction in mesh refinement workflows, potentially accelerating asset production timelines. For developers working with low-poly geometric data, MeshFIM eliminates costly regeneration cycles, translating directly to computational savings and faster iteration cycles.

Looking forward, this framework's effectiveness across mesh refinement, repair, and whole-mesh-generation-plus-stitching scenarios suggests broader applicability. Future developments might integrate MeshFIM into real-time 3D editing tools or expand capabilities to high-poly geometry, further transforming professional 3D workflows.

• →MeshFIM enables targeted local mesh editing without requiring full regeneration of entire structures
• →Five complementary design components solve mesh-specific challenges including boundary enforcement and topological preservation
• →Gated subtraction mechanism focuses generation on missing regions by leveraging reference surface differences
• →Framework demonstrates superior performance across mesh refinement, repair, and whole-mesh generation tasks
• →Direct applications include interactive brush-based editing and automatic defect repair for low-poly meshes