VoxStruct3D: Structure-Leading Flow Matching for Voxel-Space 3D MRI Synthesis

Fang Li1 Yang Gao1,✉ Shihao Zou2 Weixin Si3 Hongyu Wu1 Qing Xia1 Shuai Li1 Aimin Hao1
1Beihang University 2Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences 3Shenzhen University of Advanced Technology
Paper · soon Code
02

Abstract

Latent diffusion makes volumetric generation tractable, but its image autoencoder introduces a reconstruction bottleneck. We present VoxStruct3D, a voxel-space flow-matching framework that directly models full-resolution MRI volumes with a clean-data prediction objective.

Its Volumetric Voxel Generator (VVG) couples neighboring tokens through overlapping upsampling, refinement, and skip fusion. Structure First, Image Follows (SFIF) supplies an explicit anatomical prior using compact structure tokens, a structure-leading schedule, Patch-Aligned RoPE, and asymmetric attention.

On pathological and healthy T1-weighted brain MRI, VoxStruct3D delivers the strongest overall balance of feature-distribution alignment, sample diversity, and perceptual quality.

03

Why voxel space?

Direct generation preserves the original signal, but it exposes two problems that a plain patch-based model cannot solve on its own.

Issue 01

Patch-boundary artifacts

Independent block reconstruction leaves visible seams where neighboring volumetric patches meet along entire 2D faces.

Issue 02

Missing structural guidance

Without an autoencoder’s compact abstraction, the model must recover global anatomy directly from a high-dimensional corrupted volume.

Ablation showing block artifacts without VVG and malformed anatomy without SFIF
Complementary failure modes. VVG removes block seams; SFIF prevents anatomically implausible structure.
04

Structure first. Image follows.

A shared model follows two synchronized clocks: the compact anatomy stream stays ahead while the image stream forms in the original voxel domain.

VoxStruct3D framework with image and structure streams
Stage II overview. PA-RoPE aligns unequal token grids and asymmetric attention enables one-way structure-to-image guidance.
VVG

Volumetric Voxel Generator

A DiT backbone handles global token interactions while an overlapping volumetric decoder lets neighboring tokens jointly reconstruct shared voxel regions.

  • Overlapping 3D upsampling
  • Time-modulated refinement
  • U-Net-style skip fusion
SFIF

Structure First, Image Follows

Compact 3DINO-derived tokens act as an internal anatomical guide that develops ahead of, and only informs, the voxel stream.

  • Structure-leading clocks
  • Patch-Aligned RoPE
  • Asymmetric attention
Patch-Aligned RoPE and asymmetric attention mask
Aligned and one-way. Image and structure tokens share a common spatial lattice; image queries can read structure keys, while the reverse path is blocked.
05

Strong across anatomy and detail

Across pathological and healthy T1 brain MRI, VoxStruct3D achieves the best overall distribution alignment, diversity, and perceptual quality.

Pathological · 3DINO FID ↓ 12.89 Best distribution alignment
Pathological · MS-SSIM ↓ 0.7413 Best sample diversity
Healthy · 3DINO FID ↓ 8.476 Best distribution alignment
Healthy · MS-SSIM ↓ 0.7320 Best sample diversity
Qualitative comparison of generated pathological brain MRI volumes
Qualitative comparison. VoxStruct3D preserves global brain structure and clearer anatomical boundaries across axial, coronal, and sagittal views.