GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures

Patrick Noras1,2, Jun Myeong Choi3, Didier Stricker1,2, Pieter Peers4, Roni Sengupta3
1University of Kaiserslautern-Landau, 2German Research Center for Artificial Intelligence,
3University of North Carolina at Chapel Hill, 4College of William & Mary
Paper arXiv Code (Coming soon!)

Abstract

Recent advances in Gaussian Splatting-based inverse rendering extend Gaussian primitives with shading parameters and physically grounded light transport, enabling high-quality material recovery from dense multi-view captures. However, these methods degrade sharply under sparse-view settings, where limited observations lead to severe ambiguity between geometry, reflectance, and lighting. We introduce GAINS (Gaussian-based Inverse rendering from Sparse multi-view captures), a two-stage inverse rendering framework that leverages learning-based priors to stabilize geometry and material estimation. GAINS first refines geometry using monocular depth/normal and diffusion priors, then employs segmentation, intrinsic image decomposition (IID), and diffusion priors to regularize material recovery. Extensive experiments on synthetic and real-world datasets show that GAINS significantly improves material parameter accuracy, relighting quality, and novel-view synthesis compared to state-of-the-art Gaussian-based inverse rendering methods, especially under sparse-view settings.

Method

GAINS method pipeline
  • We introduce GAINS, a Gaussian-splatting-based inverse rendering framework for sparse multi-view inputs on both objects and scenes.
  • We first recover geometry using priors from monocular depth, normals, and latent diffusion.
  • GAINS then estimates materials and lighting using three complementary priors: segmentation guidance, intrinsic image decomposition, and latent diffusion.
  • Combining these priors enables robust material recovery, consistent lighting estimation, and stable novel-view synthesis and relighting under challenging sparse-view conditions.

Relighting Results

tensorIR (8 views)

Material Estimation

Synthetic4Relight (8 views)

GAINS Material Estimation

BibTeX

@misc{noras2025gainsgaussianbasedinverserendering,
      title={GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures}, 
      author={Patrick Noras and Jun Myeong Choi and Didier Stricker and Pieter Peers and Roni Sengupta},
      year={2025},
      eprint={2512.09925},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.09925}, 
}