Light Sampling Field and BRDF Representation for Physically-based Neural Rendering

ICLR 2023

Jing Yang1,2, Hanyuan Xiao1,2, Wenbin Teng1,2, Yunxuan Cai1,2, Yajie Zhao1,2,
1USC Institute for Crative Technologies, 2University of Southern California,
Paper arXiv Video Github
teaser
teaser

Youtube Video

Abstract

Physically-based rendering (PBR) is key for immersive rendering effects used widely in the industry to showcase detailed realistic scenes from computer graphics assets. A well-known caveat is that producing the same is computationally heavy and relies on complex capture devices. Inspired by the success in quality and efficiency of recent volumetric neural rendering, we want to develop a physically-based neural shader to eliminate device dependency and significantly boost performance. However, no existing lighting and material models in the current neural rendering approaches can accurately represent the comprehensive lighting models and BRDFs properties required by the PBR process. Thus, this paper proposes a novel lighting representation that models direct and indirect light locally through a light sampling strategy in a learned light sampling field. We also propose BRDF models to separately represent surface/subsurface scattering details to enable complex objects such as translucent material (i.e., skin, jade). We then implement our proposed representations with an end-to-end physically-based neural face skin shader, which takes a standard face asset (i.e., geometry, albedo map, and normal map) and an HDRI for illumination as inputs and generates a photo-realistic rendering as output. Extensive experiments showcase the quality and efficiency of our PBR face skin shader, indicating the effectiveness of our proposed lighting and material representations.


Pipeline

method

Starting from sampling lights from HDRI and 3D points close to geometry along the light path, we feed encoded locations, view direction to Material network and Light Sampling Field network (with extra direct illumination code). The networks output material and indirect illumination code. Our PBR equation generates radiance at each sample based on our material model and light transport in the volume and further composites all the radiance on the light path to the receiver


Light Sampling Field

method

We use various densities of sampling to bring attention to locations with data as in the Light Sampling Field.

method

Appearance at three different locations under the same illumination


Qualitative Results

method

BibTeX

        
          @inproceedings {
            yang2023light,
            title       = {Light Sampling Field and {BRDF} Representation for Physically-based Neural Rendering},
            author      = {Jing Yang and Hanyuan Xiao and Wenbin Teng and Yunxuan Cai and Yajie Zhao},
            booktitle   = {The Eleventh International Conference on Learning Representations },
            year        = {2023},
            url         = {https://openreview.net/forum?id=yYEb8v65X8}
          }