Dynamic Global Illumination for Interactive Gaussian Splatting Scenes in Real Time

Abstract

We present a real-time pipeline to approximate global illumination in interactive or dynamic scenes, including both 3D Gaussian models and conventional meshes. Building on a formulated surface light transport model for 3D Gaussians, we address key performance challenges through a fast compound stochastic ray-tracing algorithm and a hardware 3D Gaussian rasterizer, and we implement multiple RTGI (real-time global illumination) techniques for 3D Gaussians. Our pipeline enables real-time rendering of interactive scenes featuring editable materials, lights, meshes, and 3D Gaussian models, effectively capturing multi-bounce diffuse and one-bounce glossy light transport. Our approach covers a wide range of dynamic light sources, including area lights, directional lights, and environmental lighting. Comprehensive experimental results demonstrate that our approach can efficiently render global illumination for both 3DGS models and hybrid models combining meshes and 3DGS, all in real time. Our pipeline highlights the potential of 3D Gaussians in real-time global illumination, and offers insights into performance optimization.

Method

Real-time global illumination needs two things: a way to describe how light travels between 3D Gaussians, and a way to compute it cheaply, every frame. We contribute both — a simple light-transport model for 3D Gaussians, and fast ray-tracing / rasterization building blocks that make it practical.

Overview of the RTGI pipeline — The pipeline at a glance: Gaussians and meshes are rasterized into a shared G-buffer; direct light is sampled from a light grid; indirect light is gathered by tracing rays and cached in screen probes and a world-space hash grid; glossy reflections are traced separately. Everything is filtered and accumulated over a few frames.

A light-transport model for 3D Gaussians

Along a ray, we treat every Gaussian as a tiny semi-transparent surface and blend their light front-to-back — exactly the way 3DGS rasterization blends colors. This gives a light-transport equation that is cheap enough for real time and works directly with models from existing 3DGS inverse renderers.

Fast stochastic ray tracing

Testing every Gaussian a ray touches is too slow, so we flip a weighted coin at each hit instead: opaque Gaussians are likely to block the ray, transparent ones to let it through. The result is noisy but correct on average, and much faster — the pipeline's filters take care of the noise. Rays also try a cheap screen-space walk first before falling back to full tracing.

Hardware Gaussian rasterization

Every Gaussian is turned into a small hexagon and drawn by the GPU's hardware rasterizer instead of a custom CUDA kernel. Hardware interpolation of per-vertex depths yields smooth surfaces and clean normals, fast enough to rebuild a full G-buffer every frame.

Illustration of a shading ray traced by the stochastic algorithm — A shading ray randomly accepts Gaussian hits based on opacity — noisy per frame, correct on average.

Stochastic vs reference ray tracing comparison — Averaged over a few frames, our stochastic tracing matches the reference exactly — with much higher ray throughput.

Hexagon generation for hardware rasterization — Gaussians become 2D hexagons for the hardware rasterizer, with depths stored at the corners.

Reconstructed normals comparison — Corner-interpolated depths (right) give much smoother normals than constant depths (left).

Results

43 FPS1920×1088 teaser scene
1.57M Gaussians, RTX 3090

200+ FPSsingle-object relighting
at 1280×720

100×faster than R3DG offline relighting
(≈5 ms vs. ≈700 ms / frame)

32.87 / 0.94avg. foreground PSNR / SSIM
vs. R3DG

Consistency with offline relighting

Comparison between R3DG and our method under environment lights — R3DG offline relighting (left of each pair, ≈700 ms/frame excluding visibility baking) vs. our real-time rendering (right, ≈5 ms/frame) under environment lights. Our method stays highly consistent with R3DG while being two orders of magnitude faster.

Global illumination components

GI decomposition of the Family model in a Cornell box — “Family” in a Cornell box: full GI (1) decomposes into direct diffuse (2), indirect diffuse (3) and glossy reflectance (4).

Mutual indirect light transport between Armadillo and Dragon — Mutual indirect transport between a 3D Gaussian Armadillo and a mesh Dragon — the Dragon picks up a green tint, the Armadillo an orange hue.

Dynamic lighting

Relighting under various environment, area and directional lights — Real-time relighting under environment (top), area (middle) and directional lights (bottom). Area lights produce soft shadows; strong directional lights scatter indirect diffuse lighting onto walls and neighboring models. All lighting changes happen live in the application.

Material editing & fully dynamic scenes

In-application material variation — Material parameters (albedo, roughness) edited on the fly under identical lighting.

Dynamic scene with complex light settings — A fully interactive scene: objects and lights can be transformed, scaled, inserted or removed in real time, with consistent GI throughout.

Diffuse & glossy under diverse lighting

Three Armadillos with varying roughness — Three Armadillos of varying roughness: a glossy one lit by skylight through a wall opening, rough ones by multiple area lights.

Light bouncing inside a 3D Gaussian barn — Light bouncing within the barn model itself: damp grass reflects the walls; shadowed walls receive scattered light from sunlit walkways.

Performance Analysis

Measured on the teaser scene (1.57M Gaussian primitives in the view frustum) at 1920×1088 on an RTX 3090: 43 FPS on average. Gaussian rasterization — scaling linearly with total fragment byte size — is the main bottleneck, while stochastic ray tracing scales sub-linearly with model size and remains a minor cost.

Time consumption breakdown by shader type and lighting type — Per-frame time budget by shader type and by lighting effect (mesh rasterization <0.1 ms excluded).

Average color-only rasterization cost at 1920×1088 (ms)
	bonsai	garden	drjohnson
3DGS (Kerbl et al.)	4.02	11.74	9.31
FlashGS	1.48	4.05	1.77
Ours	2.21	6.96	2.88

Our hardware rasterizer sits between the original CUDA software rasterizer and the heavily optimized FlashGS. Comparison is for reference only due to non-equivalent implementations.

Robustness & limitations

Synthetic stress-test scenes — Stress tests: thin Gaussian occluders with no light leakage, interpenetrating Gaussian geometry, and complex Gaussian–mesh occlusions all render stably without noticeable artifacts.

Failure case on blurry reconstructed Gaussians — Failure case: loosely defined, “blurry” reconstructions mislead the sparse screen probes and darken the result (ours, right) compared to R3DG (left). Temporal ghosting and glossy flickering from temporal filtering also remain future work.

Citation

@article{hu2026dynamic,
  author  = {Hu, Chenxiao and Gai, Meng and Wang, Guoping and Li, Sheng},
  title   = {Dynamic Global Illumination for Interactive Gaussian Splatting Scenes in Real Time},
  journal = {IEEE Transactions on Visualization and Computer Graphics},
  year    = {2026},
  volume  = {32},
  number  = {7},
  pages   = {6919--6931},
  doi     = {10.1109/TVCG.2026.3689199}
}

Video