GaussianSplats3D

• Clean up Viewer.dispose() behavior as well as abort behavior for splat scene loading functions
• Small improvements to the way 8-bit spherical harmonics compression is implemented
• Allow splat scenes to be transformed like ordinary three.js objects, demo can be found in demo/dynamic_dropin.html

• Support 2D Gaussian splatting scenes: https://surfsplatting.github.io/, activate with Viewer parameter splatRenderMode: GaussianSplats3D.SplatRenderMode.TwoD
• Fixed rendering bug with spherical harmonics in scenes with large number of splats
• Fixed scaling bug with spherical harmonics
• Properly apply scene opacity and visibility when scene is loaded
• Fixed bug where call to remove splat scene would never complete if there was only one splat scene loaded
• Fix double dispose() call in Viewer.removeSplatScene()
• Added Viewer.removeSplatScenes()

• Updates targeting memory usage:

  • Add ability to set the compression level of a .ply that's loaded into memory for rendering (not export to .ksplat) via the Viewer parameter plyInMemoryCompressionLevel. Previously it would be stored as an uncompressed SplatBuffer, but now the compression level can be chosen and valid values are the same as when exporting to .ksplat: 0, 1, or 2. Default is 2.
  • Add option to free intermediate splat data after a scene is loaded via the Viewer parameter freeIntermediateSplatData. When true, the intermediate splat data that is the result of decompressing splat bufffer(s) and is used to populate the data textures will be freed. This will reduces memory usage, but if that data needs to be modified it will need to be re-populated from the splat buffer(s). Default is false.
  • Reduced amount of memory needed by the sorting worker

• Add ability to import reduced-memory version of INRIA .ply files (more info here: https://repo-sam.inria.fr/fungraph/reduced_3dgs/)

• Split spherical harmonics into multiple textures for large scenes

• Add ability to toggle splat scene visibility or set the opacity of individual splat scenes.

• Fixed bug preventing Viewer.addSplatScene() or Viewer.addSplatScenes() from working after scenes have already been added.

• Fixed sorting crash on iOS < 17.0 where SIMD is not supported in web assembly modules

• Fixed bug with progressively loading dynamic scenes

Breaking changes:

• Changed Viewer.addSplatScene() parameter: streamView to progressiveLoad

• Added support for 1st & 2nd degree spherical harmonics
• Added Viewer constructor parameter to enable: sphericalHarmonicsDegree. Default value is 0.
• Updated .ksplat format to include spherical harmonics
• Added new compression level for .ksplat files (2) that is identical to (1) except spherical harmonics are compressed to 8-bit
• Various bug fixes

• Fix WebXR stereo mode not rendering correctly
• Fix floating-point splat sort in orthographic mode
• Add parameter to control console logging
• General cleanup

• Fix WebXR stereo mode not rendering correctly
• Fix floating-point splat sort in orthographic mode
• Add parameter to control console logging
• General cleanup

• Fixed bug with Viewer.removeSplatScene() when there was only a single splat scene loaded (Thanks @pesa2000 for the heads up and help with testing)
• Fixed create-ksplat.js script (Thanks @gotoeasy for finding this bug and providing the fix!)

• Support orthographic rendering, enabled via Viewer.setOrthographicMode()
• Added SplatMesh.setSplatScale() to set the value by which splats are scaled in screen-space (default is 1.0)
• Added SplatMesh.setPointCloudMode() to enable/disable point-cloud mode, where each splat is rendered as a filled circle
• Added function to remove a splat scene: Viewer.removeSplatScene()
• Fixed stuttering that would sometimes occur when progressively loading scenes with gpu-accelerated sort disabled.
• Fixed bug preventing the clearing of the screen before each render

• Support anti-aliased rendering mode, as described in this Nerfstudio thread: https://github.com/nerfstudio-project/gsplat/pull/117. This will only work for models that have been trained with this mode enabled. Activated via Viewer parameter 'antialiased': true.
• halfPrecisionCovariancesOnGPU is now false by default. It was causing issues for very small models.
• Fixed bug with floating-point splat distance computation on GPU.
• Fixed bug with scene reveal not completing for very small models.

Bug fixes:

• Scenes not fully revealed in standard loading (non-progressive) mode
• Vite integration errors due to esbuild not properly handling static class members in web workers

Updates:

• Removed hard-coded maximum render distance
• Added documentation to README about CORS with Vite
• Added Viewer constructor parameter for choosing scene reveal mode: sceneRevealMode

• Support PlayCanvas' compressed .ply format.
• Support progressive loading for standard .ply files and compressed .ply files.

Fixed a bug with loading .ply files using Viewer.addSplatScene() and Viewer.addSplatScenes()

The Viewer class now accepts a new parameter renderMode, which lets you control when the viewer renders. It is of the type RenderMode:

const RenderMode = {
    Always: 0,
    OnChange: 1,
    Never: 2
};

• Always is the default and will cause the viewer to render every frame.
• OnChange will make the viewer only render when a visual element in the scene changes (camera position, camera orientation etc.)
• Never prevents the viewer from doing any rendering

Additionally the new function Viewer.setRenderMode() allows this setting to be changed at runtime

Small bug fixes for the progressive loading feature:

• Ensure loading progress bar is hidden when scene download is complete
• Ensure minimum splat alpha threshold is respected

Support progressive loading for .ksplat and .splat files. Support for streaming .ply files will be included in a future release.

• Enabled via the streamView option to addSplatScene. It is false by default to preserve existing behavior.
• Currently streaming is only supported with the loading of a single scene; streaming of multiple scenes will be added in a future release.
• Enable Viewer.dispose() to be called while scene loading is in progress

Beaking Changes:

• Updated .ksplat format so that it can be streamed, and to allow for better compression, multiple scene sections, and an optimized layout that yields better sorting performance. Older versions of .ksplat files will no longer be supported.
• showLoadingSpinner parameter for Viewer constructor is now showLoadingUI

Thanks to @gotoeasy for helping me test this feature out and find bugs!

• Built-in support for AR and VR modes via WebXR, enabled via a Viewer constructor parameter: webXRMode. Thanks to @dlazares and @chrisirhc for doing the ground work.
• Added AR/VR demo in demo/vr.html
• Cleaned up and added documentation to the rendering code in order to better explain Gaussian-related computations.
• Cleaned up dependency issues in package.json. Thanks to @sxxov for pointing those out.
• Fix bug with specifying format when loading scene where SceneFormat.Splat was ignored. Thanks to @kgenots for finding the issue.
• Upgrade to Three.js v160

• Added Viewer.dispose() to explicitly dispose of resources directly and indirectly held by the viewer.
• Added parameter to scene loading functions to force splat scene format (.ply, .splat, or .ksplat).
• Cleaned up raycasting code, now supports raycasting directly at splat ellipsoids.
• Fixed dynamic scenes demo to begin altering scene transforms after the viewer is ready.

Small release containing a fix for a bug that caused scene transforms to be incorrectly applied.

This update adds the ability to apply real-time transformations to splat scenes. Prior to this release, splat scenes could have a custom transformation applied when the scene was added to the viewer, after which the scene had to remain static.

Code changes

• Added dynamicScene parameter to Viewer and dynamicMode parameter to SplatMesh as a hint so that they won't make any optimizations that assume a static scene.

• Updated splat data access methods in SplatMesh so that by default they only apply scene transformations if dynamicMode is false.

• Added new API for accessing individual splat scenes. Each of the following methods returns a SplatScene object by which the position, orientation, and scale of an individual splat scene can be modified in real time.

  • SplatMesh.getScene(sceneIndex)
  • Viewer.getSplatScene(sceneIndex)
  • DropInViewer.getSplatScene(sceneIndex)

• Added demo page with dynamic scenes example: demo/dynamic_scenes.html

Breaking Changes

• Updated splat scene loading functions to have more consistent names that better describe their purpose:
  - Viewer.loadFile() changed to Viewer.addScene()
  - Viewer.loadFiles() changed to Viewer.addScenes()
  - DropInViewer.addSceneFromFile() changed to DropInViewer.addScene()
  - DropInViewer.addScenesFromFiles() changed to DropInViewer.addScenes()

• Changed Viewer.scene parameter to Viewer.threeScene to avoid confusion with the concept of splat scenes