Hi Ben! I would argue that it is very useful for splats to be edited in this way. I couldn't have built this application without SuperSplat for isolating, cleaning, transforming and optimizing/compressing the PLY:

https://playcanv.as/e/p/cLkf99ZV/

Integrating AI is an interesting topic and something that certainly has potential.

I don't like the "byte code" analogy for Gaussian splats. If they were like that, then we could apply compiler optimization and those sorts of math techniques to them. But they are Probability Distribution Functions with transforms, so the math tools we have to work with them are the similar to those in signal processing -- resampling, quantizing, estimating, etc.

In that model, we don't compile them, we train them; we don't run them, we sample/rasterize them.

This link came up on HN before and was a great refresher/expander on the math of Guassians which allow all this. [1].

Since Gaussians can be estimated, neural networks can model/generate them. Researchers are using this for 4D work and mesh extraction. The NNs run at lower frame rate informing the 3DGS running at interactive rates.

You are right that it is ephemeral and really a weird trick of the eye and we need new ways to edit/create it. Vectors/pixels have had a lot more time to grow tooling. People are working on it, just the toolbox is different. Very cool stuff will be coming up, I bet!

[1] https://news.ycombinator.com/item?id=41912160 I've also re-learned Fourier transforms to appreciate similar concepts.

I don't really think this is true. Gaussian splats certainly came from a context where an opaque representation is expected and normal, but they ended up being an entirely comprehensible format. They're not as simple to operate on as an SDF or voxel representation, but I think they're on par with triangle mesh geometry. A transformed fuzzy sphere is about as complex as a triangle, and spherical harmonic colors, while more conceptually difficult than textures, have fewer moving parts.

I guess in theory what you say could be correct, however in practice this tool has been very helpful for client work of editing, cleaning, cropping and even slight modification of Gaussian splats. I could see a similar argument for raster images in general -- they are hard to edit as you're modifying individual pixels and it's not efficient, but we've seen tools grow from MS Paint to modern Photoshop to become very useful. I think the same could be said here -- it's just early and we're at the "bytecode" level as you say.

I'm not really sure what you mean. Think of SuperSplat as the photoshop of gaussian splats?

- SuperSplat dev :)

That's a ridiculous take. Generated splats almost always have garbage parts that need to be truncated. An editor is absolutely needed for that.

Relightable gaussian splats - https://junxuan-li.github.io/urgca-website/

I found it interesting, I'd heard of guassian splats but not really appreciated how they worked, but this let me play with a model; so I'm not saying necessarily useful but instructive.

There are already approaches to infer bidirectional reflectance distribution functions (BRDFs, the "true surface properties") and lights: https://nju-3dv.github.io/projects/Relightable3DGaussian/

great metaphor, thanks!

Whats the performance like?

Aren't gaussian splats incompatible with most common game development styles? No shadows, no rigging/animation as main issues - or maybe I'm misunderstanding / behind on the research - please correct me.

Not a reading guide, but Computerphile have a good introductory video on Gaussian Splats: https://youtu.be/VkIJbpdTujE?si=8hoMbMx6tKuMZo2S

Individualkex also has a couple videos on the high level ideas: https://youtu.be/GQXDjzNWuPc?si=zlAN7dO9STGATKad

The blog posts from Aras are a really good starting point: https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr...

I wish all these guides used better notation. They all use the scariest possible Greek symbols (pi epsilon feta etc) that are hard to make sense of without a degree in anthropology, instead of nicely named variables like programmers use.

PolyCam I think is the most popular?

Polycam or Scaniverse

Not sure if I'm missing something, but the submission title says "open-source" and in the tool's help menu there's a link to the repo (https://github.com/playcanvas/supersplat), the tool runs in your browser, there's no server involved besides a Web-Server hosting the files.

I found its repository after searching google. The license is MIT.

[1] https://github.com/playcanvas/supersplat

I remember a time when it was considered unpolite to ask a question without googling first. Is it still the case?