Learn What A Gaussian Splat Is, Then Make One

Gaussian Splats is a term you have likely come across, probably in relation to 3D scenery. But what are they, exactly? This blog post explains precisely that in no time at all, complete with great interactive examples and highlights of their strengths and relative weaknesses.

Gaussian splats excel at making colorful, organic subject matter look great.

Gaussian splats are a lot like point clouds, except the points are each differently-shaped “splats” of color, arranged in such a way that the resulting 3D scene looks fantastic — photorealistic, even — from any angle.

All of the real work is in the initial setup of the splats into the scene. Once that work is done, viewing is the easy part. Not only are the resulting file sizes of the scenes small, but rendering is computationally simple.

There are a few pros and cons to gaussian splats compared to 3D meshes, but in general they look stunning for any kind of colorful, organic scene. So how does one go about making or using them?

That’s where the second half of the post comes in handy. It turns out that making your own gaussian splats is simply a matter of combining high-quality photos with the right software. In that sense, it has a lot in common with photogrammetry.

Even early on, gaussian splats were notable for their high realism. And since this space has more than its share of lateral-thinkers, the novel concept of splats being neither pixels nor voxels has led some enterprising folks to try to apply the concept to 3D printing.

Move Over, Lithophane: 3D Printed 3D Photos With Gaussian Splats

If you had asked us yesterday “How do you 3D Print a Photo”, we would have said “well, that’s easy, do a lithophane”– but artist, hacker and man with a very relaxing voice [Wyatt Roy] has a much more impressive answer: Gaussian splats, rendered in resin.

Gaussian splats are a 3D scanning technique aimed at replicating a visual rather than geometry, like the mesh-based 3D-scanning we usually see on Hackaday. Using photogrammetry, a point cloud is generated with an associated 3D Gaussian function describing the colour at that point. Blend these together, and you can get some very impressive photorealistic 3D environments. Of course, printing a Gaussian smear of colour isn’t trivial, which is where the hacking comes in.

Continue reading “Move Over, Lithophane: 3D Printed 3D Photos With Gaussian Splats”

Generating 3D Scenes From Just One Image

The LucidDreamer project ties a variety of functions into a pipeline that can take a source image (or generate one from a text prompt) and “lift” its content into 3D, creating highly-detailed Gaussian splats that look great and can even be navigated.

Gaussian splatting is a method used to render NeRFs (Neural Radiance Fields), which are themselves a method of generating complex scenes from sparse 2D sources, and doing it quickly. If that is all news to you, that’s probably because this stuff has sprung up with dizzying speed from when the original NeRF concept was thought up barely a handful of years ago.

What makes LucidDreamer neat is the fact that it does so much with so little. The project page has interactive scenes to explore, but there is also a demo for those who would like to try generating scenes from scratch (some familiarity with the basic tools is expected, however.)

In addition to the source code itself the research paper is available for those with a hunger for the details. Read it quick, because at the pace this stuff is expanding, it honestly might be obsolete if you wait too long.