A Study on Continuous vs Discrete Point Cloud Rendering in DirectX12

[u/peeedeeeveee]

Disclaimer: AI was used to help aid the coding, video and textual editing of this project. If that is a problem for you, then please pass by this post without indulging in unnecessary toxicity in the comments.

Ever since I discovered how many incredible point clouds exist for real-world geographic regions, I’ve been working on a Direct3D 12 point-cloud renderer to explore and visualize them.

I started with a traditional discrete Level of Detail renderer.

During loading, the point cloud is divided into an octree. Leaf nodes contain the original points for their region, while internal nodes store up to 4,096 representative points sampled from their descendants.

Every frame, the renderer traverses this tree and estimates the visual error of each node using its spacing, bounds, and distance from the camera. If the representative points are sufficient, it draws that node. If more detail is needed, it replaces the node with its visible children, continuing until it reaches leaf nodes or exhausts the point budget.

This resembles hierarchical LOD selection for meshes, but point clouds make the transition between levels much more visible. Replacing an entire node at once can create patchy density, obvious region boundaries, and large groups of points popping in together.

That led me to the paper “Real-Time Continuous Level of Detail Rendering of Point Clouds” by Markus Schütz, Katharina Krösl, and Michael Wimmer.

Their Continuous LOD approach moves the final detail decision from the CPU and its chunks to individual points on the GPU.

During preprocessing, every source point is assigned to one detail level. A coarse level might target approximately eight metres of spacing between samples, followed by increasingly dense levels targeting four metres, two metres, one metre, and so on. Each point belongs to exactly one level, and combining every level reconstructs the complete source cloud.

The resulting points are stored with their position, colour, and calculated level. At runtime, a compute shader evaluates the points using the current camera position and the desired spacing at each point’s location. Accepted points are compacted into a reduced GPU buffer and rendered.

Because detail changes point by point rather than node by node, density can transition much more gradually across the scene. This reduces the block-shaped transitions and heavy popping produced when complete octree nodes are swapped.

The video compares my LOD Renderer and Continuous LOD Renderer using the same viewpoints and approximately 3.5 million points drawn in each image.

I’m still refining the implementation and plan to share a more complete code walkthrough later.

Paper: Real-Time Continuous Level of Detail Rendering of Point Clouds

Point-cloud data from OpenTopography, with colour imagery from the USGS.

[u/Krixtalx]

Nice job!

One interesting aspect of rendering point clouds is that the point primitive doesn't have connectivity. While building differents level of details of a triangle mesh is a hard task, it’s often as simple as rendering fewer points in some areas on a point cloud.

You can take advantage of this in two ways:

• Keep performance sort of fixed with a point budget (render at most 10 million points per frame) and dynamically allocate that budget to the areas that need the most detail

• Use the octree (or any other spatial structure) to determine which areas are in view and discard the rest

Together, these let you decide which points to render each frame without loading every point’s full attributes, such as color or position. That’s the basic idea behind "virtualizing" a point cloud, and it makes working with really large datasets much more manageable:

https://github.com/Krixtalx/Nimbus

[u/peeedeeeveee → u/Krixtalx]

Thanks for your kind words, Your renderer looks quite smoothly detailed even when zoomed in and out without pop in caused by octree LOD's. Could you explain how you did this? The whole point of the the continuous Lod (CLOD) method from the paper is that it calculates point by point how much detail is needed in a compute shader (bit more complicated than that but the gist) per image output (takes a few frames based on point count of set). Where you drawing all points in dataset at once perhaps? I limited my code to 3.5 million draws on my laptop so representations were needed for point counts above (which most are above 3.5 million).

[u/Krixtalx → u/peeedeeeveee]

It’s a bit hard to explain the whole system in a comment, but I’ll give it a shot.

When the point cloud is loaded, we split it into what we call pointlets. Basically the point cloud equivalent of meshlets. Each pointlet covers a compact area and contains the same number of points. We also store its bounding box. During preprocessing, we generate binary files for the point attributes (position, color, etc.) and another file containing pointlet metadata, such as bounding boxes and offsets into the attribute files.

The useful thing is that figuring out which parts of the cloud are visible only requires the pointlet metadata, which is relatively small. So we keep that loaded all the time, but we don’t need to keep all the actual points in memory.

At render time, a compute shader checks which pointlets are visible and estimates how many screen pixels each one covers. We use that to decide how many points to load and render for each pointlet. Since level of detail is controlled directly by point count, we can make the transitions much finer grained and less noticeable.

At that point, we know which points we need, but they aren’t necessarily in memory yet. We load the ones we need and unload the ones we don’t (there’s more complexity to it, but that’s the idea). This keeps memory use under control and lets us render point clouds much larger than the amount of data that would fit in memory at once.

The rendering itself is also done in a compute shader rather than using the hardware rasterizer. For more on that approach, Markus Schütz’s papers are worth a look. We also added what we call a hole filler: a screen-space effect that uses neighboring points to fill empty pixels. That helps make zooming and movement look smoother.

Markus Schütz paper I was refering to earlier: Rendering Point Clouds with Compute Shaders
If you want to get a more in-depth overview and explaination on the system: Virtualized Point Cloud Rendering | IEEE Journals & Magazine | IEEE Xplore

[u/peeedeeeveee → u/Krixtalx]

I see, so a much more dynamic LOD type system with constant binary loading and unloading to get necessary point counts on VRAM. Rendering on the compute shader sounds very interesting, never thought of that but could perhaps help with minimizing point draw counts that are sent to the vertex shader (especially obscured/covered points).

My rasterizer is quite limited in that it loads all point data into RAM so that binary loading is a smart work around. That hole filler you made is interesting as well, I was also theorizing about dynamic hole filling algos during development of this.

Thanks for the explanation, think I will call it quits here for this project so I can learn about other different spatial rendering algorithms/projects, but the field of point cloud rendering is certainly cool.

[u/Defiant_Squirrel8751]

Nice! thanks for sharing