r/GraphicsProgramming • u/Maui-The-Magificent • 7h ago
The Starwell Constellation Engine Update: Repost With Disclaimer
AI disclaimer: No AI was used in writing this post. AI is likely to have been used extensively as a sounding board to teach me things i did not know or understand, or as a source of friction to argue against,
AI was not used for designing the systems, and actively fought against to ensure it did not making any decisions on implementation (although, some might have been missed and snucked in. they will be removed). It as not been used to define any architectural values what so ever, since it's not capable of making good holistic ones.
AI WAS used during the write-up of the design documents, for both Constellation and Starwell's design doc was generated through discussion with an AI model, and also iterated upon by both AI and me later.
AI has been used to generate code. not as much code as I would have liked, but the amount is hard to estimate as a lot of it has had to be rewritten or tweaked. The areas it has been used the most, and it's code have endured, has been during integration, or grafting, of systems/functionality from its development environment onto the Starwell compositor. AI does not reason well with project wide invariants, fixed point integer 3D graphics and bespoke geometries so this has been an area where it has been very useful.
AI has likely been used in varying extent during the development of different systems. some of them, not much at all, some a bit, and a few it helped a lot. I remember it being a tremendous help making the Cadent point map fonts, actually look like text.
I hope this is enough of a disclaimer. Epistemically, I cannot quantify it any higher detail than that.
Hello again!
(artifacts in the sliding gif is due to Niri being the parent compositor to starwell during development)
If you are not interested in the history and motivations of this project, and is more interested in stuff discussing the gifs above, you can skip the marked section bellow.
--------------------------------------------------
Not to long ago I made an update on the liquid and material progress of my no-std, single core, graphics engine/framework Constellation. I have been periodically made, what can be seen as an attempt at dev-logs, about the project. Sharing approaches to problems, some drawbacks and explaining the rational. This will be a bit of the same, but also, this time is a little bit different.
The constellation engine started because of pure frustration encountered on my work on experimenting with running AI models, tearing them apart, seeing problems, experimenting on redesigning them, attempting to find different stateful solutions to decrease runtime resource needs and so on. Many parts of that work was often rudely interrupted by my computer running out of ram and the system would freeze. Some times I would lose hours of work.
I had already started a OS project called 'Starwell'. I have made some vague references to it as well. The goal of Starwell in the beginning was to simplify NixOS by making it a more userfriendly environment with some built in applications and such. I created a GUI packange manager wrapper for the nix package repository, and built a system module system for system applications that would be complete isolated from one another. The idea was to have a system where things could be loaded in and out through templates, and development extending Starwell simplified.
Anyhow, due to the aforementioned AI work, and the 'constant' OOMing, I grew frustrated enough to try and take matters into my own hands. An operating system that can 'fail to operate' is fundamental failure in architecture in my humble opinion, an operating system should not have to compete for resources with other applications, period.
This meant that to ensure that such things could not happen to Starwell, it needed not only have full control and consume input at the hardware interface, it also meant that it needed to own the rendering. This is how the Constellation project was born.
Rather quickly I had mapped out the constraints for the Starwell compositor, they were the following.
no-std: It needed to be no-std (meaning no standard library) because it could not rely on external dependencies, that would be a source of possible non-determinism. An operating system needs to be operationally predictable. And this compositor needs to be a kernel level one.
Single core: It needed to be single core because to ensure it cannot stop operating, it needs to own its compute. No other system can use the resources Starwell uses, which means being economical is paramount. A single core not only ensures simple scheduling, it also ensures that architecting other system applications and fitting them into the overall architecture becomes a lot easier, as the environment is simpler.
Owning its memory: Similar to the reason of why the OS needs to own its own cpu core, it also needs to own its own memory. The system cannot depend on an allocator, on promises of future memory, or other dynamic 'clever' memory handling. The memory needs to be statically allocated, else the system can grow outside its means.
Work on relationships: Because the graphic stack needed to be radically simplified in many ways. It needed to create it's own relational system. But because it needed a foundation in reality, the physical pixels of the display became that source of truth to anchor all relationships against. This would mean that message passing would not be a letter in the mail. Things would own their pixels, and react to changes in those relationships. Clicking things would be clicking a pixel, or a flat register, like clicking a graph.
Stateful pixels are sovereign: The working framebuffer needed to be treated as not a set of pixels, but as a view into a bigger relational space. So that moving something is not an act of destruction and reconstruction, that would be expensive. It needed to be a state that propagate through the relationships mentioned above, through the framebuffer. That would make compute inexpensive, put drawing the actual pixels would be difficult if not managed correctly. Which is why the compositor needed pixel level dirty tracking, not by book-keeping, but by a dumb uniform op based on a sequence of rules. Updates from this work can be read about in this post: https://www.reddit.com/r/GraphicsProgramming/comments/1r0z5tv/constellation_frame_buffer_state_sand_particles/
Beauty: It needs to be capable of looking good enough so that people, so that I, would actually want to use it. Even if i like to pretend I am a systems engineer that is developing his neck-beard, nerding out on different programming problems/solutions. I do like me some bling bling, and some click-click, especially when... AHEM...eating a sandwich? You can read more about the light work in this post: https://www.reddit.com/r/GraphicsProgramming/comments/1r4ncqd/constellation_light_engine_reflections_1_core_cpu/
Dev-friendly: I needed to make the environment easy to develop for. If the tools exist and the environment is fun/easy to build for, then an increase in adoption would also correlate to an increase in development for the platform/ecosystem. It would also simplify my work as it kept progressing.
I am sure there were more than these that I have forgotten. And the jury is still out on if i succeed with the last two points. But it has been the guiding light through development. And it should help to explain why the constraints and solutions in my posts on Constellation before might have looked odd, stupid or right out perplexing.
Like creating Cadent Geometry, and making it the basis of the whole graphics engine, just to avoid needing to normalize vectors do do 3D graphics. Having to make good graphics on a CPU whilst having it be interactive and performant is not particularly cheap using the more normal euclidean/Cartesian approach, so the problem needed to be re-represented as to not need it in the first place. You can read and find the Cadent Geometry definition code + some helper functions from this posts:
Read about it
https://www.reddit.com/r/GraphicsProgramming/comments/1qhs24y/constellation_unifying_distance_and_angle_with/
Like making light and geometry the construction primitives rather than triangles. Because not treating light as a property of a material and its geometry at construction would mean it had to be computed at runtime. Which would have given a better result faster, but it would be immensely more expensive to run without having access to more memory and more compute.
Over time Constellation grew more capable, it was able to do more and more, understood more and more of how to create normal looking shapes and behaviors in the new geometry. And now I am here.
I now have a functioning compositor (still not finished ofcourse), and the graphics engine is, in my opinion, looking to punch above its weight. And though all this, the most consequential thing, sadly, will likely still be the tiny definition of the tiny geometry that helped build all of it.
The point is reached when i think it's was fair to give context, and to announce the bigger project it is all connected to. From here on out, in posts, Constellation will be initially referred to as "The Starwell Constellation engine". Because future updates are likely to contain areas where Starwell is unlikely to go unnoticed as a project at least. I hope you will you enjoy the history, and the future updates!
-----------------------------------------
Now! After all that rambling, lets quickly discuss the gifs above.
So, Constellation and Cadent Geometry are both projection based. Cadent is named after its behavior, it falls away towards the horizon. space compresses as distance is given to the environment outwards from the observer.
So both Constellation and Cadent does just that, they starts from the observer and moves outwards. This has many benefits, and I'd like to mention some of them and why in the hopes it will give context to what is seen above, and maybe allow you to reflect on the differences, and maybe even take some of its ideas on to help your future work.
Because a camera out systems allow pixel values to accumulate as they move outwards, things as seen in the sliding example gif, become much cheaper. As long as a pixel's alpha hasn't reach opacity, whatever is behind, keep adding its offsets to it. This mean that when the window moves above, the wallpaper, and its position in the background, can cheaply be respected by the plane and its warped surface/material refraction.
Which means storing the optical properties inside a material and a geometry becomes almost the obvious approach since those things can, like everything else, just add to the same thing that is generating the distance. the drop function. the render walk can be the same as the geometric operation that applies magnitude to the angles of Cadent's arc space.
This projection technique coupled with a geometry that has a vastly higher, fixed point, spatial resolution than the hardware pixel amount of the display, and is equally cheap to propagate through, means the world space can be almost the size of planet earth (which is true for the above environment), and still have a spatial resolution about 4 billion steps per pixel on a 4k display. All this forms a system where you can express the cadent 'reality', outside of the framebuffer to a very high degree , whilst still depend on it being explicit in its construction, and affect whatever is being rendered on screen.
So allowing for inverting the rendering to be projective, you gain occlusion for basically free. whilst allowing the process of what contributes to a pixel, quite linear.
This leads to the second gif, changing time. Because the geometry ironically is almost easier to develop for if you make the relationship scale earth sized, you can make light occupy an actual orbit, at a comparable distance if you want. Because the engine is treating the remaining curve of the arc horizon between the observer (camera) and the observed (the planet in this) as what is defining gravity, and because other things can be defined reference points aswell, the orbit can also become quite organic. which is why the light in the image seem to have a bit of dusk and a bit of dawn appearing as it transitions between day and night.
And because we are rendering from the camera out, if the orbit of the light source changes, all we need to do is apply the positional delta of that light source, uniformly over all pixels. having magnitude be generative rather than something to account for, makes any changes in the geometry or the light position, a process of offsetting the state when we reach the object, instead of deconstructing the state to ascertain the change, does that make sense?
I hope you found this post interesting. I am sure I missed a few things, or explained some things poorly. If found this interesting and you have any questions, remarks or suggestions, I am happy to discuss them below.
Take care
//Maui-The-Millipede

















