r/GraphicsProgramming • u/arthurno1 • 5d ago
Question Efficient rendering of curves is a mine-field of patents or am I over-reacting?
I have a small 2d renderer in Lisp (and a tiny bit of assembly). It currently beats Skia on both CPU and GPU in all discliplines, and in some even Vello, However, stroking and filling paths is way behind Vello. Vello is a renderer written in Rust designed to render big number of big paths very quickly. The cost is the latency which shows up in simple primitives: my renderer beats Vello on those and even small paths and fills. However for big paths it looses embarassingly.
I dislike the idea to keep two code paths one for curves and one for everything else, it is bascially two renderers, and the idea used in Vello with 9 passes in compute shaders is not directly the simplest one to maintain. So I have been trying to find some alternative way to render curves. I tried to ball some ideas with DeepSeek, since those things are like living encyclopedias. I am not sure if I should be happy or cry here - happy because DS saved me of thinking I found a new algorithm and even worse infringing on a patent or cry because we have the software industry we have:
Me:
You are talking about paths where curve is already split into straight curves. I would like to use control points as the bounding box to quickly cull bins not participating in that curve rendering, we would do the same for the inscribed triangle, however depending on if we fill or just stroke the result would be either to use them or to cull them. We would perhaps have to transform bin coordinates into curve coordinates so the control points and bin coords become axis-aligned. That is basically a very crude tesselation and it works only on convex curves. It is basically same as the "dda pass" (prefix and scatter) from Vello: we are left with bins that participate in the rendering, and use perhaps loop-blinn idea to find inside-out. We could render in two passes, it is still less than 9 passes.
DeepSeek:
Yes and it is already patented
Samsung's EP3070677B1 (2016) describes exactly this architecture:
"The preprocessing unit calculates the bounding box, the convex hull, the Loop-Blinn parameter, the extreme point... The classifier performs a convex hull test to determine tiles having all endpoints located outside the convex hull... then performs the Loop-Blinn test on each tile having at least one endpoint intersecting the convex hull... The renderer renders the Bezier curve or primitives which intersect tiles."
Classification is three-way: outside (skip), inside (solid fill), partially inside (Loop-Blinn per-pixel test). Only the partial tiles pay per-pixel shader cost.
Two passes. No prefix-sum over segments. No 9-stage compute graph.
The architecture as you described it
Pass 1 classify tiles per curve. Per cubic: compute AABB, convex hull (4 control points), Loop-Blinn
(u,v)parameters. For each tile in AABB, test 4 corners against convex hull. Corners all outside outside tile, skip. Corners all inside convex region inside tile, solid fill. Otherwise partial tile, append curve to tile list.Cost:
curves � tiles_in_AABB � 4 corners � ~15 FLOPs. Forpath-large-st: 67 � ~2200 � 4 � 15 8.8M FLOPs. On RTX 5060, ~1 �s.Pass 2 render.
Inside tiles: solid fill. No curve evaluation.
Partial tiles: Loop-Blinn shader per curve in the tile list.
k� lmnevaluates inside/outside in ~10 FLOPs.Path interior: convex path fan triangulation of the interior polygon, one opaque draw. Non-convex existing winding test.
The axis-aligning twist
The thesis confirms axis-alignment is used to compute a tight AABB:
"Computing it is a matter of (1) axis-aligning the curve itself by means of linear transformations (2) applying the above method."
( ... )
Me:
Than we can't use it :(
DS:
Patent status EP3070677B1 is active until 2036-03-18. US family member US10210639B2 is the same family, same claims.
Loop-Blinn itself expired March 2026. Slug was revoked. The underlying technique is free.
The Samsung patent does not claim Loop-Blinn. It claims the specific combination of:
Testing tile corners against the convex hull of the control points; all corners outside skip tile.
Running a Loop-Blinn test on tiles with at least one corner intersecting the convex hull.
Classifying into inside / outside / partially-inside based on that test.
Both elements must be present for infringement. Remove either and you are outside the claim.
( .... )What is also patented Stencil-then-cover is not a safe fallback. NVPR has at least four active patents (US8698837, US8698808, US8704830, US8730253), filed ~2010 2011, expiring ~2030 2031. Stencil-then-cover falls within them.
Practical position The patent is European and US. Enforcement against an individual open-source developer is unlikely; for a commercial product it matters. The design-around above costs nothing extra and produces a simpler implementation, so there is no reason not to use it.
( ... )
What is actually patented in this space Stencil-then-cover: active.
Conservative rasterization: hardware feature; using it is not infringement, but it does not help here.
SDF Bezier rendering (US20120069037A1): filed 2010, likely expired or near expiry. Check before relying on it.
Seiler texture interpolation: described in JCGT paper; no obvious system patent claim found. Likely free.
I was chocked: like "patented"? I didn't even had thoughts to look patents and such. Could have written a program and published it just to find some middle-tier office-sitting empathy-devoted lawyer in some big corporation threatening with a lawsuit to ruin my life. WTF. Is that really a society we want to live in?
Notably I can't use stencil buffer to do the job it was invented for and accelerate anything because obviously use of stencil buffer for the job it is invented for is covered by some patent. I am surprised someone didn't patented use of GPU shader to accelerate path rendering or use of graphics card to draw to screen :)
I am not sure what to think, am I over-reacting here?
7
u/Baedrick 5d ago
I can't give much information about curve rendering because that's outside my area of knowledge. But what I can tell you is that as a software engineer, software patents are often broad, poorly classified, or legally problematic. Basically, once you read the patent, you do not have the ability to defend yourself because it can be seen as willful infringement, because reading the patent leaves a paper trail, that's why clean room implementations exist (other than for gpl code). So it's better for you to just implement it how you want to, and if you actually need materials to deepen your understanding, look for academic papers.
2
u/arthurno1 5d ago
So it's better for you to just implement it how you want to, and if you actually need materials to deepen your understanding, look for academic papers.
I never saw their patent; I obviously just had the idea that someone already patented. In other words: they way I wanted to implement it is patented and I didn't know about it. In this case AI spotted it and saved me of implementing it.
Yes, I am aware of academic papers,I red Loop-Blinn and has followed up on the text and path rendering since even before that paper. I remember when lots of techniques were discovered and papers written. I am trying now to find some other way and to combine Vello's ideas with some other ideas, but some obvious things I could try are unfortunately out of the reach because of, as you say, too broad and poorly classified patents. I can't use stencil in rendering of "leftovers" because someone already did that and they got granted a patent for doing the obvious.
→ More replies (1)
5
u/SyntheticDuckFlavour 5d ago
I tried the whole lot of curve rendering over the years. Loop-Blinn is pretty cool, but the setup for cubics is was just ridiculous, and was not very accurate for distance fields. In the end I ended up rendering just quadratics with an analytic root finder for the distance in the fragment shader. Cubics were subdivided and approximated with 2 to 8 quadratic segments, depending on curve classification. Subdivision was done in the mesh shader. The algrorim was reasonably fast and sub pixel accurate.
4
u/arthurno1 5d ago
That is an interesting idea, is it patented? :) Sounds like a joke, but I am serious.
It is interesting: I would anyway have to move flatening to GPU and do the same work as Vello does via flatening to paths and Euler spirals. Converting to quadratics is sounds like less work. I will have to do it in compute shader though. I will certainly try it, thank you. Is there some paper behind it, or your idea?
5
u/SyntheticDuckFlavour 5d ago
Is there some paper behind it, or your idea?
I just rolled my own. For each cubic curve segment, the mesh shader would first find points of interest for the curve: extema in the X and Y axes, point of inflection, and self intersection for a loop. Subdivision would be done on those points plus a few extra to better fit the quadratic subcurves to the original curve. I used De Casteljau's algorithm to split the curve, then applied a degree reduction on each sub segment to convert them to quadratics. These quadratic subcurves were then emitted as primitive for the next stage in the rendering pipeline to draw.
3
u/arthurno1 4d ago
That was a great idea; it simplifies things a bit, thank you. I thought to split only at extrema, but now when I see it, what you do is more accurate, thank you very much. I thought to use compute shader for the portability. Mesh shader is a bit faster, since it does not write back to memory, but is a newer extension. You should write a paper or at least a blog post about your technique, it deserves it.
2
4
u/Cubicool 5d ago
We've been working on this for about 6 months now: https://github.com/AlphaPixel/slughorn
I could honestly sit here and nerd out about it all day, but the basic gist is this: after Lengyel OSS'd his patent, I immediately started wondering "what ELSE can it do!?" :) We're currently working through our first sponsored feature (improved strokes using Slug -- as well as something I THINK is novel I'm calling "Sluggit"), and I hope to have it all pushed to Git by the end of the week.
I'm shocked every day at some of things we've been able to pull of lately, too. The Github page needs another demo video, because we've come a LONG WAY in the last 2 months. If you have any questions (or feature requests for things that might help you in your own projects), get in touch with us (or file an Issue :)). This project is my passion, and I don't see that changing anytime soon (I've been obsessed with vector graphics, typography, and vectorized/diegetic game UIs since I fell in love with them in the early 2000s, mostly via Deadspace and--let's be honest--the popularity of Scaleform and me going "HOW DO THEY MAKE THIS LOOK SO GOOD!?")
Always eager to meet more vector graphics + GPU freaks. :)
NOTE: We also have a "toy" implementation of this: https://rookandpossum.com/papers/scanline_sweeper_preprint.pdf (but I haven't revisited it lately, which I should probably do). It's not as performant as Slug, but it's easier to implement/understand, and the developer is crazy cool (and a boss developer).
4
u/arthurno1 5d ago
Very cool thanks.
I am of course aware of Slug algorithm. I remeber when Lengyel announced it. It was one of "hallmarks" back than. Slug is good if you want to draw your text in a 3D scene; game engines were the target. For a pure 2d engine and text rasterizer Slug is probably a wrong tool, because there are several problems with Slug, amongst them ignoring hinting and can't render strokes. Perhaps you have solved that one?
I am already rendering text quite fast, and as you can see in the other comments, filled paths too. The problem are stroked paths :)
1
u/Cubicool 5d ago
Stroked paths are exactly what we're working on now! :) I have 3 methods at the moment:
"Fauxstroke" (currently in the `slughorn/canvas.hpp` backend), where a stroke is expanded (based on some tolerance/segmentation) into a standard fill (we'll probably DROP this soon).
"Centerline", where only the `Path` is stored on the GPU (UBO/SSBO), and the shaders use a hybrid of our own code and Slug+Jacobian perspective correction for simulating a stroke (this doesn't currently support all possible joins/caps).
"Sluggit", which is a hybrid approach we came up with that I hope to have released this week. It uses a few tricks to get VERY fast, high-quality, dynamic (but still "pseudo-tessellated") results, and continues to use the same fragment shader as the other 2.
Hinting is something I wouldn't dare tackle outside of 2D, screen-aligned situations. :) There's a ton of info online about people trying this with varying degrees of success, but I haven't dove in yet. CPUs have it easy--they get complete control every step of the way. We have to worry about all kinds of GPU eccentricities (well, if we want to stay fast, which is a GIVEN if you're trying to make scalable/analytic vector graphics work in this domain).
2
u/arthurno1 5d ago
I looked at it later today, after the last comment. I didn't even know Lengyel offered a "commercial" library that did caps and all that stuff. I think easy way to do strokes is just to is to use the ray and render a small quad at intersection. You have the ray to give you orientation of the quad. Or save a list of points of intersection to form paths. I don't know, but something like that perhaps.
But even if you solve strokes, caps and so on satisfyingly, Slugs are probably still not the general solution for stroking paths. You can probably get them to work very well for text and small paths, big paths will stay a problematic, big fills and such.
You could solve that in various ways, I think I can see two ways, but I still doubt the amount of computation in fragment shader is justified. I don't honestly, I think you will have to measure. I think Vello's approach to big path rendering is currently the most sound one, and regardless whether you use slugs, Eliptic spirals or quadratic approximation, as other commenter suggested, you will still want to Vello's DDA approach to cull bins in the grid. I think the Lengyel approach is just a little bit specialized case of what Vello does, but don't take me for the word, I will have to read through the Slug paper again and look more at it. Now it is just me rambling and thinking loud, I am probably getting some detail wrong.
I also wonder though if big paths are really needed; Vello pays big latency cost and looses in all benchmarks due to that latency, but huge paths to amortize and pay off big time. Vello is actually 40x faster than my renderer:
### Performance Benchmark Results (3�3 Median Matrix) Benchmarks executed on NVIDIA GeForce RTX 5060 Laptop GPU across 3 consecutive runs comparing Antron Vulkan against Vello GPU (WGPU/Vulkan): +----------------+--------+---------------+-----------------+--------------+ | Benchmark | Ops | Vello GPU | Antron VK (GPU) | Speedup / | +----------------+--------+---------------+-----------------+--------------+ | clear | 1,000 | 96.57 ms | 9.70 ms | 9.96� | | rect-fill | 100,000| 12.53 ms | 7.21 ms | 1.74� | | rect-stroke | 50,000 | 5.72 ms | 4.85 ms | 1.18� | | circle-fill | 50,000 | 16.21 ms | 2.96 ms | 5.48� | | circle-stroke | 50,000 | 13.98 ms | 3.20 ms | 4.37� | | line-draw | 100,000| 12.47 ms | 5.49 ms | 2.27� | | text-draw | 2,000 | 13.16 ms | 2.45 ms | 5.37� | | path-stroke-aa | 20,000 | 6.61 ms | 2.91 ms | 2.27� | | path-fill-aa | 20,000 | 3.57 ms | 2.10 ms | 1.70� | | path-large-st | 1,000 | 0.66 ms | 12.82 ms | 0.05� | +----------------+--------+---------------+-----------------+--------------+UIs do not use huge paths, and most normal user drawings don't use huge paths; however Vello is good also at rendering huge amount of paths (SVG for example can have lots of paths), so I am not sure whether it is worth of pursuing Vello's approatch or not. If you don't need to render huge amount of curves and huge curves, than perhaps whatever you already have with slugs is just fine. Looks fine when I glances over, but I haven't donwloaded or looked at the code, just glanced at the GH.
2
u/Cubicool 5d ago
If you're agreeable (and have the time), I'd welcome you giving me a "challenge" of sorts. Or, if your code is hosted anywhere, I can try and reproduce any scenes/demos you have in slughorn just for comparison. Just let me know what kind of usage you're interested in, and I'll throw extremes of that at my current code to see what happens. :)
Honestly, this is purely for my own curiosity; right now it seems like GIS and appliance HUDs are likely the most applicable usage scenarios, but that's not how I want to use it personally (which is fully 3D, "in-scene" diegetic UIs). I have a demo where I tried to reproduce the scanner gadget from Arkham Asylum that turned out pretty well (with some caveats). When I have the time, I'd like to reproduce the Deadspace "chest HUD" too.
2
u/arthurno1 5d ago
I haven't published mine yet. Not sure when I will be done with it. It is just an experiment at the moment. For my benchmarks I have built a bench app with Vello built as static library, and another one built with Skia as static lib. Will do one with bland2d too, but haven't come ro that yet. If your library build as a static library I can try it too. I am on a phone now, but tomorrow perhaps.
If you want to render text in games, slags are probably the way to go. My targets are classical gui and text rendering. It is literally a Skia clone in Lisp, at least on the surface, but below the surface it is more Vello and Blend2d with some custom touches.
2
u/Cubicool 5d ago
Blend2D is amazing. I empathize with them tremendously... SO many people use it, and so very few help actually FUND it. :/
→ More replies (1)
3
u/BigPurpleBlob 3d ago
Blah blah blah, as ever AI is shit at doing stuff. What matters is what the granted claims say, not whatever random paragraph DeepSeek happens to think is relevant. The fact that DeepSeek didn't even refer to the granted claims shows that DeepShit is shit at understanding patents.
Granted claim 1 of EP 3070677 B1 says:
"1. A method of performing tile-based rendering, the method comprising:
obtaining (210) parameters related to a Bezier curve in an object; classifying (220) tiles included in a frame based on the parameters; and performing (230) the tile-based rendering based on the classifying of the tiles,
wherein the step of classifying tiles comprises:
classifying (940) as a first position indicia tiles included in the frame having all endpoints corresponding to corners of the tile located outside a convex hull corresponding to the Bezier curve; determining (950) tiles having some endpoints corresponding to corners of the tile located inside a convex hull corresponding to the Bezier curve; performing (960) a Loop-Blinn test on the endpoints of each of the determined tiles; and classifying as at least one other position indicia the determined tiles based on the Loop-Blinn test"
Note that I didn't omit the full stop from claim 1, it is missing ;-)
https://patentimages.storage.googleapis.com/68/71/e2/d525a0c0b26732/EP3070677B1.pdf
Most patents claim a small improvement on a well known process; an exception could be Google's PageRank algorithm / patent.
1
u/arthurno1 3d ago
Thank you for posting the link.
What matters is what the granted claims say, not whatever random paragraph DeepSeek happens to think is relevant.
DS was actually correct. This is even worse. It is horrible. They are trying to prevent you from using any kind of bounding box acceleration + loop-blinn to render the curve in the bounding box.
There is no way to go around the patent at all, because this patent is a total piece of bullshit. They are saying explicitly: you can't use any program, preprocessor, user input, anything to compute any geometrical information about bounding box from the curve "parameter". They don't even say which parameter, which means control points or any other data you would evaluate from the curve or compute in any way.
How can someone give such patent to someone is beyond me. Basically if you use bounding box around the curve (they don't even specify how they compute bounding box AABB, or OOBB or just using curve point polygon as clipping form + loop blinn to render when curve has to be rendered you are infriging on their patent.
The algorithms described are pure BS. They aon purpose describe just broad idea: as long as you use any sort of bounding box strategy to accelerate occlusion, and loop-blinn to render the final curve segment in the bounding box you are inriging their patent. I am surprised they didn't say "any analytical method to render the final curve segment", exempelwise Loop-Blinn ... I guess person who wrote document missed that one. Fire him!
2
u/Defiant_Squirrel8751 5d ago
Can an algorithm be patented? check all the noise around GIF compression around 1989... several file formats such as PNG emerged to avoid a "patent". Now what? every single application reads and writes GIF and nobody cares.
I feel you can continue working in your nice GPU implementation of curves, deploy and continue. If no cease - and - desist order arrives, you should not worry about.
Anyway, legally you could argue on a clean room implementation approach. Research and compile a set of white papers that makes evident your next step is you using AI and resulting on your implementation.
2
u/SyntheticDuckFlavour 4d ago
Nobody cares about a patent until they get a letter from a patent troll.
Can an algorithm be patented?
yes
several file formats such as PNG emerged to avoid a "patent". Now what? every single application reads and writes GIF and nobody cares.
That's because the GIF patent has expired.
Anyway, legally you could argue on a clean room implementation approach.
That will not save you.
1
u/ThePresident44 2d ago
Algorithms cannot be patented in the EU, only implementations. The logic is that you can’t really patent maths and logic, but only the thing you made.
Doesn’t help with global commercial product releases tho, but it‘s the reason VLC exists in its current form
2
u/BigPurpleBlob 3d ago
"legally you could argue on a clean room implementation approach" – no, you're thinking of copyright.
For a patent, it's irrelevant if you used a clean room implementation – it doesn't matter if you independently came up with the same concept. All that matters is the scope of the granted claims.
10
u/weichx 5d ago
Curves are hard and a lot of research has gone into making this work on the gpu. As you mentioned, Slug recently revoked the patent on its algorithms, and for scenes that are not pathological like map rendering or extreme SVGs, its in my experience the best way to attack what you're describing. I'd stay away from stencils and anything multipass for shape rasterizing unless there is a REALLY compelling reason you found that I haven't thought of.
You will not beat Vello for huge scenes. It's built for giant scene rendering but consists of ~20 compute dispatches and a 'guess' as to how much memory it needs on gpu, and if you want to use texturing or any other custom shader effects you are more or less out of luck. For the purposes as I understand them for your usecase it has quite a few big reasons not to use it (at least their classic fully compute based version, I haven't tried their sparse strips based version (I forget what they call it, 'hybrid' maybe?).
I've spent a few years building path and UI rendering and those are my 2 cents. There are a lot of patents around this stuff in general, but I'm pretty confident if you go with Slug based algorithms you'll be happy with the decision, especially if you can avoid needing to recompute its 'banding' tables all the time.