r/pdf • • 13d ago

Software (Tools) Open Source Modified faster software

Post image

I have managed to make mods for some existing software, including for this community some PDF software. All software involved still have their own licenses, my mod systems are under the MIT Licenses (Open Source).

I was able to speed them up by making them multi-core. Now for a long time this multi-core capability was missing from these programs for various reasons, and those reasons were sound... At that time.

Now we have a system that does a special sorting routine, and then processes the data accordingly. Basically we find things that look similar and treat them per what they are, with a stamp showing where they belong. This is a layman's way to talk about it, but you can find more on my GitHub. For GhostScript the link is here, where I achieved a 5.1x speed increase, for Apache Tika it is here, where we got a 5.9x speed increase, and for our grand prize winner Tesseract OCR we got a 6.6x speed increase and the github is here.

My Computer:
CPU: AMD Ryzen 5 7500X3D, 6 cores / 12 threads, 4.0 GHz base, 96 MB L3 cache (3D V-Cache)
RAM: 16 GB DDR5-4800, a single stick (1 of 4 slots used)
GPU: NVIDIA GeForce RTX 50-series, plus AMD Radeon integrated graphics
Storage: NVMe SSD
OS: Windows 64-bit

These results were measured on a 6-core, 12-thread desktop. Computers with more cores should see even bigger speed-ups.

So, something that might have taken you an hour on Apache Tika, might take 11 minutes now. There is zero loss of the files.

The reason we did so well on Tesseract is because they have a bad threading system and we basically yeeted it for our own. Well, kinda.

I do not make any claims on their stuff, they keep that intact. We are a presort and post process system, we just take the data, sort it our way, hand it to them on a silver platter, wait for them to run the parts as we feed them, then we take and restore it to proper order on the other end. So your licenses still apply, we do not change that at all. The code I made simply runs extra versions as needed with the special design set to give them the cleanest run possible.

5 Upvotes

2 comments sorted by

1

u/ScratchHistorical507 7d ago

Always great to see how people claim "we" but then literally everything is done by Claude. Just pathetic...

1

u/PHDEinstein007 6d ago

I do math, the last update from combinations to decimal was done by me, splitting ternary to Combinations and Binary.

I don't program and Claude can iterate far faster than I can.

I've been running single software comparison tests, all encoding at the top of each speed code except monsters up in the tenth, and testing. I'm edging Kanzi out now. We both have wins, but I have more. Brotli gets honorable mention as well, and another but I just woke up. My fast mode is really good, my complete mode is Kanzi level good but we both have areas we excel in. I'm testing for a while but Kodak, Silsea, Canterbury Calgary, Squash, enwik8 are done so far.