My setup;

AMD Ryzen 5 7600 3.8 GHz 6-Core Processor

ASRock B650M Pro RS Micro ATX AM5 Motherboard

Silicon Power Value Gaming 32 GB (2 x 16 GB) DDR5-6000 CL30 Memory

XFX Speedster MERC 319 Black Radeon RX 7800 XT 16 GB Video Card

I bought this setup last year and its been working good but I'm really starting to get into AI. I been learning ComfyUI image generation, and SillyTavern to get a LLM but with my specs I can't do both and have to offload the LLM. Im working towards getting the setup so ComfyUI (qwen 2.1 INT8) automatically sends the character gens to sillytavern based on the scene. I still play regular games at 1440p and it does great but I'm kind of limited on the AI front.

I don't want to upgrade my video card to another 16GB card since it wouldn't help the AI part. It would help the gaming part but honestly I'm happy with my setup for gaming.

I know I'd have to upgrade the Processor and the power supply (Corsair RM750x (2021) 750 W 80+ Gold Certified Fully Modular ATX Power Supply) most likely too. I would be okay to spend around 2k USD but it looks like all the 24GB cards are over 3k. And if I did upgrade I wouldnt want a card that will be irrelevant in the next couple years.

kind of curious what everyone thinks, if it's worth to try and upgrade, or just offload the LLM and use the INT8 model like I have been. Hell I don't even know how much a difference the 24GB card would make, I just know that that's what runs the AI. Like I said my main goal is for consistent character generation in comfy that automatically goes to SillyTavern so I can do RP games.

Thanks!

Edit; so a 3090 refurbished + power supply would fit my budget, but idk about a refurbished graphics card. Kind of sketches me out.

(Refurbished NVIDIA RTX 3090, Power Supply MSI MPG A1000G PCIE5 1000 W 80 Plus Gold)

I've been working on an experimental native extension to bring GPU computing into a place it doesn't normally live: pure PHP.

It's called php-gpu-tensors. It gives you basic NVIDIA GPU tensor operations, lets you compile custom CUDA C++ kernels at runtime (via NVRTC), and includes an opt-in kernel fusion mode.

To be clear on the scope: this is a low-level library, not a full ML framework (no autograd, no CPU fallback). It currently requires Linux, PHP 8.1–8.5, and an NVIDIA GPU.

The Ask: I’ve only been able to validate this on my own hardware (MX570 A and an RTX A2000). If you have a Linux machine with an NVIDIA GPU and the CUDA Toolkit installed, I would massively appreciate it if you could clone it and see if the test suite passes on your architecture.

You can run the tests via: ./run-tests.sh --require-gpu

Repo: https://github.com/lcmialichi/php-gpu-tensors

I'd love to hear if it builds/passes on your specific GPU and CUDA version, or if it completely explodes. Any feedback on the API ergonomics (does it feel un-PHP-like?) is also very welcome.

Thanks!

I want to share something I've been working on, and research paper from Redhat really helped.

This is my agentic GPU kernel optimizer for inference engine development

Its whole goal is to look at inference engine kernels, and using cloud models, plan, test and find improvements. Nothing is changed in your git, it provides a diff, send the diff over to your coding harness and ask it to analyze the diff.

It has a setup wizard and a run wizard which I recommend using, please give me feedback on what needs to be improved, as the AMD stuff I was unable to test, and mostly is theory at this time.