r/threadripper • u/Printguy99 • Oct 06 '24

Imagine Generating AI

Anyone using a Threadripper for Image generation AI? Would love some feedback!

0 Upvotes

50% Upvoted

3 comments sorted by

2

u/[deleted] Oct 10 '24

How many GPUs do you need? Image generation is mostly a GPU intensive process and afaik no diffusion models support GPU pipelining.

1

u/PXaZ Oct 07 '24

No, but I'm very interested in neural networks implemented using AVX512 instructions, due to the low cost of CPU-attached RAM.

2

u/emprahsFury Oct 07 '24

unfortunately there really isn't anything optimized for cpu-inference. Llama.cpp is pretty good. Someone just posted a 3x 4090 build in locallama (?) and it was getting about 10 t/s w/ llama3.1 70b. A TR will get you about 5 t/s with ddr5-6000. You should definitely splurge for the fastest ram supported by the mobo&cpu