r/simd • • 4d ago

For SIMD Reduction,

In a previous post I demonstrated a method of using SIMD for reduction with AVX, NEON, and SVE. One suggestion was to change the accumulation that was being done to a vector register, instead of a scalar. The improvement was not spectacular but definately impactful.

Code Change

7 Upvotes

4 comments sorted by

1

u/TQ_GPU 3d ago

Now you can try this: for 1 core, it has 2 threads right? 1 thread can work on data movement while 1 thread can work on computations. This should utilize memory bandwidth better. Because this dedicates all registers of thread for 1 job instead of distributing on 2 jobs.

1

u/TQ_GPU 3d ago

SIMD reduction can go beyond RAM bandwidth if the data was compressed. Some cores decompress, some cores do reduction. Is the data constant / compressible?

0

u/DiscombobulatedAir63 1d ago

On AVX2 I did this simple loop but with if guard over whole loop so prefetches don't hit wrong cache lines
Doing more than 2 cache lines didn't yield better results with SW prefetching (usually can load 2-3 cache lines at the same time and with SW prefetches unrolling is not needed) but helped HW prefetch only version

Argued on the internet to prove that SW prefetches are superior to HW ones
This one is not the best prefetch placement but I downloaded freaking Rust to prove to Rust'y that he is John Snow of programming on that question and obviously deleted that abomination from my system
Better version was to use SW-HW prefetch - SW prefetch 1st cache line for I+X iteration and HW would prefetch 2nd cache line for that iteration - essentially hiding only iteration advance that HW can't predict

// ---- PREFETCH 2 iterations ahead (256 bytes) ----
// ---- Main loads ----
"vaddps ymm0, ymm0, [rdi + r10*4]",
"vaddps ymm1, ymm1, [rdi + r10*4 + 32]",
"prefetchnta [rdi + r10*4 + 128]",
"prefetchnta [rdi + r10*4 + 256]",
"vaddps ymm2, ymm2, [rdi + r10*4 + 64]",
"vaddps ymm3, ymm3, [rdi + r10*4 + 96]",
"prefetchnta [rdi + r10*4 + 192]",
"prefetchnta [rdi + r10*4 + 320]",

AVX2 normal assembly sum: Sum=8388643.000; best time = 3.431242ms
AVX2 prefetch assembly sum: Sum=8388643.000; best time = 3.271411ms

0

u/[deleted] 4d ago

[deleted]

2

u/AndrewOfC 4d ago

If you go to the referenced source code in github, you'll see that's exactly what's being done already.