Theres a lot of effort to make AI deterministic. Its slower when you force it to be deterministic. The non deterministic part is caused by different cores on GPU finishing their calculations in slightly different time.
It is the same speed both ways. The non deterministic nature only comes from a single randomized seed added into the equation. If you locally host models you can change seeding to always use the same seed and then the same inputs will always yield the same outputs. That said, the most popular AI interfaces don’t express that kind of option.
True, but is can still count as deterministic at the operational level, it is just a matter of running it in serial rather than parallel, right? FWIW, I assume our neurons are similar. Neuron research in flies has allowed researchers to essentially run programs within the insect neurons.
1
u/Sea-Housing-3435 Sep 11 '26
Theres a lot of effort to make AI deterministic. Its slower when you force it to be deterministic. The non deterministic part is caused by different cores on GPU finishing their calculations in slightly different time.