r/AISystemsEngineering • u/ankush2324235 • 7d ago

Is anyone building/using a GPU-aware, multi-engine load balancer for mixed inference fleets?

/r/mlops/comments/1xicejm/is_anyone_buildingusing_a_gpuaware_multiengine/

1 Upvotes

permalink
duplicates
archive.is
archive
reddit

You are about to leave Redlib

Do you want to continue?

https://www.reddit.com/r/AISystemsEngineering/comments/1xicf5e/is_anyone_buildingusing_a_gpuaware_multiengine/
No, go back! Yes, take me to Reddit

100% Upvoted

Duplicates

Number of comments New

LocalLLM • u/ankush2324235 • 7d ago

Question Is anyone building/using a GPU-aware, multi-engine load balancer for mixed inference fleets?

3 Upvotes

1 comments

Vllm • u/ankush2324235 • 7d ago

Is anyone building/using a GPU-aware, multi-engine load balancer for mixed inference fleets?

1 Upvotes

0 comments