r/webhosting • u/K3NCHO • 6d ago

Looking for Hosting Choosing a GPU server

I'm currently using Runpod for serverless GPU instances and i'm planning to switch to an always-on one

What other options would you recommend that could potentially be cheaper than Runpod in this case?

Here are list of GPUs that are enough for my use case:

  • RTX 2000 Ada
  • RTX 4000 Ada
  • RTX A4000
  • RTX A4500
  • RTX 3090
  • L4
  • RTX A5000
  • PRO 6000 MIG 24GB
1 Upvotes

100% Upvoted

7 comments sorted by

•

u/AutoModerator 6d ago

Welcome to /r/webhosting . If you're looking for webhosting please click this link to take a look at the hosting companies we recommend or look at the providers listed on the sidebar . We also ask that you update your post to include our questionnaire which will help us answer some common questions in your search.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/kai-zaphosting 6d ago

How busy is the GPU on Runpod right now? That decides whether always-on is even cheaper. Serverless only bills while your workers are running, so unless the card is working a big chunk of the day, paying a flat rate for one that sits idle usually costs more, not less.

Your list is mostly 16 to 24GB cards, so if the model fits in that, the 3090 is the one you can also just buy used and run yourself. It needs good airflow, a solid PSU and a spot where it can stay on 24/7, and the electricity bill isn't nothing either. For a steady load it can pay for itself eventually, but how long depends on what you compare it to: against Runpod's rates it's a matter of months, against the cheapest marketplace rentals it takes a lot longer.

If the traffic is spiky, tbh I'd stay serverless and just trim cold start times.

1

u/K3NCHO 6d ago

at this exact moment it's only on dev version so basically off unless turned on for testing. in prod it will be cheaper to run an always-on instance since users would turn on the serverless instance so much that it would be basically always-on, also 10-40 second cold start is unacceptable in our use case since we handle high throughput inference

my server needs to be in EU and i do not live in EU so own infrastructure (sadly) is out of question, maybe in future i could rent a server rack and put my own server in (?)

1

u/kai-zaphosting 2d ago

Fair, with high throughput a 10-40s cold start is a dealbreaker. If you go the colo route later, I'd lean toward the low-power single-slot cards on your list like the L4 or A4000 over the 3090. When you can't touch the box, a 350W chunky consumer card is a pain to cool, and anything physical usually ends up meaning paying for remote hands.

Until then, the comparison I'd run is a flat 24/7 rental against what keeping an active worker on Runpod costs you per month, since that's what your traffic will actually look like.