Selfhosted

49590 readers

637 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago

MODERATORS

[email protected]

Very large amounts of gaming gpus vs AI gpus (ani.social)

submitted 4 days ago by [email protected] to c/[email protected]

43 comments fedilink hide all child comments

GPU	VRAM	Price (€)	Bandwidth (TB/s)	TFLOP16	€/GB	€/TB/s	€/TFLOP16
NVIDIA H200 NVL	141GB	36284	4.89	1671	257	7423	21
NVIDIA RTX PRO 6000 Blackwell	96GB	8450	1.79	126.0	88	4720	67
NVIDIA RTX 5090	32GB	2299	1.79	104.8	71	1284	22
AMD RADEON 9070XT	16GB	665	0.6446	97.32	41	1031	7
AMD RADEON 9070	16GB	619	0.6446	72.25	38	960	8.5
AMD RADEON 9060XT	16GB	382	0.3223	51.28	23	1186	7.45

This post is part "hear me out" and part asking for advice.

Looking at the table above AI gpus are a pure scam, and it would make much more sense to (atleast looking at this) to use gaming gpus instead, either trough a frankenstein of pcie switches or high bandwith network.

so my question is if somebody has build a similar setup and what their experience has been. And what the expected overhead performance hit is and if it can be made up for by having just way more raw peformance for the same price.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 1 points 3 days ago

Products targeted towards businesses have always been unreasonably more expensive than those targeted towards consumers. It sucks for us AI hobbyists that Nvidia are stingy with VRAM on consumer cards, but I don't find it surprising.

Personally I only have a single RTX 3090, but I know a lot of people online who are stacking multiple consumer cards to run AI. Buying used 3090s and putting them in a mining rig is probably still the best value for money if you need a large amount of VRAM.

How much VRAM do you actually need btw?