I am zero surprised.
this post was submitted on 19 Feb 2025
110 points (100.0% liked)
Privacy
1831 readers
129 users here now
Welcome! This is a community for all those who are interested in protecting their privacy.
Rules
PS: Don't be a smartass and try to game the system, we'll know if you're breaking the rules when we see it!
- Be civil and no prejudice
- Don't promote big-tech software
- No reposting of news that was already posted
- No crypto, blockchain, NFTs
- No Xitter links (if absolutely necessary, use xcancel)
Related communities:
Some of these are only vaguely related, but great communities.
founded 5 months ago
MODERATORS
No way... you're telling me a free AI is profiting off my data?
Always run AI locally!
Is that feasible for someone with an office PC with integrated graphics? Asking for a friend.
If you have a lot of RAM, you can run small models slowly on the CPU. Your integrated graphics I would guess won't fit anything useful in it's vram, so if you really want to run something locally, getting some extra sticks of RAM is probably your cheapest option.
I have 64G and I run 8-14b models. 32b is pushing it (it's just really slow)
Don't iGPUs use the RAM as VRAM directly? You'd only need to configure how much in the BIOS (eg. by default it uses 1.5GB of 8GB or smth and you can set it to 6/8GB)
Yes for gaming, but for LLMs I've heard that the bandwidth limitations of using system RAM as vram hurts performance worse than running on the CPU using system memory directly, since smaller models are more memory bandwidth limited.
I've never tried to run AI on an igpu with system memory though so you could try it, assuming it will let you allocate like 32GB or more like 64GB. I think you'll also need a special runner that supports igpus.
Yeah, AI is even being trained in data provided by the Nazi Steve Huffman's website.
What!? What a complete and utter shocker!!
Tbf, I don't use any of these corporate llms for exactly that reason. At best, they just use user interaction to "improve" the models, and they're more likely using it to profile and track them as well. Fuck that.
Surprised pikachu face.
Everyone had it wrong! It was the Chinese Government stealing your data to give to TikTok!
What? But the tankies assured me this wouldn’t happen!