Technically correct (tm)
Before you get your hopes up: Anyone can download it, but very few will be able to actually run it.
this post was submitted on 24 Jul 2024
212 points (100.0% liked)
Technology
70498 readers
2621 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
This would probably run on a a6000 right?
Edit: nope I think I'm off by an order of magnitude
"an order of magnitude" still feels like an understatement LOL
My 35b models come out at like Morse code speed on my 7800XT, but at least it does work?
When the RTX 9090 Ti comes, anyone who can afford it will be able to run it.
That doesn't sound like much of a change from the situation right now.
So does OSM data. Everyone can download the whole earth but to serve it and provide routing/path planning at scale takes a whole other skill and resources. It's a good thing that they are willing to open source their model in the first place.
Wake me up when it works offline "The Llama 3.1 models are available for download through Meta's own website and on Hugging Face. They both require providing contact information and agreeing to a license and an acceptable use policy, which means that Meta can technically legally pull the rug out from under your use of Llama 3.1 or its outputs at any time."
WAKE UP!
It works offline. When you use with ollama, you don't have to register or agree to anything.
Once you have downloaded it, it will keep on working, meta can't shut it down.
Well, yes and no. See the other comment, 64 GB VRAM at the lowest setting.
Oh, sure. For the 405B model it's absolutely infeasible to host it yourself. But for the smaller models (70B and 8B), it can work.
I was mostly replying to the part where they claimed meta can take it away from you at any point - which is simply not true.
It's available through ollama already. i am running the 8b model on my little server with it's 3070 as of right now.
It's really impressive for a 8b model
load more comments
(4 replies)
I'm running 3.1 8b as we speak via ollama totally offline and gave info to nobody.
I was able to set up small one via open webui.
It did ask to make an account but I didn't see any pinging home when I did it.
What am I missing here?
Through meta...
That's where I stop caring
Yo this is big. In both that it is momentous, and holy shit that’s a lot of parameters. How many GB is this model?? I’d be able to run it if I had an few extra $10k bills lying around to buy the required hardware.
Kind of petty from Zuck not to roll it out in Europe due to the digital services act.. But also kind of weird since it's open source? What's stopping anyone from downloading the model and creating a web ui for Europe users?
Did anyone get 70b to run locally?
If so what, what hardware specs?
That looks good on paper, but while I find ChatGPT good to create critical thinking, I've found Meta's products (Facebook and Instagram) to be sources of disinformation. That makes me have reservations about Meta's intentions with LLMs. As the article says, the model comes pre-trained, so it's most made up of information gathered by Meta.
Neither Meta nor anyone else is hand-curating their dataset. The fact that Facebook is full of grandparents sharing disinformation doesn't impact what's in their model.
But all LLMs are going to have accuracy issues because they're 1) trained on text written by humans who themselves are inaccurate and 2) designed to choose tokens based on probability rather than any internal logic as to whether an answer is factual.
All LLMs are full of shit. That doesn't mean they're not fun or even useful in some applications, but you shouldn't trust anything they write.
view more: next ›