this post was submitted on 07 Feb 2024
203 points (100.0% liked)

Technology

69772 readers
3686 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

Abacus.ai:

We recently released Smaug-72B-v0.1 which has taken first place on the Open LLM Leaderboard by HuggingFace. It is the first open-source model to have an average score more than 80.

top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 13 points 1 year ago

Oh yay another model I can't run on my computer :'(

[–] [email protected] 12 points 1 year ago

i thought this was about the MUD server software, and got excited. alas.

[–] [email protected] 12 points 1 year ago (3 children)

That's nice and all, but what are some FOSS models I can run on GPU with only 4GB?

I've tried Deepseek Coder, and it's pretty nice for what I use it for. Then there's TinyLlama, which... well it's fast, but I need to be veeeery exact in how I prompt it.

[–] [email protected] 8 points 1 year ago* (last edited 1 year ago) (1 children)

Unfortunately LLMs need a lot of VRAM. You could try using koboldcpp, it runs on the CPU but let's you offload layers onto the GPU. That way you might be able to stay withing those 4gb even with larger models.

Edit: I forgot to mention there's a fork of koboldcpp with rocm for AMD cards, which is about twice as fast if I remember correctly. Only relevant if you have an AMD card tho.

Edit 2: This is the model I use btw

[–] [email protected] 2 points 1 year ago (1 children)

I'm currently playing around with the Jan client, which uses the nitro engine. I think I need to read up on it more, because when I set the ngl value to 15 in order to offload 50% to GPU like the Jan guide says, nothing happens. Though that could be an issue specific to Jan.

[–] [email protected] 2 points 1 year ago (4 children)

Maybe 50% GPU is already using too much VRAM and it crashes. You could try to set it to 0% GPU and see if that works.

load more comments (4 replies)
[–] [email protected] 8 points 1 year ago

4GB is practically nothing in this space. Ideally you want at least 10GB of dedicated vram if you can't get even more. Keep in mind you're also probably trying to share that vram with your operating system. So it's more like ~3GB before you even started.

Kolboldcpp is capable of using both your GPU and CPU together, you might wanna consider that. (Using a feature called layers) There's a trade-off that occurs between the memory available and the quality of its output and the speed of the calculation.

The model mentioned in this post can be run on the CPU with enough system ram or swap.

If you wanna keep it all on the GPU check out 4bit models. Also there's been a lot of work into trying to do this with the raspberry Pi. I suspect that their work could help you out here as well.

[–] [email protected] 2 points 1 year ago

Depends on your needs. Best look around in [email protected] or similar. (I don't wanna say reddit but r/localLlama is much larger.)

If you're more into creative writing, maybe look for places that discuss SillyTavern (r/SillyTavernAI is an option). It's software for role-play chats, which may not be what you want. But the community is (relatively) large and likely to have good tips for non-coding/less technical applications.

[–] [email protected] 8 points 1 year ago* (last edited 1 year ago)

Where can we all see the leader board?
Edit: never mind

[–] [email protected] 6 points 1 year ago (1 children)

If only I had the $ to get a rig that could run this locally

[–] [email protected] 5 points 1 year ago (1 children)

OOTL: What is a LLM and what does it do?

[–] [email protected] 19 points 1 year ago (1 children)

Large Language Model AI. Like ChatGPT.

[–] [email protected] 4 points 1 year ago (1 children)

And at 72 billion parameters it's something you can run on a beefy but not special-purpose graphics card.

[–] [email protected] 8 points 1 year ago (2 children)

Based on the other comments, it seems like this needs 4x as much ram than any consumer card has

[–] [email protected] 4 points 1 year ago

It hasn't been quantized, then. I've run 70B models on my consumer graphics card at a reasonably good tokens-per-second rate.

load more comments
view more: next ›