Technology

69947 readers

3251 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago

MODERATORS

[email protected]

160

Someone made a GPT-like chatbot that runs locally on Raspberry Pi, and you can too (www.xda-developers.com)

submitted 1 year ago by [email protected] to c/[email protected]

12 comments fedilink hide all child comments

all 17 comments

sorted by: hot top controversial new old

[–] [email protected] 31 points 1 year ago (2 children)

Direct link to the GitHub repo:
https://github.com/nickbild/local_llm_assistant?tab=readme-ov-file

It's a small model by comparison. If you want something that's offline and actually closer to comparing to ChatGPT 3.5, you'll want the Mixtral 8x7B model instead (running on a beefy machine):

https://mistral.ai/news/mixtral-of-experts/

[–] [email protected] 6 points 1 year ago (1 children)

Nice! Thats a cool project, ill have to give it a try. I love the idea of self hosting local LLMs. Ive been playing around with: https://lmstudio.ai/ and it directly downloads from hugging face.

[–] [email protected] 2 points 1 year ago

There's also ollama which seems to be similar. Not sure if LMStudio is open source but ollama is.

[–] [email protected] 26 points 1 year ago (1 children)

Can we have smaller more domain specific models. that shouldn't require more than casual hardware. like a small model for coding, one for medicine, one for history, and so on. ???

[–] [email protected] 16 points 1 year ago (1 children)

Check out hugging face! Honestly fine tunned models for specific domains seems very popular (if for nothing else because training smaller models is just easier!).

[–] [email protected] 6 points 1 year ago

That’s gonna be a no from me dawg