LocalLLaMA

3306 readers

3 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

founded 2 years ago

MODERATORS

[email protected]

Training a model without a GPU (sh.itjust.works)

submitted 2 years ago by [email protected] to c/[email protected]

9 comments fedilink hide all child comments

Hi, I'm currently starting to learn how LLM works in depth, so I started using nanoGPT to understand how to train a model and I'd like to play around with the code a little more. So I set myself a goal to train a model that can write basic French, it doesn't to be coherent or deep in its writing, just French with correct grammar. I only have a laptop that doesn't have a proper GPU, so I can't really train a model with billions of parameters. Do you think it's possible without too much dataset or intensive training? Is it a better idea if I use something different from nanoGPT?

TLDR: I'd like to train my own LLM on my laptop which doesn't have a GPU. It's only for learning purpose, so my goal is that it can write basic French. Is it doable? If it is, do you have any tips to make this easier?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 4 points 2 years ago* (last edited 2 years ago) (1 children)

If you want to learn machine learning, you could maybe play around with the examples recognizing single digit handwritten numbers with that MNIST dataset or something in that kind of league.

I think training an LLM that can be somewhat useful will be way out of scope with the RAM and computing capabilities such a laptop has to offer. Maybe correct grammer if you don't care to wait for a long long time. Something with the level of intelligence of autocomplete. But definitely not coherent or intelligent or answering your questions.

You could rent a VM in the cloud. Services like runpod.io or vast.ai offer you a proper GPU for like $2 an hour. There is also Amazon, Google, Azure, Lambda...

[–] [email protected] 1 points 2 years ago* (last edited 2 years ago)

Do cloud services see everything - text/images data for training? And finished trained model? If so, runpod.io etc are no solution.