It works with windows 10 as well; it didn't like a lot of the older pdfs that were in my documents; and it didn't work offline. The last bit is probably the python dependencies trying to update themselves, but it doesn't fail gracefully into a working state when the network isn't available.
this post was submitted on 14 Feb 2024
19 points (100.0% liked)
LocalLLaMA
3297 readers
58 users here now
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
founded 2 years ago
MODERATORS
Thanks for the info. I'm not on Windows, so I can't try it. Doesn't work offline? That's what I was afraid of.
It's a great proof of concept. Hopefully we'll see more text-gen-webui extensions soon.
Interesting that it only works on windows, a lot of AI projects I've seen have been the other way around.
It will also be available as an open-source reference project, so developers can easily implement the same capabilities in their own applications.
Not truly open source but at least you can build your own and control what happens with all the data, if I understand this correctly?
It means that they want people to consult the code as a reference for how to best use the hardware acceleration.
If all software uses their cards to best effect, that makes their cards more useful and thus more valuable; making them money. If only their own frontend can do that, they lose out on most of that, while also having to spend money to make sure that the rest of the software, like the UI, is competitive.
Ah, that makes sense. Thank you!