LocalLLaMA

2865 readers

107 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

founded 2 years ago

MODERATORS

[email protected]

NVIDIA Chat With RTX (www.nvidia.com)

submitted 1 year ago by [email protected] to c/[email protected]

6 comments fedilink hide all child comments

This is an interesting demo, but it has some drawbacks I can already see:

It's Windows only (maybe Win11 only, the documentation isn't clear)
It only works with RTX 30 series and up
It's closed source, so you have no idea if they're uploading your data somewhere

The concept is great, having an LLM to sort through your local files and help you find stuff, but it seems really limited.

I think you could get the same functionality(and more) by writing an API for text-gen-webui.

more info here: https://videocardz.com/newz/nvidia-unveils-chat-with-rtx-ai-chatbot-powered-locally-by-geforce-rtx-30-40-gpus

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 4 points 1 year ago (1 children)

It will also be available as an open-source reference project, so developers can easily implement the same capabilities in their own applications.

Not truly open source but at least you can build your own and control what happens with all the data, if I understand this correctly?

[–] [email protected] 3 points 1 year ago (1 children)

It means that they want people to consult the code as a reference for how to best use the hardware acceleration.

If all software uses their cards to best effect, that makes their cards more useful and thus more valuable; making them money. If only their own frontend can do that, they lose out on most of that, while also having to spend money to make sure that the rest of the software, like the UI, is competitive.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago)

Ah, that makes sense. Thank you!