LocalLLaMA

2775 readers

70 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

founded 2 years ago

MODERATORS

SkySyrup@sh.itjust.works

pax@sh.itjust.works

noneabove1182@sh.itjust.works

Smokeydope@lemmy.world

MonsterBug@sh.itjust.works

WizardLM/WizardCoder-33B-V1.1 released! (huggingface.co)

submitted 1 year ago by noneabove1182@sh.itjust.works to c/localllama@sh.itjust.works

11 comments fedilink hide all child comments

Based off of deepseek coder, the current SOTA 33B model, allegedly has gpt 3.5 levels of performance, will be excited to test once I've made exllamav2 quants and will try to update with my findings as a copilot model

you are viewing a single comment's thread
view the rest of the comments

[–] noneabove1182@sh.itjust.works 5 points 1 year ago (1 children)

First few quants are up: https://huggingface.co/bartowski/WizardCoder-33B-V1.1-exl2

4.25 should fit nicely into 24gb (3090, 4090)

Smaller sizes still being created, 3.5, 3.0, and 2.4

[–] vinnymac@sh.itjust.works 2 points 1 year ago (1 children)

I am curious if you prefer WizardCoder over Phind https://huggingface.co/Phind/Phind-CodeLlama-34B-v2

I am running them on my 3090, although the machine also has 32GBs of RAM as well.

[–] noneabove1182@sh.itjust.works 2 points 1 year ago (1 children)

I don't have a lot of experience with either at this time, I've used them here and there for programming questions but usually I stick to 7b models because I use them for code completion and I only find that useful if it completes the code before I do lol

That said, I've had overall good answers from either whenever I've decided to pull them out, it feels like wizard coder should be better since it's so much newer but overall it hasn't been that different. Wish phind would release an update :(

[–] vinnymac@sh.itjust.works 2 points 1 year ago

That makes sense, thank you for sharing.

I tend to use CoPilot for IDE code completion, and I use the 34B models for automated refactors and code transforms where accuracy is a requirement.