hendrik

I'm developing a small Python webapp as some sort of finger exercise. Mostly a chatbot. I'm using the Quart framework, which is pretty much alike Flask, just async. Now I want to connect that to a LLM inference endpoint. And while I could do the HTTP requests myself, I'd prefer something that does that for me. It should support the usual OpenAI style API, in the end I'd like it to connect to things like Ollama and KoboldCPP. No harm if it supports image generation, agents, tools, vector databases, but that's optional.

I've tried Langchain, but I don't think I like it very much. Are there other Python frameworks out there? What do you like? I'd prefer something relatively lightweigt that gets out of the way. Ideally provider agnostic, but I'm mainly looking for local solutions like the ones I mentioned.

Edit: Maybe something that also connects to a Runpod endpoint, to do inference on demand (later on)? Or at least something which I can adapt to that?

We've had a bit of a conversation, over in the big NoStupidQuestions community:
https://lemmy.dbzer0.com/post/37540045

While I have my own opinions on lemmit.online, I think it's relatively uncontroversial, that copying content from amateur and indie creators is unethical.
I'd like to request differentiating between the regular Reddit content, and amateur pornography plus OF creators and their original content. And deactivating the bridging for subreddits that contain a decent amount of the latter.

My rationale is more or less that it's not very Robin Hood to take things from people who aren't well off in the first place. And that more or less regular people have the right to decide what happens with pictures of their naked bodies, and we can't just spread them across the internet without their consent or ability to closely control their intimate stuff.

I've been using Etar for years now. But the Samsung calendar app on my wife's phone looks way better, while I'm missing things like the titles in the appointments once it gets crowded. And the all day events and birthdays aren't that prominent either. Plus I don't have some features on Etar like adding notes/emojis to days.

Is there a better calendar app out there? It has to be open source and somehow connect to my Nextcloud. That'd be my requirements. But I believe all calendar apps can connect to webdav.

Seems Meta have been doing some research lately, to replace the current tokenizers with new/different representations:

• Large Concept Models: Language Modeling in a Sentence Representation Space [Github] (December 11, 2024)
• Byte Latent Transformer: Patches Scale Better Than Tokens [Github] (December 12, 2024)

I got a new phone. Skipped a few generations and now I'm running the current GrapheneOS, based on Android 15. I've moved most of the apps, but now I'd like to install my 3 banking apps and 5 discount program spyware apps. I guess I best separate them from the rest of the arbitrary stuff. Banking apps so they can't be messed with, and shady discount programs so those apps can't mess with me and my data...

The internet has a lot of information about Shelter, work profiles, the new(?) private spaces... But I don't know what is current advice and what's outdated advice... What's the current best practice?

During the summer the European Commission made the decision to stop funding Free Software projects within the Next Generation Internet initiative (NGI). This decision results in a loss of €27 million for software freedom. Since 2018, the European Commission has supported the Free Software ecosystem through NGI, that provided funding and technical assistance to Free Software projects. This decision unfortunately exposes a larger issue: that software freedom in the EU needs more stable, long-term financial support. The ease with which this funding was excluded underlines this need.

CC BY-SA 4.0 - SFSCON 2024

Cross-posted from the FSFE Peertube Channel

Seems they recently changed something on Spotify and all the tools I've tried fail now. And DownOnSpot which seems promising has received a cease and desist letter and got taken down. What do you people use? I want something that actually fetches the audio from Spotify, not just rip it from YouTube. And it has to work as of now. Does the latest commit from DownOnSpot work? Back when I tested it a few weeks ago it failed due to some API changes. Are there other tools floating around?

I just found https://www.arliai.com/ who offer LLM inference for quite cheap. Without rate-limits and unlimited token generation. No-logging policy and they have an OpenAI compatible API.

I've been using runpod.io previously but that's a whole different service as they sell compute and the customers have to build their own Docker images and run them in their cloud, by the hour/second.

Should I switch to ArliAI? Does anyone have some experience with them? Or can recommend another nice inference service? I still refuse to pay $1.000 for a GPU and then also pay for electricity when I can use some $5/month cloud service and it'd last me 16 years before I reach the price of buying a decent GPU...

Edit: Saw their $5 tier only includes models up to 12B parameters, so I'm not sure anymore. For larger models I'd need to pay close to what other inference services cost.

Edit2: I discarded the idea. 7B parameter models and one 12B one is a bit small to pay for. I can do that at home thanks to llama.cpp

tl;dr: Be excellent to each other, do something constructive here?

I'm not sure anymore where the Threadiverse is headed. (The Threadiverse being this threaded part of the Fediverse, i.e. Lemmy, MBin, PieFed, ...)
In my time here, I've met a lot of nice people and had meaningful conversations and learned lots of things. At the same time, it's always been a mixed bag. We've always had quite some argumentative people here, trolls, ... I've seen people hate on and yell at each other, and do all kinds of destructive things. My issue with that is: Negative behavior is disproportionately affecting the atmosphere. And I'd argue we have nowhere enough nice behavior to even that out.

I don't see Lemmy grow for quite some time now. Seems it's now leveling off at a bit less that 50k monthly active users. And I don't see how that'd change. I'm missing some clear vision/idea of where we want to be headed. And I miss an atmosphere that makes people want to join or stay here, of all of the places on the internet. The saying is: "If you don't go forwards you go backwards". I'm not sure if this applies... At least we're not shrinking anymore.

And I'm always unsure if the tone and atmosphere here changes subtly and gradually. I've always disagreed with a few dynamics here. But lately it feels like we're on the decline, at least to me. I occasionally keep an eye on the votes on my comments. And seems I'm getting fewer of them. Sometimes I reply to a post and not a single person interacts. Even OP seems to have abandoned their post moments after writing it. And also for nuanced and longer replies, I regularly don't get more than one or two upvotes. I think that used to be a bit better at some point. And I see the same thing happening with other peoples' comments. So it's not just me writing low-quality comments. What does work is stating simple truths. I regularly get some incoming votes with those. But my vision of this place isn't spreading simple truths, but have proper and meaningful discussions, learn things and new perspectives or just mingle with people or talk. But judging by the votes I observe, that isn't appreciated by the community here.

Another pet peeve of mine is the link aggregator aspect of Lemmy. I'd say at least 80% of Lemmy is about dumping some political (or tech) news articles. Lots of them don't generate any engagement. Lots of them are really low-effort. OP just dumps something somewhere, no body text added, no info about what's interesting about it. And people don't even read those articles. They just read the title and react (emotionally) to that. In the end probably neither OP nor the audience read the article and it's just littering the place. Burying and diminishing other, meaningful content. (With that said: There are also nice (news) discussions going on at the same time. And Lemmy is meant to be a link aggregator. It's just that my perception is: it's skewed towards low quality, low engagement and random noise.)

A few people here also don't really like political debate. And there's no escape from it here on Lemmy since so much revolves around that. And nowadays politics is about strong opinions, emotions and emotional reactions. And often limited to that. The dynamics of Lemmy reinforce the negative aspect of that, because the time when you're most incentivized to reply or react is, when it triggers some strong emotion in you, for example you strongly disagree with a comment and that makes you want to counter it and write your own opinion underneath. If you agree, you don't feel a strong emotion and you don't reply. And the majority of users seems to also forget to upvote in that case, as I lined out earlier. And we also don't write nuanced answers, dissect complex things and examine it from all angles. That's just effort and it's not as rewarding for the brain to do that as it is pointing out that someone is wrong. So it just fosters an atmosphere of being argumentative.

Prospect

I think we have several ways of steering the community:

1. Technology: Features in the software, design choices that foster good behavior.
2. Moderation: Give toxic people the boot, or delete content that drags down the place. Following: What remains is nice people and not adverse content.
3. The community

I'd say 1 and 2 go without saying. (Not that everything is perfect with those...) But it really boils down to 3: The community. This is a fairly participatory place. We are the ones shaping the tone and atmosphere. And it's our place. It's kind of our obligation to care for it if we want to see it go somewhere. Isn't it?

So what's your vision of this place? Do you have some idea on where you'd like it to go? Practical ideas on how to achieve it?
Do you even agree with my perception of the dynamics here, and the implications and conclusions I came up with?