18
Audio Flamingo 3 - Fully Open Large Audio Language Models
(research.nvidia.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 16 Jul 2025
18 points (100.0% liked)
LocalLLaMA
3413 readers
2 users here now
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
founded 2 years ago
MODERATORS
Yes, thanks. I think I agree with you here. The copyright model is rent-seeking by nature. And we could likely do better.
Ultimately a book author wants to sell his product to me. How it's done isn't ideal at all, but that's kind of his motivation. So I don't think you want him to starve because books aren't a valid product to sell, but it's about the way it's done.
My single argument here is: Look at the AI industry and compare what you just said. They're doing exactly the same thing, just 20 times worse. And you should be opposed to that, too!
I've laid down how the big AI companies do nothing for the benefit of other people. I've asked you what you think they contribute (in case I'm wrong) and you also came up with zero things they do for other people. So it boils down pretty much to the same. A book author creates intellectual property to sell a product to people "trying to get paid". An AI company creates intellectual property to sell a product in order "to get paid". It's the same thing.
Let's tackle monopolies: Everyone can read a book in case they can get ahold of it. And with some intelligence and time, everyone can write a book. That's a monopoly in your eyes. And while we have weird concepts like Fixed book price, that's mainly meant to foster healthy competition and promote the sales of interesting books rather than just blockbusters. Though, I really have a wide selection of sci-fi books available and I've bought several of them for 50ct. And I have a public library card for 26€ a year and I can read 500 books a day if I like, and I get a selection of blue-rays on top. That's what the monopoly does to me. (With everything else I agree with you. It's bad that they pile that information up and that it's not freely available but a business model.)
Now AI: I wanted to try Sora because they pioneered video models on that scale. For a long time they said "no thanks" to me. We won't provide that service to you, it's just for testing and a select few people we like. You get none of it. You can't even pay, no matter how much. Then I waited for half a year and wanted to try Google's Veo 3 and seems the interesting stuff is in the $100 a month tier. And what the fuck, the output is supposed to come with copyright? And terms and conditions?
I can't get that service anywhere else and they just say tough luck, it's gonna be $100 to try 8s video snippets because the company is amongst the select few who offer that (...cough...monopoly...). Or use Sora, now that it's available, but they've changed the model to their likings and it became a bit worse than the initial trailers, and by the way: that is $200 a month.
So yeah, fantastic prices, also quite random, offered by less than a handful of mega corporations, based on their IP, they design "the food" so I need to eat what they devised for me. And I can't even eat the food the way I like, but have to follow their terms and procedures.
Same applies to text gen AI. It's a monopoly of billion dollar companies who get to shape it. Me or you, we can't do it. It's almost impossible to train a base model on that scale. And I can't even use them for what I like. I wanted to try story-writing and chose some dark sci-fi and a murder mystery story, and it's designed to refuse service to me. Instead it'll give long lectures about ethics to me about how murder is wrong. Yeah, no shit sherlock. Interestingly, AIstudio did help me write exploits for computer security vulnerabilities for some blue-teaming I did.
In you analogy with the food: I'm hungry. Now a company comes. Of course they don't offer me the food I like, but they say I have to eat what they designed for me. And it's going to be a random $100 or $200. And I can't touch the food or eat it myself, they're adamant in spoon-feeding it as a very specific service to me. I can never cook my own food, since the resources for that cost like $100 million. And they keep the recipe a closely guarded secret and they're so obsessed with it, they don't even tell me the nutritional value or anything about what went in to the designer food I need to lick off their spoon.
If you want in on the business. Also tough luck. You now need to start from scratch with everything, since the data is hoarded by the big players and they don't share. On the level of ChatGPT... Well, you can get in like Microsoft and pay some billion dollars. But with that kind of money, it's not super accessible, exactly like you'd expect from a monopoly. Other players can get in, like the Chinese. And how do they do it? It's sponsored/subsidised with billions of dollars by the government. And that's what it takes and they do it this way for more than a decade now.
That's kind of a difficult example. I think archiving and digitizing is okay and in most cases he can do it. Copying for own use is always fine and that's phrased so it applies to companies as well. Archival is such an allowed use. Public libraries have a seperate paragraph. They can copy and can do necessary changes like digitizing. That applies to commercial libraries as well, as long as they're open to the public. So we have you covered here. And there is more. For some works it's mandatory to preserve them. They need to be sent to a library and the government specifically takes care to preserve (European) culture with these things. They're mandated for example to show up in the shelves of the national library.
I seriously doubt the AI companies are going to help with preserving culture, though. The incident with Meta torrenting books for example had them on the opposite side. They took care to "leech". That is, they took out information from the network and made sure not to balance that out. Resulting in a negative balance on the network and "free" information exchange.
If your worried "our guy can get more. If he destroys all remaining copies of these newspapers [...]" I believe you found him. It's not exactly that, since that kind of information is duplicated and can't be burnt that way. But Meta do the closest thing there is to it. There are resources to exchange information and culture, and they deliberately "burn" those resources for their own benefit and to the disadvantage of everyone else.
And that has also already happened in the realm of AI. They change their service or cease operation. And since AI is just a service and the users don't own anything, they're then left with nothing. First big thing I'm aware of is how Replika AI dropped the main use-case of their service and millions of people were affected. And that is way worse than books. I have been banned from services. They just said "suspicious behaviour" and deactivated my account and I was stripped access. A book author cannot do that. I can still buy his book even if he doesn't like me. Cancelling service and doing whatever they like with the userbase is what big tech companies do.
So my argument is: You've really made a good argument in pointing out countless severe shortcomings of current copyright culture. And I've learned a lot. The AI industry is an even worse manifestation of that. They also pile up intellectual property for their product. And contrary to a book, I don't even own the darn physical thing, but they introduce all kinds of other shenanigans and make it something I rent, boarded-up, and then they often also apply copyright on top. They stepped up everything that is bad about copyright, several notches.
And then the successful players are all ruthless. They're not just selling me a book. Currently they're mainly interested in investment money and I'm not really 100% their customer. They happily weigh down on society. In my last comment I addressed how they deliberately evade law and some big players even pirate and do things that are currently illegal. Just for their own benefit. Enshittification of the internet is a side-effect they gladly accept. And they're expected to displace more things with their product (including culture) and neither do they contribute back, nor do they care about the consequences.
I think Fair Use might be a nice concept. It definitely is a regulation mechanism. The government/society is taking away privileges of people (copyright holders) with that. To the benefit of society and progress. Now go ahead and apply the same thing to AI companies! Regulate them as well!
Sorry, but you have not understood the concept yet.
You demand that AI companies should work for free and give things away for free. But they also should pay people that make no contribution.
They do, just like farmers. If people did not find their services beneficial, they would not pay.
This is called a barrier to entry (Marktschranke).
It doesn't have to be very bad. For example, you can't just become a farmer. You must buy a farm. There are problems with that, but they aren't big. Food is cheap and plentiful.
The people who make AIs want to be paid for their work. The people who build and maintain the datacenters, the hardware, the electricity, and so on. Should they work for free?
The problem starts when people want more than that.
Have you ever noticed how many of these books were written in the USA and cheaply translated into German?
No. I think you misunderstood. An exclusive copyright is a monopoly by definition.
They were legally required to do that. Downloading the books for their purposes was fair use. Uploading would certainly not have been.
I don't understand how this accusation makes the slightest bit of sense. These torrents are a violation of EU copyright law. Your argument means that these torrents shouldn't exist in the first place. You are not demanding that Meta should be allowed to upload these books. You're saying they shouldn't be allowed to download them, either.
The issue is just, I don't see how any of those are arguments to distinguish between the two. I can twist them so almost every single argument applies to book authors and I don't see any contradictions with that:
"You demand that ~~AI companies~~ [book authors] should work for free and give things away for free. But they also should pay [in content] people that make no contribution."
"They do [create something], just like farmers. If people did not find their services beneficial, they would not pay [for the books]."
"[The price of the pile of books] is called a barrier to entry (Marktschranke)."
"It doesn't have to be very bad. For example, you can't just become ~~a farmer~~ [a big AI company]. You must buy ~~a farm~~ [data]. There are problems with that, but they aren't big. ~~Food~~ [data] is cheap and plentiful."
Alright, we have the one issue here, because data is cheap and plentiful in the digital age, and they gather my data as well, but theoretically that should be limited in the EU, and we get the copyright issue with the books here. But I don't think the farmer/AI comparison goes all the way. For example graphics cards are the opposite of cheap and plentiful, and there isn't a problem with that. So it's not like there is a rule that resources or products have to be cheap or plentiful. It's surely benefitial, but there's also the real world, like with GPUs. And farmers also use intellectual property crops, and they use machines that cost hundreds of tousands of dollars. Sometimes you just have to pay for supplies and resources. That applies to farmers and for AI companies buying their supplies.
"The people who make ~~AIs~~ [books] want to be paid for their work. The people who build and maintain the ~~datacenters~~ [book press], the ~~hardware~~ [online shops and distribution chain], the ~~electricity~~ [author's computer and studies and travels for the content], and so on. Should they work for free?"
"The problem starts when people want more than that."
I really fail to see the difference here. Unless I start with a proposition: writing books is not a valid business model, but AI is... But why is that? Both are built on the grounds of intellectual property, both are products and require effort to be created. Why does a book author work 6 months and doesn't get paid for his job and an AI researcher works for 6 months and needs to be paid?
Or phrased differently - Why isn't it a valid product if a human reads a lot and then creates something and wants to sell the result... But if a big company devises a mechanism that reads a lot and they want to sell the result, then it suddenly is a valid product?
Why do you not want book authors to be paid now?
Because that's the Fair Use. It doesn't involve monetary compensation for the use. Meaning they don't get paid.
Why do you want fair use to work that way?
You said you praise the American Fair Use model. I said I don't like it to work in that way. And most of all not grant exceptions to certain business models. And I agreed that there are some issues in the underlying copyright model, which might change the entire picture if addressed. I mean the interesting question is: How should copyright work in conjunction with AI and in general? And who needs to be compensated how?
I understand, But why do you want a fair use model that means that authors don't get paid at all?
I'm fairly sure the term "Fair Use" by definition means unauthorized and unpaid use. I mean we can try to twist the meaning of these words. Or maybe I misunderstood it. But paying would be kind of contradictory to the entire concept. It'd be (forced) licensing or something within the realm of copyright, depending on what you mean. But I think we need a new/different word for it.
In the US, it almost certainly wouldn't be fair use if it meant that the author doesn't get paid. Of course, you don't get paid for the fair use, but there are a lot of things you don't get money for.
You're talking about authors not being paid at all. What's that about?
That was about abolishing copyright altogether. Since we discussed that as an option. We're now discussing what I called "subsidies" earlier. Authors do get paid, but for certain "uses" and not for others. And authors get financed by a different group of people.
In your example with the farmers, they're not paid by me buying the product in the supermarket and that money gets handed down the chain to every supplier... But Nestle got the cocoa beans for free and society now gets to pay the farmer by a different method. Unless you have a specific proposal here, that'd be likely the definition of a subsidy to help Nestle and make their products look cheaper on a supermarket shelf.
Let me try to follow this.
A cocoa farmer is paid for some uses of their cocoa beans but not others. For example, Nestle has to pay to turn their beans into chocolate and sell it in supermarkets. On the other hand, no one has to pay to take a photo of their beans and sell it to Nestle for ads. Right? I'm with you so far.
I don't get the next step. Because some uses are free, all uses should be free? Then Nestle gets a subsidy and we pay the farmers some other way?
Pretty much. The AI companies are Nestle in that analogy. They get their supplies for free?! While I and everyone else had to pay for the very same supplies, when I needed the textbook to study CS to become a computer programmer. The professor gets to brush up their salary, and I think it's a bit unfair to me that I'm asked to take out 60€ from my mediocre turnover of a few hundred bucks a month as a student. I think I should have been asked to pay 30€ and a company with a billion dollar budget should be asked to pay something like 100€ since they make use of it multiple times. And they should hand that cost down to their customers. And my use was transformative as well. The information from the textbook is now modeled in my brain.
I think the analogy with the picture is kind of alright as well. I mean analogies are hard here, since it's a labour intensive task to duplicate crops and coffee beans, while duplication is pretty much for free in case of information. And it doesn't take away the original.
Now what is a picture? It's kind of a summary, a depiction of the outer appearance. And snapping a picture of a book cover would make sense for Fair Use. That's kind if what it's made for. If you now snap a picture of each and every one of the 400 pages inside, that's where law says Fair Use stops. And what do AI companies use for training? A picture/summary of the book? Or the content within?
Now I'm even more confused.
Your professor abused their monopoly. That's the sort of thing I've been condemning. You are basically fine with that. You just think they should adjust their price policy to income. Well, yes, that would be the profit maximizing move. You make everyone pay as much as they are able to. That's what the copyright lobby wants. But I have to point out: There is no reason why they should lower the price for you. After all, you were able to pay. Rather, there seems to be room to raise the price.
Do you actually think this kind of monopoly abuse is a good thing?
No, that's not what the law says. I think, the problem is that we have different ideas over how Fair Use in the US actually works. I'll have to think about that.
Well, it's complicated. And depends on which theoretical option we're talking about. I for example think writing the textbook when you're the professor and selling that to the students is a very bad thing. I'm not fine with that at all. They should be funded mainly by taxpayer money (at least that's what we do). And the fruit of their labour should then be owned by the taxpayer. The US does similar things, like government texts, NASA pictures etc used to be owned by the people. And everyone is "the people" from a random student to a big AI company.
It's a bit a special example though, and doesn't translate 1:1 to the private book market.
I believe your regular book author does it the other way around. They aren't commissioned by anyone, they generally write it and only after that does the product get monetized. And I believe that's where your "rent-seeking" comes in. Somehow the author managed to feed themselves for the time it took them to write the book, and now they have it as an asset which they can try to turn into as much money as they can. It's two things mushed together. Their valid desire to eat and be compensated for their labour, plus the rent from the asset which might be huge for popular books and doesn't reflect labour cost. And all of this is very different from a university professor with a salary. It could and should be decoupled for them. But it's straight up impossible for the majority of authors, given our current copyright model. I think that's a fundamental limitation of capitalism.
And I wonder if those regulatory mechanisms are even applied correctly. I had that with the textbooks in university to some lesser extent. School was fine. But I heard in the US for example education is a complete rip-off and we get news articles every year on how parents can't afford the several hundred bucks for school textbooks for their children. And that is despite a different copyright doctrine. Maybe our model here leads to better results some times, I don't really know.
And concerning the Fair Use: Is there law which offers an option for compensation? I thought that was contradictory per definition.
Hmm. You seem to treat an economic rent as being the same as a return on investment. Any particular reason for that?
You aren't fine with that. But why are you fine with the copyright industry doing it to everyone in the country?
I don't think my opinion as some random dude matters here. I could uphold arbitrary stupid believes. But this is kind of a factual question. So whether I personally, as one person, am fine with something is of no concern here. The question is, how do we arrive at a consistent economy model for immaterial goods...
And I think I wrote like 5 times now that I'm NOT fine with that. I said I view it as a (necessary) evil. It is evil in the sense of bad, I'm not fine with it, it comes with severe issues, we should do better than that. However "is" and "should" are two seperate things. We happen to live on a world that came up with copyright. It exists. We made a pact with the devil to address one thing. And I'm merely acknowledging that. Since it does exist, I need to deal with it. That's not agreement from my side. Copyright serves one legitimate purpose. It applies our capitalist economy to immaterial goods. It's supposed to allow individuals and companies to create, and trade with more than just cocoa beans. But it's complicated and we might have come up with a stupid way to do it. And a way that simultaneously has lots of negative side-effects.
And now what? That is the question. Do we abolish it? Do we replace it with something else that handles the one legitimate purpose a better way? Do we retrofit it and try to "patch" it? Do we do that just for AI? Or for more than just one use-case?
And I think I make a point about how return on investment and an economic rent are two distinct things. Yet they're in practice falsely(!) mushed together, which again is bad... Or am I mistaken and I can pay an artist for their investment but not pay a rent? I don't think there is a good way to do it with the current model. That means I get to treat both as the same. You seem to be under the impression I like it. But I don't. It's just that I have to abide by law and that currently mandates me to do it.
I see. I think this is the big one:
A return on investment is not the same as an economic rent.
Let's go back to the farmer example. You agree that a monopoly on the food supply is a bad thing. It can and will be abused.
Sidenote: You suggested that the government should produce textbooks to prevent abuse. Would that also be your solution here? Would that be preferable to the current arrangement?
Now, let's look at the situation of a farmer more closely. A farmer has to do a lot of work before they can harvest. They also need stuff like seeds, fertilizer, pesticides, fuel, machinery, spare parts and maintenance, and so on.
In the old times, one held back part of a grain harvest as seed grain for next year. That is an investment in the economics sense. You don't consume everything, but keep it so that you have more in the future. The finance meaning is subtly different but never mind.
Farmers gets a return on investment. They invest money and labor so that there is a harvest in the future. They could sell the equipment they already own to have more spending money now.
A ROI is part of a farmers' income but is not economic rent.
Back to authors. An established author will get an advance before they write the next book. That's investment by the publisher. If they don't get an advance, then the author is making the investment, but let's ignore that for simplicity. Investments are always risky. In this case, some books don't sell well and don't make back the money.
As a publisher, how much money would you invest in future books to maximize your profit? It depends on the expected payout and the cost of money.
Cost of money: You could borrow the money. Then the cost of the money is the interest on the loan. Or you could use the money for something else, eg buying safe government bonds. In that case, the cost is an opportunity cost. It's what you miss out on by not investing elsewhere.
Expected payout: It's the average profit/loss on each book. It is something you estimate based on experience.
The more books there are on the market, the lower the average profit. There must be a limit to how much of their income people are willing to spend on books. At some point, you have a lot of similar books chasing the same audience. That lowers the average. To maximize your profit, you invest in the production of more and more books, until the average return on each book is equal to the cost of money.
I'll leave it at that for now.
Yes. That's economy and investment how we usually do it today. The conclusion of that is, the "manufacturers" sell their product at the end of the day. I think in the realm of what we're discussing, it means an AI company is then the client of the book authors. And they pay for the books, or more the content within. That's the traditional model and doesn't make sense unless it results in some product being sold.
Now that's a really interesting question. Some intelligent people have proposed similar things, economy being controlled by the government instead of the free market. And we've tried it. Turns out it's tricky to get it right. When they tried applying it to the entire economy, it often resulted in lots of corruption, an underperforming economy, up to outrageous things like famine and starvation in the population. Though I'm making it sound simpler than it is. Lots of different factors were involved with that.
And then sometimes we get it somewhat right. For example education is done by the government. Public infrastructure like roads, trains... And the government already produces books and TV. One example is public broadcasting like the BBC or ARD/ZDF here. I think what they produce is far superior than news in the USA. On the downside it's a very bloated organization and they waste lots and lots of money doing it.
So... My answer to your question is: yes and no. Yes, government should produce books and other content. Like local news from my region, which is not a profitable business so the private companies regularly fail due to that. And education would be another topic. It'd be great if education were accessible to everyone, at no cost. Maybe some other things.
And no, I don't think government should produce all books and content. That'd be kind of a monopoly on information. It's hard to choose which book should be written and which discarded. Which wannabe autor to put on the payroll... We'd need a lot of trust and faith in the government, which we don't have. And it's likely going to fail because of a multitude of reasons. I'd say it's somewhat a nice idea. But I give it zero chance to work as intended in reality.