this post was submitted on 03 Mar 2025
887 points (100.0% liked)

Technology

67338 readers
3850 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

But the explanation and Ramirez’s promise to educate himself on the use of AI wasn’t enough, and the judge chided him for not doing his research before filing. “It is abundantly clear that Mr. Ramirez did not make the requisite reasonable inquiry into the law. Had he expended even minimal effort to do so, he would have discovered that the AI-generated cases do not exist. That the AI-generated excerpts appeared valid to Mr. Ramirez does not relieve him of his duty to conduct a reasonable inquiry,” Judge Dinsmore continued, before recommending that Ramirez be sanctioned for $15,000.

Falling victim to this a year or more after the first guy made headlines for the same is just stupidity.

top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 198 points 3 weeks ago (3 children)

Haven't people already been disbarred over this? Turning in unvetted AI slop should get you fired from any job.

[–] [email protected] 35 points 3 weeks ago

Different jurisdiction

[–] [email protected] 14 points 3 weeks ago

Immediately there should be a contempt charge for disrespecting the Court.

load more comments (1 replies)
[–] [email protected] 159 points 3 weeks ago (12 children)

“Mr. Ramirez explained that he had used AI before to assist with legal matters, such as drafting agreements, and did not know that AI was capable of generating fictitious cases and citations,” Judge Dinsmore wrote in court documents filed last week.

Jesus Christ, y'all. It's like Boomers trying to figure out the internet all over again. Just because AI (probably) can't lie doesn't mean it can't be earnestly wrong. It's not some magical fact machine; it's fancy predictive text.

It will be a truly scary time if people like Ramirez become judges one day and have forgotten how or why it's important to check people's sources yourself, robot or not.

[–] [email protected] 65 points 3 weeks ago (9 children)

No probably about it, it definitely can't lie. Lying requires knowledge and intent, and GPTs are just text generators that have neither.

[–] [email protected] 12 points 3 weeks ago (1 children)

A bit out of context my you recall me of some thinking I heard recently about lying vs. bullshitting.

Lying, as you said, requires quite a lot of energy : you need an idea of what the truth is and you engage yourself in a long-term struggle to maintain your lie and keep it coherent as the world goes on.

Bullshit on the other hand is much more accessible : you just have to say things and never look back on them. It's very easy to pile a ton of them and it's much harder to attack you about any of them because they're much less consequent.

So in that view, a bullshitter doesn't give any shit about the truth, while a liar is a bit more "noble". 0

[–] [email protected] 14 points 3 weeks ago

I think the important point is that LLMs as we understand them do not have intent. They are fantastic at providing output that appears to meet the requirements set in the input text, and when they actually do meet those requirements instead of just seeming to they can provide genuinely helpful info and also it's very easy to not immediately know the difference between output that looks correct and satisfies the purpose of an LLM vs actually being correct and satisfying the purpose of the user.

load more comments (8 replies)
[–] [email protected] 50 points 3 weeks ago (3 children)

AI, specifically Laege language Models, do not “lie” or tell “the truth”. They are statistical models and work out, based on the prompt you feed them, what a reasonable sounding response would be.

This is why they’re uncreative and they “hallucinate”. It’s not thinking about your question and answering it, it’s calculating what words will placate you, using a calculation that runs on a computer the size of AWS.

[–] [email protected] 9 points 3 weeks ago (1 children)

It's like when you're having a conversation on autopilot.

"Mum, can I play with my frisbee?" Sure, honey. "Mum, can I have an ice cream from the fridge?" Sure can. "Mum, can I invade Poland?" Absolutely, whatever you want.

load more comments (1 replies)
load more comments (2 replies)
[–] [email protected] 12 points 3 weeks ago (2 children)
[–] [email protected] 20 points 3 weeks ago (12 children)

a lie is a statement that the speaker knows to be wrong. wouldnt claiming that AIs can lie imply cognition on their part?

[–] [email protected] 11 points 3 weeks ago* (last edited 2 weeks ago) (8 children)

I've had this lengthy discussion before. Some people define a lie as an untrue statement, while others additionally require intent to deceive.

E: you can stop arguing about definitions and logic. The fact remains that some people will refer to untrue statements as lies, no matter what the dictionary says.

[–] [email protected] 10 points 3 weeks ago (2 children)

I would fall into the latter category. Lots of people are earnestly wrong without being liars.

load more comments (2 replies)
load more comments (7 replies)
load more comments (11 replies)
load more comments (1 replies)
[–] [email protected] 12 points 3 weeks ago (2 children)

It can and will lie. It has admitted to doing so after I probed it long enough about the things it was telling me.

[–] [email protected] 20 points 3 weeks ago* (last edited 3 weeks ago) (17 children)

Lying requires intent. Currently popular LLMs build responses one token at a time—when it starts writing a sentence, it doesn't know how it will end, and therefore can't have an opinion about the truth value of it. (I'd go further and claim it can't really "have an opinion" about anything, but even if it can, it can neither lie nor tell the truth on purpose.) It can consider its own output (and therefore potentially have an opinion about whether it is true or false) only after it has been generated, when generating the next token.

"Admitting" that it's lying only proves that it has been exposed to "admission" as a pattern in its training data.

[–] [email protected] 14 points 3 weeks ago

I strongly worry that humans really weren't ready for this "good enough" product to be their first "real" interaction with what can easily pass as an AGI without near-philosophical knowledge of the difference between an AGI and an LLM.

It's obscenely hard to keep the fact that it is a very good pattern-matching auto-correct in mind when you're several comments deep into a genuinely actually no lie completely pointless debate against spooky math.

load more comments (16 replies)
load more comments (1 replies)
[–] [email protected] 10 points 3 weeks ago* (last edited 3 weeks ago) (2 children)

Its actually been proven that AI can and will lie. When given a ability to cheat a task and the instructions not to use it. It will use the tool and fully deny doing so.

Edit:

Not sure why the downvotes because when i say proven i mean the research has been done and the results have been known for while

https://arxiv.org/abs/2407.12831

load more comments (2 replies)
load more comments (7 replies)
[–] [email protected] 108 points 3 weeks ago (2 children)

Hold them in contempt. Put them in jail for a few days, then declare a mistrial due to incompetent counsel. For repeat offenders, file a formal complaint to the state bar.

[–] [email protected] 48 points 3 weeks ago (1 children)

Eh, they should file a complaint the first time, and the state bar can decide what to do about it.

[–] [email protected] 8 points 3 weeks ago (2 children)

"We have investigated ourselves and found nothing wrong"

[–] [email protected] 32 points 3 weeks ago (1 children)

The bar might get pretty ruthless for fake case citations.

[–] [email protected] 9 points 3 weeks ago

I would hope that gross negligence and incompetence with come with severe consequences.

[–] [email protected] 11 points 3 weeks ago

The state bar is not the state cops.

[–] [email protected] 22 points 3 weeks ago

From the linked court document in the article: https://storage.courtlistener.com/recap/gov.uscourts.insd.215482/gov.uscourts.insd.215482.99.0.pdf?ref=404media.co

"For the reasons set forth above, the Undersigned, in his discretion, hereby RECOMMENDS that Mr. Ramirez be personally SANCTIONED in the amount of $15,000 pursuant to Federal Rule of Civil Procedure 11 for submitting to the Court and opposing counsel, on three separate occasions, briefs that contained citations to non-existent cases. In addition, the Undersigned REFERS the matter of Mr. Ramirez's misconduct in this case to the Chief Judge pursuant to Local Rule of Disciplinary Enforcement 2(a) for consideration of any further discipline that may be appropriate"

Mr. Ramirez is the dumbass lawyer that didn't check his dumbass AI. If you read above the paragraph I copied from, he gets laid into by the judge in writing to justify recommendation for sanctions and discipline. Good catch by the judge and the processes they have for this kind of thing.

[–] [email protected] 66 points 3 weeks ago

I’m all for lawyers using AI, but that’s because I’m also all for them getting punished for every single incorrect thing they bring forward if they do not verify.

[–] [email protected] 62 points 3 weeks ago* (last edited 3 weeks ago) (2 children)

I hate people can even try to blame AI.

If I typo a couple extra zeroes because my laptop sucks, that doesn't mean I didn't fuck up. I fucked up because of a tool I was using, but I was still the human using that tool.

This is no different.

If a lawyer submits something to court that is fraudulent I don't give a shit if he wrote it on a notepad or told the AI on his phone browser to do it.

He submitted it.

Start yanking law licenses and these lawyers will start re-evaluating if AI means they can fire all their human assistants and take on even more cases.

Stop acting like this shit is autonomous tools that strip responsibility from decisions, that's literally how Elmo is about to literally dismantle our federal government.

And they're 100% gonna blame the AI too.

I'm honestly surprised they haven't claimed DOGE is run by AI yet

load more comments (2 replies)
[–] [email protected] 41 points 3 weeks ago (1 children)

Great news for defendants though. I hope at my next trial I look over at the prosecutor's screen and they're reading off ChatGPT lmao

[–] [email protected] 23 points 3 weeks ago (1 children)

So long as your own lawyer isn't doing the same, of course :)

load more comments (1 replies)
[–] [email protected] 26 points 3 weeks ago (4 children)

But I was hysterically assured that AI was going to take all our jobs?

load more comments (4 replies)
[–] [email protected] 23 points 3 weeks ago* (last edited 3 weeks ago) (4 children)

The judge wrote that he “does not aim to suggest that AI is inherently bad or that its use by lawyers should be forbidden,” and noted that he’s a vocal advocate for the use of technology in the legal profession. “Nevertheless, much like a chain saw or other useful [but] potentially dangerous tools, one must understand the tools they are using and use those tools with caution,” he wrote. “It should go without saying that any use of artificial intelligence must be consistent with counsel's ethical and professional obligations. In other words, the use of artificial intelligence must be accompanied by the application of actual intelligence in its execution.” 

I won't even go that far. I can very much believe that you can build an AI capable of doing perfectly-reasonable legal arguments. Might be using technology that looks a lot different from what we have today, but whatever.

The problem is that the lawyer just started using a new technology to produce material that he didn't even validate, without determining whether-or-not it actually worked for what he wanted to do in its current state, and where there was clearly available material showing that it was not in that state.

It's as if a shipbuilder started using random new substance in its ship hull without actually conducting serious tests on it or even looking at consensus in the shipbuilding industry as to whether the material could fill that role. Meanwhile, the substance is slowly dissolving in water. Just slapped it in the hull and sold it to the customer.

EDIT: Hmm. Actually, I thought that the judge was saying that the lawyer needed to use AI-generated stuff in a human-guided role, but upon consideration, I may in fact be violently agreeing with the judge. "Actual intelligence" may simply refer to what I'm saying that the lawyer should have done.

[–] [email protected] 18 points 3 weeks ago

But this is exactly what AI is being marketed toward. All of Apple's AI ads showcase dumb people who appear smart because the AI bails out their ineptitude.

[–] [email protected] 8 points 3 weeks ago

I've been saying this for ages. Even as someone who's more-or-less against the current implementation of AI, I think people who truly believe in AI should be fighting the hardest against bad uses of it. It gives AI a worse black eye every time something like this happens.

load more comments (2 replies)
[–] [email protected] 22 points 2 weeks ago (1 children)

For the last time, people need to stop treating AI like it removes their need for research, just because it sounds like it did its research. Check the work your tools do for you, damn it.

[–] [email protected] 9 points 2 weeks ago (1 children)

It's Wikipedia all over again. Absolutely feel free to use the tool, e.g. Wikipedia, ChatGPT, whatever, but holy shit check the sources, my guy. This is embarrassing.

[–] [email protected] 9 points 2 weeks ago (1 children)

The best use, for me, is asking ChatGPT to give me five (or however many) scholarly, peer-reviewed articles on a topic. Then I search for said articles by title and author name on my school library database.

It saves me so much time compared to doing a keyword search on said same database and reading a ton of abstracts to find a few articles. I can get to actually reading them and working on my assignment way faster.

AI is a great tool for people who use it properly.

load more comments (1 replies)
[–] [email protected] 17 points 3 weeks ago

Works tirelessly? No, AI here!

[–] [email protected] 11 points 3 weeks ago

Nice all the work that the lawyers saved will be offset by judges having to verify all the cases cited

[–] [email protected] 7 points 3 weeks ago (2 children)

All you do is a quick search on the case to see if it's real or not.

They bill enough each hour to get some interns to do this all day.

[–] [email protected] 7 points 3 weeks ago (1 children)

I'm pretty sure that just doing "quick searches" is exactly how he ended up with AI answers to begin with.

load more comments (1 replies)
load more comments (1 replies)
[–] [email protected] 7 points 3 weeks ago

Why would one even get the idea to use AI for something like this?

"Two things are infinite: the universe and human stupidity, and I'm not sure about the universe."

load more comments
view more: next ›