this post was submitted on 10 Apr 2024
1308 points (100.0% liked)
Programmer Humor
22729 readers
1497 users here now
Welcome to Programmer Humor!
This is a place where you can post jokes, memes, humor, etc. related to programming!
For sharing awful code theres also Programming Horror.
Rules
- Keep content in english
- No advertisements
- Posts must be related to programming or programmer topics
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
This is what GPT 2 did. One day it bugged and started outputting the lewdest responses you could ever imagine.
Thanks for sharing! Cute video that articulated the training process surprisingly well.
Dude what a solid video! Stoked to watch more vids from that channel!
The best part is they don't understand the cost of that retraining. The non-engineer marketing types in my field suggest AI as a potential solution to any technical problem they possibly can. One of the product owners who's more technically inclined finally had enough during a recent meeting and straight up to told those guys "AI is the least efficient way to solve any technical problem, and should only be considered if everything else has failed". I wanted to shake his hand right then and there.
Laughs in AI solved problems lol
Using another AI to detect if an AI is misbehaving just sounds like the halting problem but with more steps.
Generative adversarial networks are really effective actually!
As long as you can correctly model the target behavior in a sufficiently complete way, and capture all necessary context in the inputs!
Lots of things in AI make no sense and really shouldn't work... except that they do.
Deep learning is one of those.
The fallout of image generation will be even more incredible imo. Even if models do become even more capable, training off of post-'21 data will become increasingly polluted and difficult to distinguish as models improve their output, which inevitably leads to model collapse. At least until we have a standardized way of flagging generated images opposed to real ones, but I don't really like that future.
Just on a tangent, openai claiming video models will help "AGI" understand the world around it is laughable to me. 3blue1brown released a very informative video on how text transformers work, and in principal all "AI" is at the moment is very clever statistics and lots of matrix multiplication. How our minds process and retain information is by far more complicated, as we don't fully understand ourselves yet and we are a grand leap away from ever emulating a true mind.
All that to say is I can't wait for people to realize: oh hey that is just to try to replace talent in film production coming from silicon valley
Building my own training set is something I would certainly want to do eventually. Ive been messing with Mistral Instruct using GPT4ALL and its genuinely impressive how quick my 2060 can hallucinate relatively accurate information, but its also evident of limitations. IE I tell it I do not want to use AWS or another cloud hosting service, it will just return a list of suggested services not including AWS. Most certainly a limit of its training data but still impressive.
Anyone suggesting to use LLMs to manage people or resources are better off flipping a coin on every thought, more than likely companies who are insistent on it will go belly up soon enough
I see this a lot, but do you really think the big players haven't backed up the pre-22 datasets? Also, synthetic (LLM generated) data is routinely used in fine tuning to good effect, it's likely that architectures exist that can happily do primary training on synthetic as well.
AIs can be trained to detect AI generated images, so then the race is only whether the AI produced images get better faster than the detector can keep up or not.
More likely as the technology evolves AIs, like a human, will just train real-time-ish from video taken from it's camera eyeballs.
...and then, of course, it will KILL ALL HUMANS.
I'm sure it would be pretty simple to put a simple code in the pixels of the image, could probably be done with offset of alpha channel or whatever, using relative offsets or something like that. I might be dumb but fingerprinting the actual image should be relatively quick forward and an algorithm could be used to detect it, of course it would potentially be damaged by bad encoding or image manipulation that changes the entire image. but most people are just going to be copy and pasting and any sort of error correction and duplication of the code would preserve most of the fingerprint.
I'm a dumb though and I'm sure there is someone smarter than me who actually does this sort of thing who will read this and either get angry at the audacity or laugh at the incompetence.