this post was submitted on 20 Jun 2024
527 points (100.0% liked)
Technology
67987 readers
3298 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I interviewed a candidate for a senior role, and they asked if they could use AI tools. I told them to use whatever they normally would, I only care that they get a working answer and that they can explain the code to me.
The problem was fairly basic, something like randomly generate two points and find the distance between them, and we had given them the details (e.g. distance is a straight line). They used AI, which went well until it generated the Manhattan distance instead of the Pythagorean theorem. They didn't correct it, so we pointed it out and gave them the equation (totally fine, most people forget it under pressure). Anyway, they refactored the code and used AI again to make the same mistake, didn't catch it, and we ended up pointing it out again.
Anyway, at the end of the challenge, we asked them how confident they felt about the code and what they'd need to do to feel more confident (nudge toward unit testing). They said their code was 100% correct and they'd be ready to ship it.
They didn't pass the interview.
And that's generally my opinion about AI in general, it's probably making you stupider.
I've seen people defend using AI this way by comparing it to using a calculator in a math class, i.e. if the technology knows it, I don't need to.
And I feel like, for the kind of people whose grasp of technology, knowledge, and education are so juvenile that they would believe such a thing, AI isn't making them dumber. They were already dumb. What the AI does is make code they don't understand more accessible, which is to say, it's just enabling dumb people to be more dangerous while instilling them with an unearned confidence that only compounds the danger.
Spot on description
Yup. And I'm unwilling to be the QC in a coding assembly line, I want competent peers who catch things before I do.
But my point isn't that AI actively makes individuals dumber, it's making people in general dumber. I believe that to be true about a lot of technology. In the 80s, people were familiar with command-line interfaces, and jumping to some coding wasn't a huge leap, but today, people can't figure out how to do a thing unless there's an app for it. AI is just the next step along that path, soon, even traditionally competent industries will be little more than QC and nobody will remember how the sausage is made.
If they can demonstrate that they know how the sausage is made and how to inspect a sausage of packages, I'm fine with it. But if they struggle to even open the sausage package, we're going to have problems.
Yeah, I honestly don't have any real issue with using it to accelerate your workflow. I think it's hit or miss how much it does, but it's probably slightly stepped up from code completion without "AI".
But if you don't understand every line of code "you" write completely, you're being grossly negligent and begging for a shitshow.
I just don't bother, under the assumption that I'll spend more time correcting the mistakes than actually writing the code myself. Maybe that's faulty, as I haven't tried it myself (mostly because it's hard to turn on in my editor, vim).
Nah perfectly fine take. Each their own I say. I would absolutely say that where it is, not bothering with it is completely fine. You aren't missing all that much really. At the end of the day it might have saved me ten-fifteen minutes here and there. Nothing that's a tectonic shift in productivity.
Yeah, most of my dev time is spent reading, and I'm a pretty fast typist, so I never bothered.
Maybe I'll try it eventually. But my boss isn't a fan anyway, so I'm in no hurry.
It can be useful in explaining concepts you're unsure about, in regards to the reading part, but you should always verify that information.
But it has helped me understand certain concepts in the past, where I struggled with finding good explanations using a search engine.
Ah, ok. I'm pretty good with concepts (been a dev for 15-ish years), I'm usually searching for specific API usage or syntax, and the official docs are more reliable anyway. So the biggest win would probably be codegen, but that's also a relatively small part of my job, which is mostly code reviews and planning.
Yup, you're hired as whatever position you want. :)
Our instructions were basically:
It was technically different (we phrased it as a top-down game, but same gist). AI generated manhattan distance (
abs(x2 - x1) + abs(x2 - x1)) probably due to other clues in the text, but the instructions were clear. The candidate didn't notice what it was doing, we pointed it out, then they asked for the algorithm, which we provided.Our better candidates remember the equation like you did. But we don't require it, since not all applicants finished college (this one did). We're more concerned about code structure, asking proper questions, and software design process, but math knowledge is cool too (we do a bit of that).
College? Pythagorean Theorem is mid-level high school math.
I did once talk to a high school math teacher about a graphics program I was hacking away on at the time, and she was surprised that I actually use the stuff she teaches. Which is to say that I wouldn't expect most programmers to know it exactly off the top of their head, but I would expect they've been exposed to it and can look it up if needed. I happen to have it pretty well ingrained in my brain.
Yes, you learn it in the context of finding the hypotenuse of a triangle, but:
So when I'm interviewing, I try to provide things like algorithms that they probably know but are likely to space on, and focus on the part I care about: can they reason their way through a problem and produce working code, and then turn around and review their code. Programming is mostly googling stuff (APIs, algorithms, etc), I want to know if they can google the right stuff.
And yeah, we let applicants look stuff up, we just short circuit the less important stuff so they have time to show us the important parts. We dedicate 20-30 min to coding (up to an hour if they rocked at questions and are struggling on code), and we expect a working solution and for them to ask questions about vague requirements. It's a software engineering test, not a math test.
I don't want to believe that coders like these exist and are this confident in an AI's ability to code.
My co-worker said told me another story.
His friend was in a programming class, and made it nearly to the end, when he asked my friend for help. Basically, he had already written the solution, but it wasn't working, and he needed help debugging it. My friend looked at the code, and it looked AI generated because there were obvious mistakes throughout, so he asked his friend to walk him through the code, and that's when his friend admitted to AI generating the whole thing. My friend refused to help.
They do exist, but this candidate wasn't that. I think they were just under pressure and didn't know the issue. The red flag for me wasn't AI or not catching the AI issues, it was that when I asked how confident they were about the code (after us catching the same bug twice), they said 100% and they didn't need any extra assurance (I would've wanted to write tests).