Almost every engineering team we’re working with right now is navigating the same thing.

AI has changed how engineers do their jobs. It has also changed how they interview. And most hiring processes have not caught up.

That second part is the real problem. Interviewing engineers has always been hard. But over the last 18 months, we have watched it get harder, and it is not because the engineers changed. It is because the tools they use every day quietly stopped matching the tools most companies still use to evaluate them.

When your process falls behind like this, you lose signal. You get to a final round with a candidate and you are not entirely sure you can trust what you just saw. That is a rough place to land, especially after your team has already spent weeks of calendar time getting there.

What we’re seeing across the teams we partner with.

A few patterns keep showing up, no matter the industry or stage of the company.

Coding interviews built around syntax recall are measuring the wrong thing. Asking a candidate to remember the exact method name for reversing a linked list tells you what they memorized, not whether they can build or maintain your product. That gap existed before AI. AI just made it impossible to ignore.

A well prompted model can breeze through most standard algorithm challenges. The classic whiteboard style problem, the kind that has anchored technical interviews for a decade, is now something a candidate can solve with help in real time if your process allows any outside tool access, and sometimes even if it does not. That does not mean every candidate is cheating. It means the exercise itself no longer proves what it used to prove.

The result is the part that should worry every hiring manager. Engineers who could have been great hires are potentially being filtered out. Someone who is a strong collaborator, a clear communicator, and genuinely excellent at shipping real product work can still stumble on an artificial, high pressure puzzle that has nothing to do with the job. Meanwhile, a candidate who is skilled mainly at prompting a model convincingly can sail through the same puzzle and look like the stronger hire on paper.

Neither outcome is good for your team. One costs you a great engineer. The other costs you a bad hire down the road, once the puzzle solving skill that got them through your process does not show up in their day to day work.

Why this is accelerating now.

This is not a slow, gradual shift. It has moved fast over the past year and a half, and it will keep moving. Engineers at every level, from new grads to staff engineers, are expected to work fluently alongside tools like Claude and Claude Code now. Product teams increasingly ask candidates to demonstrate exactly that skill in the interview process itself.

That creates a strange tension. Companies want to see whether a candidate can use AI well, because that is genuinely part of the job now. But most interview formats were never designed to separate “uses AI well to solve real problems” from “used AI to get through an artificial test.” Those are two very different skills, and right now, a lot of processes cannot tell them apart.

A resource worth sharing.

We have the privilege of working closely with CoderPad, which gives us a behind the scenes look at how their team is thinking about this exact problem across the industry.

They recently put together a short quiz to help talent acquisition teams understand how much risk AI actually poses to their current hiring strategy. We have shared it with a few of our clients already, and it has been a genuinely useful starting point for identifying where a process needs attention. It will not hand you a finished new interview loop, but it is a fast way to see where the gaps are.

Take the quiz here to see how much risk AI poses to your current hiring strategy.

Where to start fixing your process.

If your team is feeling this gap, here is where we would tell you to start.

Separate reasoning from output. Instead of grading whether a candidate produced the correct final answer, pay attention to how they got there. Ask them to talk through their thinking out loud, question their own approach, and explain tradeoffs. That is much harder to fake, whether or not a model is involved.

Decide your AI policy on purpose, and say it out loud. Some teams want to see how a candidate uses AI tools, since that mirrors the actual job. Others want a clean, unassisted read on core problem solving. Both are defensible. What is not defensible is leaving it ambiguous and then being surprised when candidates make different assumptions than you did.

Move toward real work, not puzzles. A short, realistic scenario pulled from the kind of work your team actually does will tell you more than a classic algorithm challenge ever will. It does not need to be elaborate. It needs to be honest about what the job requires.

Bring in a live component. A pairing session, a code review exercise, or a system design conversation is much harder to outsource to a model in the moment. It also gives you a much better read on how someone will actually work with your team.

Debrief fast, and compare notes across interviewers. If two people on your panel walked away with very different reads on the same candidate, that is often a sign your rubric needs updating, not that the candidate is the problem.

Keep your north star.

None of this means the fundamentals of a good interview process have changed. You are still trying to find out whether someone can do the work, communicate clearly, and grow with your team. AI has not changed what you are looking for. It has changed which exercises can actually tell you that.

We understand this is a hard problem to solve on your own, especially while you are also trying to fill the role. If you want a second set of eyes on your interview process, or you just want to talk through what you are seeing with your own candidates, we are here to help.

Frequently asked questions

How has AI changed technical interviews for engineers? AI tools have changed both how engineers do their jobs and how easily they can get through traditional interview formats. Classic coding challenges that test syntax recall or standard algorithm knowledge no longer reliably show whether a candidate can do the actual work, since a well prompted model can solve many of them.

Why do coding interviews that test syntax recall no longer work? They were always testing memorization more than real ability, but AI tools have made that gap much harder to ignore. A candidate can now get through a syntax heavy question with outside help, which means the exercise stops telling you anything useful about their actual skill.

Is my current hiring process at risk from AI assisted interviewing? Most teams have some exposure, especially if their process still leans on take home coding tests or algorithm puzzles without a live, real time component. A short risk assessment, like the quiz CoderPad has put together, is a fast way to see where your specific process stands.

Should candidates be allowed to use AI tools during a technical interview? There is no single right answer, but there needs to be a clear one. Decide whether you want to evaluate AI assisted work or unassisted reasoning, communicate that policy to candidates up front, and make sure every interviewer on your panel is applying it the same way.

What should engineering teams do first to update their interview process? Start by identifying where your current process still relies on memorization or puzzle solving that AI tools can shortcut. Then shift toward live problem solving, real scenarios pulled from your actual work, and a clear, consistent policy on AI tool use during interviews.