You’re Always Hiring the Wrong Engineers

I’ve worked for a dozen companies, and only a couple of those companies have actually been a good fit for me. In most cases, I’ve blamed myself for not being able to change to meet the needs of my employer, or I’ve blamed my employer for not being able to change to meet my needs. Until recently, though, I never really thought about how this was just a gargantuan failure of the interview process.

Interviews are the process by which you vet your candidates and hire the right person for the job…at least in theory. In practice, the interview process is just an exercise in frustration where you talk to lots of people and then eventually settle for the least bad candidate. This is true of my experience as a candidate, as an interviewer, and as a hiring manager. Most of the hiring managers I talk to have the same experience and end up with the same number of bad fits for the roles they have to fill.

Managers I’ve worked for have claimed that hiring poorly is part of the experience–that there’s nothing you can do to vet candidates perfectly for technical ability in a role or to determine whether or not they’ll be able to work in the culture you’ve created. They’ve also claimed that the only way to be good at hiring is to have an unquantifiable quality that allows you to see who candidates really are by “getting a feeling” for them.

I think that’s nonsense. The real problem is that interview processes and the criteria we use to evaluate candidates are completely different from the criteria we use to assess their performance once they’re in the job.

For example, I once designed an interview process that required candidates to reverse a linked list and find the lowest set of three values in an array. Each of the problems had some sort of story around it–finding the lowest set of three values was supposed to be valuable for determining the coldest days of summer or something. I thought that this was clever and that it would help me understand how people write their code, how they approach problems, and what experience they’d had with standard solutions to algorithms. What I found, though, was that the candidates who did well in the interview process would often not be great fits for the role I was hiring them to fill.

This started my retrospective on my hiring process. I questioned why my clever interview questions didn’t give me the insight I needed into how people would actually perform on the job. So, I added more segments to the process. I asked about previous projects; about what they were proud of; about how they solved problems, and how they admitted mistakes. I asked them how their friends would describe them and posed them riddles that were supposed to show how generally bright they were.

Still, I always seemed to end up with the same mix of people on my teams–one or two top performers, two to four adequate engineers, and one or two poor performers.

How could that be if I was constantly “improving” my process? Shouldn’t I start seeing more top performers and fewer middling and poor performers? Why wasn’t it working? When I stumbled upon the answer, it seemed so obvious that it stopped me cold. I was evaluating candidates on one set of criteria during the interview process and judging them by another set when I did their annual reviews. How could I be so dumb?

So, I set out to completely redesign my interview process. I diligently thought through the job description and made sure I was reviewing my engineers against those criteria. I designed a work sample test that was representative of the work they would actually do and set out a clear rubric by which the work sample would be judged to avoid individual preferences on how to write code. I designed a process that allowed me to see how they interacted with code in real time and how they asked questions to get clarification when the requirements were ambiguous. Once I was done, I was massively proud of the process.

And then, I hired one person, and I needed to do this all over again for the next role.

Dozens–maybe hundreds–of hours of work, and all I’d done was hire one senior engineer. The engineer did turn out to be a great fit for the role, though, so at least there was that. Unable to keep up with the demands for this, I turned to some of the online platforms that are supposed to help. I found that all you get on those platforms is the software puzzles that I already determined to be poor predictors of performance. Unless I was hiring someone to apply their rote knowledge of algorithms to implement bubble sort, I wasn’t going to get an online platform that would be able to tell me if an engineer would perform well in the role I had for her.

Ultimately, I found that there was no service available that would do what I wanted it to. I could use some of the online submission tools like CodeSubmit that would provide some analysis for a take-home test I designed, but I’d still have to design the test and the rubric to grade it. That was the hardest part of the process and the most time-consuming. If all I was doing was hiring, that would be fine, but I was also managing two teams of eight, writing code myself, and dealing with internal and external teams that didn’t understand how engineering worked.

For a while, I gave up. I wasn’t actively hiring or looking for a job, and it wasn’t that important to me. Then I landed an interview with a big company–you know the ones I’m talking about–and I was subjected to some of the same software puzzles I’d inflicted upon engineers years before. Plus, I had to solve these puzzles in front of a guy half my age that was staring at me the whole time. My anxiety kicked in, and I froze up. I couldn’t remember how to do half the things he was asking me to do, and the other half I’d never done.

I couldn’t implement bubble sort–I’d never had to–because I could just call sort(). I couldn’t remember how to dump variables out to examine them because the blood was pounding in my head. I failed the interview segment miserably.

The interviewers were kind and said that it was only one part of the interview process and that failing that didn’t mean that I’d failed the interview, but they were lying. Even if they’d loved everything else I’d said, the only thing that mattered to them was the image of the red-faced, sweaty candidate who couldn’t even do basic computer things when asked.

The experience threw me for a loop–I’d never really thought about how uncomfortable that process could be for folks, and I’d never been that anxious during an interview before. I started researching whether or not that process was necessary and found that not only was it not necessary, it was harmful.

I knew from my research that live coding interviews weren’t terribly informative and that they screen out people with test anxiety and with anxiety in general. They screen out more women than men, and they screen out older programmers in greater numbers than younger programmers. In short, they make engineering departments look like the young, male-dominated places they are and have been.

My experience kick-started more research into how to interview better and how to help others do the same. I helped to launch a project with my employer to overhaul the interview process and help eliminate bias and improve candidate experience. I worked with other hiring managers to share my views with them and talk about how to measure candidates for positions accurately. All this went precisely nowhere. HR didn’t want to revamp their process. Hiring managers didn’t have the time to make sure their needs matched HR’s job description. Engineers didn’t have the time or inclination to develop representative tests for candidates, and they didn’t want to have to interview anyway.

Finally, I realized that companies wouldn’t be able to do this internally. They’d need someone to come in to consult or offer a product that would help them do this. That’s why I founded Gordian Knot. We’re a consulting and engineering service that will work with you to develop a better hiring process that will improve candidate experience, help you be more confident in your choice of engineers, help hire a more diverse workforce, and save you money and time by allowing your engineers and managers to focus on their existing jobs.

If you’re interested in learning more, schedule a consultation with me today.