More effective interviews
David Smith over at baleful.net makes some interesting points about the length of most interviews:
So mathematically, you will most likely get the highest confidence interval with: 1) Resume screen, 2) Phone interview, 3) In-person interviews 1-3. From the above, this should represent about 50% of the total causes, but should produce 91% of the total effect. Adding additional interview steps after that 91% brings only incremental improvement at best and backslide at worst.
He makes an extremely compelling argument, and I encourage you to read the entire piece. That said, I still prefer a full day of interviews as both the interviewer and interviewee.
The interviewee angle is easy. I enjoy interviews. I like to dig into my potential employer. I want to grill your second-string players. I want to hear how junior people feel treated. I want as much information as possible before making my choice. But I know this is just me, and people who are less comfortable with interviews probably prefer shorter ones. I also admit I don’t think I’ve learned anything in the second half of a day of interviewing that would have made me turn down a job. But I have learned things that helped me in my job once hired.
The benefits of full-day interviews for the interviewers is much more complex. There are several factors:
- We have diverse backgrounds and expertise, and each group brings a unique perspective. Candidate postmortems are not dominated by the same couple interviewers.
- I want to give as many people experience interviewing as possible. I consider it an important skill. Limiting things to three in-person interviews means the interviewers are all “musts” and I don’t get to experiment at the periphery with groups or combinations.
- People want to be a part of the process. I’ve personally felt frustrated when left out of the process, and I know I’ve frustrated others when I’ve left them out.
For a developer role, I want them to meet with at least: founders, ops, lead developer, two developers, myself. We’re at an absolute minimum of 7. That is with a narrow set of views, without inexperienced interviewers, and leaving good people out. What am I supposed to do?
- For starters, the interview process should be more transparent and collaborative. Ask the interviewer if they want a full day, two half days, morning or afternoon, etc.
- No group lunches. I’ve never gotten useful feedback from a group lunch. Keep it down to one or two people. A candidate just doesn’t want to embarrass themselves, so they just shut up, and side conversations dominate.
- Avoid solo interviews. I used to hope to solo interview everyone. But over time, I’ve found that pairing on interviews enhances the benefits listed above. There are still times I will want a solo interview, but in general I will pair.
- Cut the crap. Interviewers should state their name and role. Don’t bother with your history unless asked. Don’t ask questions that are answered by a resume. Instead of “tell us about yourself” how about “tell us what you’re looking for”.
- Keep a schedule. Some people are very bad at managing time. If someone isn’t done, too bad, keep things moving. They will eventually learn how to keep interviews to their allotted time.
Thanks to David for the insightful post. I’ll continue to keep full-day interviews, but we’ll definitely change some things up.
I can’t quite imagine that this works in the 1000+ people shop where I work. And I don’t think that it will ever be possible to judge the most important thing in an interview – how you work WITH somebody when you do actual work. Used to be that probation was for finding that out.
But usually it’s 2 hours. I don’t think I’ll know that much more after 3 or 4 or 5 hours. 80-20 rule. But I guess the time when you have 80% of confidence varies by candidate/company/job/your need for that guy. But as I said above, I don’t think you can get 100% confidence unless you actually work with the candidate for a while.
I definitely agree on bringing other people to the interview. Especially since I myself don’t always know the itty gritty details some of the people on my team would care about. I usually stick to 3 maximum. I think any more might actually intimidate people. Those are then people who would spend the most time working with the new guy.
Group lunches? Umm nope. I don’t really want to give people a “welcome to the team!” impression when they’re not yet on the team. Also no company is perfect. People may gripe about small issues, but a candidate who has little inside knowledge may just interpret it all wrong and get the wrong picture of the company. Not good. Plus you learn if the guy is a nice fellow to have lunch with, but that doesn’t necessarily mean he’s a nice fellow to work with, someone with good work habits.
“Don’t ask questions that are answered by a resume” – I think you should! Some people have a hard time remembering their resumes – and not just details! (alarm bells?) I usually make one or two broader references to the resume, sort of as a cross check. We had one candidate who wrote himself a needlessly complicated 5 page resume, who then forgot what he wrote there. He also forgot to bring a copy of his own resume. He was a Sr guy working for a well known vendor of 3D software. It was pretty awkward when he asked to borrow my copy of the resume with all my notes. Sorry, nope. Can’t give that to you.
I wouldn’t talk to someone with a 5 page resume!
I’m not sure what you mean by “I can’t quite imagine that this works in the 1000+ people shop where I work”. The interview process at companies like Google, Apple, or MSFT is far more rigorous, often taking multiple days. Google is well known for having people interview you outside of your direct sphere of employment. They are also extremely cautious with who they hire, so I’ve heard. I’ve touched on this rigor before: https://www.robg3d.com/2014/06/hiring-your-cake-and-eating-it-too/
So it’s definitely a factor of culture and not size. I was hired at my first job at a small company with maybe one phone interview, and then I moved across the country. There’s no one-size-fits-all, but definitely re-examining our thinking and practices will yield better hires.
@rob. Yes you’re right. Not so much an issue of size, now that I think of it. More an issue of capability and resources. I never worked for one of those shops that seemed to have both, at least not in sufficient quantity. It never seemed to be possible to spend that much time interviewing (and I’m not convinced that more time automatically results in a better outcome) I’m pretty sure the top companies of the world have awesome hiring processes – or at least you’d expect that they have.
About 5 page resumes… yeah I’m totally with you. But I’m in China ;) A country catapulted into the 21st century in just 30 years, where not long ago their dictator tried to turn everyone into farmers and simple workers, and where the state would hand you a job. Resume writing skills, and self marketing skills are almost non existent. Parents can’t help with that. Universities can’t help either. There’s also no sense yet, among the masses, that self marketing is important in a market driven society.
But 5 pages ain’t the worst. The worst are those fill-in-the-blank resumes, which even include a field for the political party you belong to… strangely enough, most people here are communists, or so they claim ;)
I’m glad you liked my thoughts and gave some thought to your interview setup. For what it is worth, I didn’t feel wrecked after interviewing with you guys. That was a first for me–interviews are stressful for me.
My post is partly influenced by the CTO of a company who told me that he knew his interview process was flawed but it was what it was (paraphrasing). When I received the schedule of twelve interviews over two days with five hours of Skype interviews in the middle and a group lunch, I politely opted out of the process.
Interviewing is very hard for me. I’m a fully-functioning adult with a pretty decent career behind me, but to this day interviewing wrecks me as much as the first one did. It all has to do with how my brain is wired. The brain that gives me intelligence, meticulousness, creativity, and calm demeanor also gives me social anxiety and introversion.
Then, I realized that interviewing is an attempt to solve a problem, but I don’t think a lot of processes have put much thought into the problem they’re trying to solve. A lot of processes have taken on that Google/Apple/Amazon/Facebook vibe, and added complexity and length. However, I question whether the length/complexity has been added to address a specific flaw.
If I viewed interviewing as a solution to a problem, then I realized that there are a lot of things _I’ve_ done traditionally in interviews that don’t make sense. I also realized that process could inject a lot of noise into the signal. For example:
– Group lunches/happy hours: destructive to candidates with social anxiety and tests something that doesn’t affect job performance.
– Coding quizzes: a 20 minute problem with a tricky solution doesn’t correlate to 2 weeks of inter-related simple problems. See also trivia quizzes.
– Large panel interviews: having half a dozen interviewers firing questions at you at once feels like a firing squad.
– Super-long interviews: people with a variety of non-job-affecting disorders will eventually crash out.
– Stressor interviews: if your culture is so stressful that you need to upset a candidate during the interview, fix your culture first.
– Excessive use of video chat: often has setup issues and some people just flat out don’t like being in front of a camera.
Ever since then, I’ve started to wonder about how you could make the kindest, gentlest interview process with decent confidence levels. I’ve been considering how you could remove the most noise and get the candidate at ease, and I’ve come up with ideas like:
– Before the interview, make sure the address is well-known and tell the interviewee that the dress code is “no holes in your clothes”.
– Interviewee is shown where the bathroom is right away.
– Five-minute breaks between interviews.
– Behavioral questions on previous projects rather than coding questions.
– Limit whiteboard usage but have one available to allow candidates to draw about answers.
– Interviews are no more than 2 interviewers.
– If interviews run over lunch, give the interviewee the option of eating alone or with a buddy.
– Never run an interview over a mealtime without letting the interviewee eat.
If space permitted, I might even create a dedicated interview room with:
– Comfy armchairs
– No table separating the candidate and interviewer
– Water/soda/snacks stocked for the candidate in the room
Essentially, I want to make the process more like a convivial chat. I think I would get a more accurate read on who the candidate is then. It’s a slightly utopian plan, and I don’t think it would work exactly the way I envision. It was a fun thought exercise, though.
I don’t think any of those ideas are utopian! In fact I’ll be sending your comments to Barb to see what we can do (most likely we won’t have armchairs or lose a table since we’re limited on space). And I’ll definitely be taking the suggestions to heart. Thanks David!
You’re most welcome. i’m glad you found my thoughts interesting. I’ve been thinking a lot about how to build great teams and culture, and it’s edifying to know someone else appreciates my musings.
The dedicated parlor-style interview room is the least practical of my ideas. It’s the end result of thinking about how to remove all sub-conscious queues from the process.