Let’s talk about the interview process

Tech company interviews have become the stuff of legend. Tales of impossible questions, nerve-wracking whiteboard coding, bizarre brain teasers, you name it: the Internet is full of examples of strange and wonderful experiences during interviews at leading technology companies.

In this article, we’ll explore some of the aspects of interviewing candidates that I feel are important. Some of these aspects have no definitive answer or best practice, but I’ll offer my own opinions on what I think works and what doesn’t. A lot has been written on the process of technical interviews from the side of the candidate. I’d like to explore what it’s like to be on the interviewer’s side of the table, where you may need to define and run this process yourself.

Let’s start with the obvious stuff: why do we do interviews? Well, we’re looking to make a decision as to whether we would want to work with that particular person. Can they help our company succeed by bringing themselves and their skills into the organization? I divide what we, as a technology company, are looking for into three major facets: technical skill, culture fit, and their suitability for the role that they applied for. How can we perform interviews so that we can find these characteristics out as quickly as possible, whilst leaving the candidate as happy as possible with the experience, regardless of the outcome?

How many interviews?

I’ve read horror stories of candidates wading through over ten interviews before being offered a job. This may be a luxury that larger companies with full candidate pipelines can afford, but my own experience of hiring candidates into smaller organizations, especially at the start-up or small business phase, is that candidates are often interviewing at numerous companies – a number of which that could be more widely known than you – so having a fast and efficient interviewing process is beneficial to both the interviewee and the interviewers. It demonstrates that you’re efficient and, most importantly, that they are wanted.

When interviewing for niche or very senior roles, you may not want to follow a particular template. However, for most engineers at varying levels of experience, I have had success with an interview process consisting of two to three stages: two actual interviews and one optional take home test. The take home test is not given to all candidates; it’s used when there is limited evidence of their coding ability (e.g. graduates; those who are unable to put lots of their work on Github; or those changing their primary programming language) or if we are on the fence about their ability.

Some companies do more, or less, interviews. I can’t vouch for the success of approaches that are wildly different to what I outline in this article. For example, Google and Facebook have been known to do intensive day-long sessions of about six interviews, but I do not have any data on whether this reduces the false-positive rate for quality compared to the process that I am used to following. I do know they make candidates quite intimidated, however…

The structure that I have been using is as follows:

• 1st interview: here, both parties are getting to know each other, and we are assessing cultural fit. The hiring manager is present here along with another person if they wish to bring them along. This is typically a member of staff on their team, or someone with a similar skillset from elsewhere in the organization.
• Optional take-home test: a fairly open-ended coding challenge that allows the candidate a number of different avenues to showcase their ability in a self-directed way.
• 2nd interview: a technical deep dive with people who would be peers of the new hire. For example, if the hiring manager is the team lead, the 2nd interview would be with 2 of their engineers.

Let’s have a look at each of these interview stages in turn.

Interview 1: Getting to know you, and cultural fit

The first interview is where the candidate (usually) is the most nervous. One important thing to remember is that no matter the outcome of the interview process, even if the candidate is nowhere near experienced enough for the role, they should come away thinking that your company is a great place to work and that they would reapply again in the future. I cannot stress this enough. Be nice. Interviews are not a place for you to show how clever you are. It should be the opposite.

In this interview, it’s good to stay broad and let the candidate do the talking. This uses a similar technique to the one discussed in our article about 1 to 1s: listen, nudge and absorb. Some good questions here are:

• Tell me about your current role and what you’re currently working on.
• How did you get into technology/computer science?
• What’s the most interesting piece of technology at your current company?
• What’s the project that you’ve been most proud of? What made it great?
• What’s the worst project you’ve worked on? Why was it the worst?
• What’s the most interesting open source project out there at the moment?

These questions are just a sample, but the idea remains the same: you want to allow the interviewee the space to talk and tell their story, allowing them the chance to put aside the nerves and let their true everyday selves show. Along the way, you’ll begin to get an idea of what motivates them: is it technology, people, business, or all three? Do they enjoy being hands on or do they tend to delegate tasks to others? Are they well-read on the state of the art, or are they working at a company that has limited their technical choices? Are they aware of them at all?

I also like to try and assess their analytical brain: for example, if they mention that they worked on a file transfer application, probe deeper: what protocols were they adhering to? Did it support multiple concurrent connections? Did it have to deal with the out-of-order delivery of packets? How were the consistency of the transferred files checked? Excellent candidates would have these thoughts whizzing through their brains while they were doing the project, even if they didn’t need to implement all of it.

At the end of the first interview, I assess the candidate around three areas:

• Technical knowledge: does this candidate impress with their technical prowess when their years of experience are factored in? Remember that seniority is not just years under the belt.
• Cultural fit: would I be happy sitting next to this candidate and working with them every day? Would I be equally happy grabbing lunch with them and talking about something other than work?
• Suitability for the role: do I feel that the current role is right for them? Are they too junior and therefore need more mentoring than we can offer right now, or are they too senior which means that there is a risk of them getting bored?

If those three areas are satisfied, then I use one final check: if the hiring committee was on the fence and I had to argue the case for this candidate with the CTO, which way would I think he or she would swing on their judgment? I find my gut feel is usually correct here. Imagine yourself presenting this candidate to your own manager or even your CEO: would you argue their case if your own reputation depended on it?

Optional: take home test

Depending on the candidate, I’ve found it useful to give a take home programming test. Let me stress that we treat this as optional, as the need for it varies from candidate to candidate. If we are recruiting a very senior member of staff who is prolific on Github, then it’s unnecessary. The evidence is clear. However, if we don’t have a lot of evidence of their coding ability, which is especially true of graduates or very junior candidates, then it can be helpful to see how they tackle it.

I won’t reveal exactly the coding challenges we use, but we don’t set ridiculous brain teasers that require intricate knowledge of bit-shifting. That’s not an accurate reflection of day-to-day work as an engineer at a SaaS company like ours. Instead, we have written a toy system that performs some basic functionality, and then specified a variety of different ways that the system could be improved. For example, there is a list of new and improved functionality that could be added, a whole host of tests that could be made better, refactoring that could be done, and documentation that could be written. We let candidates know that they can take any of these areas to any level of depth that they like. We’ve had some very novel submissions this way, and it doesn’t pigeonhole our successful candidates to be the ones best at solving advanced exercises in Knuth’s The Art Of Computer Programming. Our toy system performs some functionality that our real system does, which makes it much more interesting to the candidate as well.

Once they submit the coding challenge, we have it forwarded to an internal mailing list that our engineers monitor. They are encouraged to volunteer to review it and give positive and negative feedback and a recommendation as to whether they should proceed in the interview process. Those that do not proceed still get the feedback so they can understand why we decided not to continue with their application. Again, candidates that fail at this stage should still feel that the company is fair and supportive, and they should feel that they could improve and reapply in the future.

Interview 2: let’s get technical

At the second interview stage, we invite two people who would be peers of the candidate in the organization. They can spend this interview getting deep technically and working out whether they would like to have the candidate working on their team. To add some continuity to the interview stages, if they have succeeded at a take-home test, then beginning the interview by talking through it is a great way of making them feel comfortable (after all, they’ve already succeeded!) But, inevitably, you’ll need to do some technical exercises together.

At this point, the candidate has already had an opportunity to show you the kind of code that they write, either by completing the take-home exercise or by sharing their Github with you. The success of the technical interview isn’t hinging on them working out the correct answer for all of the problems that you set (although that’s nice also!), but instead, you’re looking to see how they work on problems in person, and also how they communicate with others when doing so.

I would recommend either using a whiteboard or using an IDE of their choice on a computer, but these two workspaces are used for different types of interview question.

Firstly, whiteboards. There are many horror stories online. The nervousness of standing up in front of interviewers; accidentally leaning on the board and getting covered in ink; sweating so much that the pen rubs off; you name it, it’s happened. The truth is that whiteboards aren’t for everybody. They also suit a particular style of problem. For me, these are questions that involve drawing an architectural diagram of a system, describing the steps of an algorithm, or writing in a loose pseudo code style. Expecting whiteboard code to be transcribed and then compiled successfully is missing the big picture of the exercise: understanding the candidate’s thinking process. I also encourage standing at the whiteboard with the candidate and working with them by asking probing questions, rather than sitting there silently making notes while they have their back to you.

Good whiteboard questions could be:

• Imagine you’re building a chat application from scratch. What would the architecture of the system look like, and how would you build it?
• Imagine that you’re writing a Java method that is being given tweet objects in real-time. How could you keep track of the user that has tweeted the most?

Whiteboards are not where your engineers are going to be writing most of their code in their day job. If you want to do an in-depth coding exercise in person, then a good technique to use is pair programming on a computer, just like they would if they were working with you. Ask the candidate ahead of time what their OS and IDE preference is and get the computer set up ahead of time. You can then spend some coding together. Even better, you could try and fix a real bug in your system. This will give good insight into how they think, work, use git (or similar) and organize themselves. Try and have some fun while doing it too.

Deciding whether to make an offer

Once both interviews have been done, you should have feedback from all of your interviewers. We have most recently been asking for a score of 1-4 along with written notes. All notes should be written in such a way that if they were revealed to the candidate they would learn how to improve rather than be offended. Keep in mind that in the UK, candidates can request feedback via the Data Protection Act.

• 1 – Definite no hire: The candidate was definitely not right for the role, or didn’t demonstrate the experience required.
• 2 – Marginal no hire: The candidate may have been a consideration, but there were red flags during the interview that would result in us arguing against their application.
• 3 – Marginal hire: The candidate performed suitably, but didn’t impress enough to make us instantly want to offer them the job, however, we would argue their case.
• 4 – Definite hire: The candidate was excellent, and we would be silly not to hire them right now.

As an interviewer, there is an internal emotional conflict between being nice and doing what is right for the organization. When faced with this situation, you have to leave as much emotion at the door as possible. If you feel like you want to give a candidate an opportunity because their poor performance could have been due to nerves, but you know, truly, that they are not meeting your standards, then you need to suppress that emotion and reject them. It is much easier to break bonds with a person at the interview stage than it is after you’ve hired them and are struggling on critical projects.

If the consensus averages on a high 3 or a 4, then go ahead and make that offer. If the average is a low three, then get together in a room and debate. Any lower than that and it’s a no. There are of course other factors such as the market conditions and the number of candidates that are, and are predicted to be, in the pipeline. It’s never easy: a candidate with a few reservations may actually be the best you’ll see all year for that role. But how do you know? Well, you don’t. You can only make your best judgment at any particular moment in time. Good luck.