Design the Interview Loop

Published on January 4, 2025

Once candidates are vetted by both recruiting and the hiring manager, and we have enough info to think that the candidate could be a good fit, it's time for the candidate to meet more people on the team. The goal at the end of this stage is to make an offer.

Interviews are expensive. The reason we have two vetting stages before bringing in the candidate to meet with the team is to mitigate cost. We should filter out candidates that are obviously not a fit as quickly as possible.

We want every candidate that makes it to the final round stage to have a strong chance of being hired. A good metric for this is gauging what percentage of interviews end with all interviewers saying "no" to the candidate. If it's all no's, there may be a calibration issue. If there is at least one person on the interview team who is inclined to hire after a final round interview, you at least know you're getting close to the right person. All no's could indicate everyone's time was wasted and that there is a calibration or filtration issue upstream. There is probably a defect that needs to be sorted earlier on.

Let's break down the interviewer's responsibilities with some time estimates:

Prep: 30 minutes to review interview prep notes, pick interview questions, and review candidate resume.
Interview: 60 minutes to conduct conversation with candidate
Feedback: 30 minutes to clean up notes and draft feedback for debrief.
Debrief: 30 minutes for debrief

This nets out to two hours of work per teammate. When you have five interviewers, that quickly increases to ten work hours spent per interview that makes it to this stage. That can add up quickly on an hourly rate, especially when you consider it's often your most effective employees are involved in the interview process.

However, this cost should not be a deterrent from being thorough.

Interviewing is an investment in the company's future. To be good stewards of the team's time, it's important we have a clear and thorough process that makes it simple for the team to gather the information needed to make a decision.

In this post, we'll talk about how to design the "interview loop", or the series of interviews that make up a final round prior to giving an offer.

Values & Functional Areas

Start with your core evaluation areas. This is derived from the job scope and requirements.

We want to assess candidates across two key areas: values (as aligned to the company) and skills (as aligned to the job role/scope)

Values reflect how a person thinks and operates. They show how candidates will approach work, whether they'll mesh with the team, and their potential for growth. Values shouldn't be a proxy for the nebulous, bias-ridden "culture fit", but tangible, discrete attributes that can be evaluated more precisely.

Skills demonstrate role-specific competence. They reveal mastery of required skills and how candidates approach their craft. Functional skills are important for the team to build muscle in different parts of the business and filling short-term needs. A job does need to be done, and functional skills are required to do that.

Each area needs associated questions to guide interviewers. While you might identify that the candidate demonstrates the value or skill in any answer, splitting these deliberately helps build a complete picture of each candidate. Picking questions that allow us to work towards getting this info increases our likelihood of hearing what we need to hear from the candidate to make a hiring decision.

A candidate needs both functional skills and values alignment. Strong skills without values fit means they may perform tasks well but struggle within the culture. Strong values without skills means they may need a different role or more experience before they can contribute effectively.

When evaluating the risk of functional gaps, ask yourself: can we train this person on these areas quickly enough for them to contribute meaningfully?

The acceptable size of these gaps depends on your resources - larger companies can often invest more in training while startups may need candidates who can contribute immediately. Evaluate candidates' current abilities against your training capacity to make informed decisions. In general, I lean towards hiring for values over skills, as skills are usually easier to teach -- especially for junior hires.

If you aren't in a position to hire this candidate just yet, make sure you build a relationship with them. Even non-hires can become advocates of the company, share referrals, and join the company down the road: but how you reject the candidate matters. The average company ghosts a candidate they aren't interested in hiring, while top end companies recognize people can grow and change, and look at rejections today as potential hires for tomorrow.

Using these for the interview

Each of these should be split up and assigned to different people across the interview loop.

We do it this way to ensure we get as complete a picture of the candidate as possible. The hiring manager cannot possibly cover everything in a single conversation, nor does the hiring manager have time to do all of the interviews themselves. We can ensure we don't repeat questions and cover the candidate's experience from more angles.

This distribution ensures comprehensive coverage while avoiding repetitive questions. Each interviewer should document both the candidate's responses and their own assessment. Sharing these detailed notes helps the full team benefit from different perspectives, whether that's hiring managers hearing concerns from peers or junior staff learning from senior evaluations. We'll cover what good interview feedback looks like later.

In addition, everyone involved benefits from getting diverse perspectives: whether that's the hiring manager hearing concerns from their peers, junior employees hearing how senior people think and make hiring decisions, and the candidate getting to meet a number of great people from the company that will help them get a sense of who they're signing up to work for.

Picking Interviewers

Picking interviewers is important. Interviewing is one of the biggest privileges you can give a teammate. They not only get to contribute to some of the most important decisions companies will make, but they also are given the opportunity to represent the company to candidates. It's a duty to be taken seriously. Choose interviewers who demonstrate good judgment, recognize their biases, and can evaluate candidates effectively. These are often high performers and people who embody the company's values.

Interviewing can feel like an extra chore: for most roles, especially individual contributors, it is an incremental responsibility on top of their job duties. It's easy to let an email that needs to be sent, an upcoming meeting, or a pressing deliverable distract the interviewer. This hurts both the interviewer's ability to evaluate effectively, but also harms the candidate experience.

Good candidates play careful attention to how the interviewers behave. Their body language. Their word choice. Their tone. All of these are data points for the candidate to gauge if this is a work environment they want to be part of and a sense of the caliber of person they'll be surrounded by. Interviewers embody your company's values and standards. Their conduct directly influences whether top talent chooses to join the team.

When interviews go well, it can be the decision maker for the candidate. If the candidate is impressed and energized by prospective teammates, stakeholders, and peers, they're going to be more interested in the job and company. This is critical to continuing to uplevel the talent in the organization as the organization scales.

As a candidate myself, I've picked jobs based on the quality of my interviews. I've turned down opportunities to join "big name" companies to instead pick the team that I was more excited to work with. Brand may help you attract strong talent, but good interviewers will help close the deal. This is especially true for smaller, upstart companies that don't have the prestigious brand to fall back on.

So how do we pick good interviewers? While specifics will vary based on the role and seniority of the person being interviewed, there are a few archetypes to consider to ensure a well-balanced perspective.

Peer: Direct teammates experienced in the role. For junior positions, include someone promoted internally to demonstrate growth potential. The peer will give the candidate good visibility into the day-to-day realities of the team and role, while also being in a good position to assess if they would be able to hack it.
Partner: The partner should be a team they work with to deliver for customers. For a sales person, this could be a colleague in sales enablement or marketing. For an engineer, it may be a product person. The partner will be able to give the candidate insight into who outside the candidate's immediate team they will be working with, while also being able to assess if the candidate would be of help. A good partner should have experience working with multiple people in a similar capacity and can make a good judgment call based on what they hear if this person would be able to function similarly or better.
Stakeholder: Similar to a partner, stakeholders are internal customers of the person doing the job. Stakeholders are likely more senior or could be the person that the interviewee would have a "dotted line" to. For example, some organizations may have sales enablement functions give a dotted line to a learning & development leader. Another example could be having a regional marketing manager interview with the general manager of the region that they'll report to in their day-to-day, even if their manager is the head of marketing at HQ. Stakeholders should be at the same seniority or higher seniority than the manager.
Skip level manager: This is the manager of the hiring manager. They are great to have as a backup and when working with a new hiring manager or less tenured group overall. The skip level manager should provide a higher level perspective on the hire and help the team avoid mistakes. Skip levels can also make the candidate feel important, knowing a more senior person is invested in hiring the right person for the role. I wouldn't always count on having a skip level, but they should be abreast of the process and ready to step in if necessary.

It's important to, if possible, have more than one person in each role on deck to interview. This is important in periods of high volume hiring where it could be burdensome or difficult to schedule the same group over and over. Having a bench that can fill different interview roles will be critical to keeping things moving quickly. Everyone who could be brought in to interview candidates for the position should have access to the job description and any notes needed to prepare.

I recommend having four to six interview slots. This could mean having more individuals involved as shadows or pairs, which could be beneficial for certain areas like coding exercises or sales role plays. You should flex up or down pending the seniority of the role. Remember, there are diminishing returns for more interviews, and junior employees likely won't have that much to actually talk about compared to more senior hires. However, senior hires likely have more options and are more discerning: a convoluted or lengthy process will turn them away.

Assigning interviewers to values and functional areas

We now know what we want to interview for and who will do the interview. The next part is figuring out who is doing what. This is like taking a group of multi-talented musicians and figuring out which instruments they will play for the next number. There are cases where each interviewer could cover every value or skill, but they likely have a specialty or are well-calibrated to judge a specific area best.

Recruiters and hiring managers should partner here to figure out who will be best suited to assess different areas based on the unique perspectives and skillsets of the interview team.

If you're struggling here, it helps to visualize and map it out. Get a white board and use sticky notes with the name of the value or skill and move them around to different names. You should see if you're able to get a mix that feels right.

The hard work is already done: assigning the right values and skills for interviewers to assess is like garnish on the dish. With the right interview team, you should be able to have anyone on the team comfortably assess different values and skills. You should aim to optimize towards the values and skills the interview team is strongest in, or at least most in tune with evaluating.

At minimum, you want to ensure you don't ask non-technicians to do coding assessments!

Preparing team to interview

Strong interviewers require training and preparation. It's a skill that can be cultivated, and rarely will an untrained interviewer be primed to get the right data needed to make a decision. At minimum, start with compliance and legal requirements. If you can do one thing, make sure the interviewers know to steer clear of questions or conversation topics that could lead to bias or discrimination.

I cannot emphasize this enough: interviewers need to be prepared to interview. It boggles my mind how often interviewers are thrown into a conversation without adequate training or preparation to make the most of their time with the candidate. Worse, they don't know who they're interviewing or what they're interviewing for. How can we make a good decision if we don't know this? But it does happen.

My view is that anyone can be a good interviewer, but like any other skill, it takes time and practice to cultivate the skills and interviewer instincts to juggle engaging conversation, gathering feedback, and preparing follow-up questions to probe effectively.

They should know what they are being asked to assess and have guidance on how to conduct interviews effectively. These can be learned through training courses and reinforced with supplemental reference material.

Once the interviewer knows the theory: they need to practice before we let them interview on their own. Have new interviewers shadow experienced ones or pair up with a more tenured interviewer. They should practice running things end to end with a more experienced buddy to help them calibrate.

How many times they need to shadow or pair up really depends on the person and how quick they take to the interview. In general, I advise at least letting people shadow once or twice as a silent observer, then lead with a tenured interviewer backing them up once or twice more to ensure they can apply what they learned.

When a search is kicking off, I recommend briefing all of the interviewers on the job role. This should be a summary of what's discussed with the hiring manager when scoping the role. It should provide an overview of what the team is looking for in a hire and what each person will cover. It's an opportunity for interviewers to ask questions and calibrate towards what the ideal candidate looks like.

This can be done as a meeting or as a note. Emails are fine, but ideally these "interviewer prep notes" exist in the applicant tracking system for easy reference and editing. I only recommend a meeting if it is a new role or function the team has limited experience in. Otherwise, save time and go for an asynchronous approach.

Making the hiring decision

After all of the interviews are conducted, the team comes together to make the hiring decision.

The interview debrief is where insights become decisions. Structure these conversations to focus on evidence, not feelings.

A good debrief has a facilitator that leads the conversation. At Amazon, there is an internal role called the "bar raiser" the facilitates the debrief and defends against bad hiring decisions. This article explains Amazon's philosophy and need for bar raiser.

To summarize: recruiters and hiring managers have incentives that benefit them personally, but may be bad for the company long-term. For example, if a recruiter is incentivized to fill roles quickly (whether by performance incentives or just the goal of "getting this job off my plate") and a hiring manager wants a role filled because they're overburdened by tasks from team turnover, the quality of decision making will diminish. The search for a "quick win" and "getting a warm body in seat" outweigh the long-term effects of making a poor hiring decision. It's like being hungry and reaching for a bag of chips versus a salad. It may solve the immediate problem, but it's not the best long-term choice for your health.

I've not heard of any other company besides Amazon ever doing something like this. Having served as a bar raiser, I am a believer in this approach and think it would benefit companies to have a more "neutral" third party involved in each hiring decision to mitigate risks from those involved feeling short term pain.

However, I recognize Amazon is unique: it's a huge company with diverse business units. Teams are so large that it is easy to interview every week and never work with the same hiring manager twice as a bar raiser. This makes it a lot easier to be objective when you don't have a pre-existing relationship with the hiring team and are only connected through a shared understanding of company values and are specifically tapped to ensure that candidates joining the company are of a certain quality.

The majority of companies are smaller. This makes it hard to find someone who can serve in this role that won't be at least mildly impacted by the presence of the hire. When there is familiarity, there is an increase in bias. Maybe I am inclined to help them hire someone quickly versus advocating to wait for the best person to come through to give the partner team a quick win and boost my own standing with them. These types of scenarios create conflicts of interest, again pitting employees and managers versus the broader goals of the company. At startups, founders could play this role, but as teams scale, it becomes difficult for them to be involved in every hiring decision: especially during periods of rapid growth.

So who should facilitate the debrief? There are pros and cons here, but I learn towards the skip level manager or a recruiting leader (not necessarily the recruiter who is responsible for the open job), with the hiring manager getting the most decision making power assuming skip level manager or recruiting do not veto the candidate.

The skip level manager is also good pick here, even if they aren't involved in the interview, because the hire affects their organization. They'll also know the skill level and experiences of the interview team and hiring manager, which helps them gauge the commentary and validate their judgment. Recruiting, familiar with the full pipeline and insight into all interviews, should also be equipped to facilitate the debrief and probe. However, recruiting isn't often incentivized by quality of hire, rather just that the hire is made. So for recruiting to be best for the company here, incentives need to be aligned for long-term decision making versus quick wins.

For smaller startups and companies, having owners, CEOs, or founders play this role can make a lot of sense, too. There needs to be someone with enough perspective that has enough space between themselves and the pain of the open role to think from the perspective of the company long-term. A sales leader will likely feel more pressure to hire a salesperson than the CEO would.

With a facilitator selected, let's get into what goes into the decision making process.

Before meeting, everyone should read the feedback and notes. If the team is well-trained, notes should be clear and concise, with overall ratings and summaries pre-written. The goal here is not to get the "right" vote, but for each team member to share their impressions of the candidate and an initial pass at "yes or no" based only on what they saw. It's not uncommon for teammates to change their mind in the debrief stage after seeing all of the feedback. This is intentional! We want to go in with a point of view, but be open to feedback and change in the face of new data.

Think of it this way -- would you stick by your rating after one interview when you now have five interviews of data to refer to? I'd be more inclined to make a decision based on all the data rather than the slice I gathered. It's good to encourage the team to be open to changing their perspective after seeing all of the data and be honest about what they may have missed that informed their perspective. This requires the psychological safety to openly be wrong and change your mind (something that can be super hard in most environments), but it also requires good training on how to approach interviews effectively.

Assuming everyone reads the feedback ahead of the debrief, we can jump into discussion.

The debrief should kick off with a reminder of the role and who the team is meeting to evaluate. The facilitator should provide a summary of trends and lead the round robin discussion. Typically, this means giving each person in the meeting a chance to speak to their experience with the candidate and share their thoughts with the group. A good facilitator will ask probing questions on their feedback to provide the group with greater clarity and assess how the candidate feels now

I often observe that interviewers will open up by rehashing the entire interview and restate their notes. This is usually a poor use of time. I only ask for a general overview if the notes are unhelpful on their own, and I need additional color because the notes are not well-written. If notes are clear, you don't need to rehash live: you can probe into specific parts of the feedback that need to be discussed.

I recommend doing a round robin style, usually starting with the most junior person first and working my way up to the most senior person. You're more likely to get honest feedback from junior employees if they can speak before a senior person goes, because it is more difficult for junior employees to dissent.

When doing the round robin, it's important for the interviewer to speak to the following

Concrete examples that showcase the candidate's skill and fit
Strengths and areas of concern for the candidate. What is the reason to hire? Why would you not hire this person?
Areas that seem coachable in role versus gaps that seem insurmountable.

As the group goes around, open up the floor to others to provide commentary to see if these areas can be addressed. It's important to let everyone in the room share their insight and add color to their notes.

After going around the room, I typically like to close off with the hiring manager to share their thoughts and ask the group any pointed questions if needed. If done well, the hiring manager can make a decision easily at this stage of the conversation.

If advancing the candidate, document:

Key strengths of the candidate to guide where they may want to get started immediately.
Growth areas to address; coaching opportunities or onboarding buddies that will help them close these gaps.
Specific support needed in the first 90 days.
Recommended resources or training.

If declining, capture specific reasons tied to job requirements. This helps defend the decision and provides valuable feedback for the recruiting team.

Remember rejections may not be a no forever -- the candidate may be who you're looking for down the road. Keep in touch with candidates and make sure you don't ghost them after the interview ends. It's rare for companies to think long-term when it comes to hiring, and you can save recruitment costs down the road if you build a network of talent.

The hiring manager owns the final decision, but should clearly articulate their reasoning to the team. This builds trust in the process and helps interviewers calibrate their evaluations for future candidates. Without discussion and review, it can be easy to over-index into certain areas that may be more coachable than interviewers think.

Test and learn

This, like everything we talk about, is not a perfect process. Different groups of people will interview the same candidate with the same questions and get different outcomes. That is to be expected. What we can do is ensure we are thorough in our evaluation and provide a positive experience for the candidate.

Remember, the interview is an opportunity for the public to engage with your brand and organization.

If we believe in the process and invest in doing it well, we will increase the odds of being able to effectively forecast a candidate's success in role and sell them on the opportunity.

Don't be afraid to tweak the process and move things around based on what you learn. It's good to experiment and give people a chance to try different areas. But this is a good starting point for you to consider as you begin interviewing candidates.

How would you design a final round interview? Let me know: coniferblog@proton.me