Technical interviews are meant to be proxies for the actual job responsibilities of a developer. However, it seems like every day on Reddit, there are anonymous developers who criticize the technical interview. I'm publicly in the "most technical interviews are terrible camp," and I have taken my fair share of ones that made me scratch my head trying to figure out how it related to the job or what information the company and hiring committee was hoping to learn.
Recently, I was tasked with creating a new technical interview for a vacancy on my team. I desperately wanted to come up with something that improves upon some of my dislikes from technical interviews I have had in the past.
Criticism of technical interviews
The criticism of technical interviews is widely varied. LeetCode gets criticized because it is irrelevant for the job. Take home tests get criticized because they are more time intensive and favor candidates with more free time, potentially limiting opportunities for people who have jobs or family responsibilities. Pair programming gets criticized because pairing with an unfamiliar person is a high-pressure scenario, and the pressure affects everyone differently. There is also the criticism that technical interviews in general discredit experience.
What is the actual job
First off, the interview should be appropriate for the position. If you have a particular way of working, then it makes sense to have your interview model that. For example, in the 6 years I have worked in my current role, I have never pair programmed. Not even once. That means that I should not have someone pair program with me in the interview.
There should also be different levels of difficulty based on the role. This is more difficult to do if you have a general job opening for developers, and you decide on the leveling based on how someone does. My employer has a really simple career ladder for engineers with only 3 levels (developer, senior developer, and principal), and we list the job for the specific level we have open.
Designing a relevant interview
The specific interview I designed was for a senior backend web developer, however, the approach I took would work for any level or specialty.
At the senior level, I analyzed the work that our other backend developers did. The things they were asked to do the most often were to 1) troubleshoot issues with existing features and services and 2) design or evolve systems.
The interview consists of an evolutionary system design question and a troubleshooting question. Each question is timeboxed to 30 minutes. The key for relevance is to simply take a previously solved issue and re-frame it as an interview question.
System design
The evolutionary system design question was a real issue that we faced and solved. I took a prior RFC that I authored and turned it into the interview question, complete with all the context and background information from our RFC template. It is representative of the type of problems our team is regularly asked to solve and is 100% relevant to the position. I let the candidate know that the problem is from a previously solved issue, so they know we aren't trying to get free work out of them.
Troubleshooting
Similar to the system design question, the troubleshooting question came from a real issue that the team tackled previously. In this case, the source of troubleshooting questions are previous Jira tickets that were assigned to the level that the candidate is interviewing for. The candidate is given sufficient background information on the environment and assumptions that are needed to get started with troubleshooting.
I mentioned that I have never paired in my current position, but my team and I do regularly talk to discuss any tricky troubleshooting issues over Slack. For this part of the interview, the interviewer is there to interact with the candidate as if it were a real team situation.
Administering the interview
The interview process is meant to scale, so along with the questions, there are instructions that are given to the interviewer for each discipline on how to conduct the interview. Anyone should be able to conduct the interview for the level equal to or below them on the ladder.
A series of rubrics were created so each candidate is assessed with the same criteria. Categorizations for excellent, satisfactory, and needs improvement are defined in order to make the interview process more consistent, and these were based on rubrics I created during my master's thesis back in 2014.
The rubric that I created for my master's thesis is below. Keep in mind that this was for assessing soft skills for a specific task, so this isn't the one that is used for the technical interview, but it does demonstrate how skills might fit into specific categorizations:
| Needs Improvement | Satisfactory | Excellent | |
|---|---|---|---|
| Organization | The facilitator was unprepared to lead the group’s activities and did not improve throughout the week. | The facilitator was prepared to lead but took time to get up to speed on the project tasks. | The facilitator was prepared to lead the group’s activities as evidenced by contributions on the first day. |
| Prioritization | The facilitator did not provide a clear prioritization of the tasks for the week. | The facilitator prioritized tasks, but a few bottlenecks were present for concurrent tasks. | The facilitator prioritized appropriately and eliminated bottlenecks for concurrent tasks. |
| Communication | Communication was largely unclear, unprofessional, or subjective. | Portions of the communication were unclear or there were instances of subjectivity. | Communication was clear, professional, and objective. |
The rubric also is helpful in the event that there are multiple people on an interview panel, so we can compare notes on the candidate post-interview.
Conclusion
In my opinion, the technical interview is often the most stressful part other than the negotiations at the time of the offer. There is room for improvement in the process, but by making it relevant to the position, over the standard LeetCode grind or multiple hours-long take home assignment. Since the interview questions are based on things we've actually faced before, there is no shortage of material to draw from if we need to change things up or as needs evolve.