Hiring and Technical Interviews
by Mike Mangino on August 16, 2018

Hiring is one of the hardest problems in software development. When we hire, we’re trying to determine if a person has not only the technical skills necessary to work at HubTran, but also the interpersonal and communication skills. We need to know if they share our values and are capable of working remotely. At the same time, we need to help them decide if HubTran is the kind of company they want to work for.

As if this challenge isn’t hard enough, we’re trying to do it with relatively limited resources. Life would be much easier if we could just let every candidate do the job for a few months to see how they perform. That’s obviously not practical though. At HubTran, we want to evaluate how well we believe a candidate will do as quickly as possible. In the past, our interview has been composed of three steps:

The initial phone screen

In the initial phone screen, I have several goals. First, I want to make get a feel for what motivates this person. Are they in it for the technology, or does this person build software to solve a problem. How curious are they and how do they show that curiosity? Are they a team player and how have they demonstrated this? At the same time, there are some things I explicitly don’t do. I won’t ask them to do any whiteboard coding1 or quiz them on their algorithmic knowledge. I don’t believe either of those skills is predictive of how well they will do on the job. Instead, I focus the skills that I’ve found are related to being a good developer.

An often overlooked part of this interview is to help sell HubTran to our candidates. I want to be as up front as possible about what it’s like to work at HubTran. We work hard to have a great working environment. Even so, startup life isn’t for everybody. We move quickly and often need to make decisions with limited information. My goal in this interview is to help people decide if HubTran is the type of company they want to work for.

If this conversation goes well, the next step is coding interview.

The coding interview

Because we’re a remote company, we do our coding interviews using slack or another screensharing tool. Our interview consists of Michael working with the candidate on a simple challenge. We’ve been using the same coding challenge for all candidates for the past 4 years or so. We’ve picked something that’s straightforward and not overly algorithmic because we believe that represents the type of work we do. We allow you to do the work in whatever language you choose, and there’s no penalty for looking things up2. After performing this interview on more than 50 candidates and developers that we know, we find that it is highly correlated with the type of developer we want to work with.

If you make it through the coding interview, the last step is another conversation with me.

The final interview

The final interview serves two main purposes. First, it’s also an opportunity to reflect on the coding interview. Coding interviews don’t always exactly how you want. By talking afterwards, we have the opportunity to see what you have learned from the experience. Additionally, the final interview gives me an opportunity to sell you on HubTran. If you’ve made it this far, you’re very likely going to get an offer letter. I want to make sure that offer is as strong as possible.

What we’re changing

In general, we’re happy with this structure. The candidates that make it all the way through the process have been very good. Our main problem is that what we’re doing now is wasteful. Particularly, the number of people that completely fail the technical interview is too high.

We’ve considered several changes that might help this. For example, we could tighten up our criteria on either an initial interview or for what we look for during the phone interview. After testing, this didn’t work. For several weeks of interviews, I made a prediction of how each candidate would do on the coding interview based on their resume and the initial interview. In the end, we found very little correlation between my estimation and performance on the coding interview.

That terrifies me for two reasons. First, I can’t reduce the number of people who fail on the technical interview without reducing the number of people who succeed. Second, how many people that I turn down based on resume alone would be excellent?

With this problem in mind, we’ve decided to add a smaller coding challenge at the beginning of our interview process. This isn’t a decision we made lightly. We recognize that by adding another step to our interview, we’re asking for more time from each of our candidates. We also recognize that there is more to a person than just their ability to churn out code.

At the same time, coding ability is a necessary but not sufficient part of working at HubTran. Overall, we believe that adding this challenge will reduce the time it takes for the average candidate to get a decision. We also believe that we will be able to interview more candidates who may be great developers but don’t yet have a resume that stands out in the crowd.

The challenge itself

In creating an initial coding challenge, we tried to balance several objectives. We want to make sure how you do on the challenge is correlated with how you will do at HubTran. We also want to make sure the challenge is fair. Finally, we want to be respectful of your time.

To that end, we’ve created a challenge that is a simplified version of an actual feature at HubTran. We’ve added a time limit of one hour to make sure we’re seeing work that’s representative of what you would do on the job, and not what you do with hours of polishing. We know that not everybody uses Ruby as their primary language, so we allow you to do the challenge in whatever language you like. We also allow you to use any resources that are available to you.

The reactions so far

So far, the reaction has been mixed. We’ve had several people complete the initial challenge and do well. Several people have started the challenge and not completed. We also got this feedback:

Mike,

Coding challenges are lame. Best of luck with your search.

Will this work? It’s too early to know for sure, but I certainly hope so. Are you interested in working for HubTran? Let us know! Send a resume and letter of introduction to me at mmangino@hubtran.com

  1. An applicant sent an article in response to us sending them a coding challenge. I completely agree with the article. That’s why we don’t do whiteboard coding. ↩

  2. I learned to program before search engines existed. I remember the days of searching books for answers. There’s no reason we should force people to go back to that. ↩