Designing a hiring process for developers
Judge developers without putting them in a narrow, very specific box
In this article we will explore the current state of how many companies hire developers, what’s a good representation of a developer’s skills and we will move on to discussing generalists — developers that don’t necessarily fall under a single tech stack.
At the end, I’ll lay out a few ideas about improving hiring processes to more closely reflect what constitutes a good developer. These tips will highlight the misalignment between what skills we look for when we hire, versus the skills that we really want from new team members. I’ll make a case for being more liberal towards missing particular skills, and being more rigorous about taking a holistic approach and looking for deep understanding of technology and potential.
A modern hiring process
Based on what I’ve seen in the last (at least) 5 years, many companies align on what a good hiring process for developers looks like. You have your typical initial screenings, hands-on development part, technical discussion part and one or few soft skill interviews (cultural fit, meet the team, meet the managers etc.). All these combined tend to give a good idea if a candidate is good and if they fit in a particular team. And I’d go as far as saying that I wouldn’t really skip any of these. For me, the devil is in the details.
Let’s look into what each of these steps is there to check for.
Initial screening
It’s not the nicest thing to say, but the reality is that this step is a lot about checking for initial fit before investing too much time on the candidate. It’s important to establish personal contact with them and look at inter-personal traits. Sometimes there’s a language or communication gap that’s too big, for instance. Plus, standard things such as different salary expectations or misalignment between what the different parties understand the position would be can be reasons to part ways. It’s also a very good opportunity to present the company in a good way.
A nice thing some companies are doing is asking a few easy technical questions right there in order to see if the candidate is even close to having good technical know-how. There are many questions that even a non-technical interviewer can ask that are very effective. For instance, we used to ask backend developers what the different HTTP methods are (GET, POST etc.) and if it was possible to use a GET to modify data. You’d be surprised how many candidates fail to answer this even remotely correctly. From the candidate perspective, it also gives a good impression that the company has high standards and has technological expertise (at least that’s my impression).
Technical interview (practical part)
Another often used interview type is the practical technical interview. There are a few popular options I’ve seen:
Take-home assignment
Live coding session
Live code review
System design interview
The point is to see candidates in action. Whereas I definitely agree there should be a practical part of the process (since many people very good at talking, but fail to perform actual work well), I have particular opinions on many of these options. I will explain later on in the article :)
Technical interview (theory)
This is often times either an informal conversation about tech, or more of a Q&A interview. An interviewer would either have a set of questions to ask, or alternatively, they could just ask around about the candidate’s previous experience and follow up on that to understand their background and expertise better. If this interview comes after a practical coding challenge, part of this technical interview could be discussing the solution.
Tip: From my experience, the more informal such an interview is, the better the experience for the candidate. In fact, being asked question after question feels artificial, formal and impresonal. A casual conversation about tech, on the other hand, can be even enjoyable — it’s like talking about coding with friends, or someone at a conference. However, being an interviewer in such an interview is not easy. You need to juggle between being a good conversationalist, and steering the discussion into topics you want to cover.
Soft skills interviews
Different companies are doing different things, but the overarching theme is that there are one or several interviews where the candidate is evaluated in skills, not necessarily connected to their day-to-day job. Most of the time it’s either a “cultural fit interview” or a “meet the team/founders”. I’m also seeing a rise in the popularity of meeting people from adjacent teams. For instance, some companies ask software developers to talk to product people in an effort to understand if they play well with them and if they understand the product mindset. It’s also very common to ask questions about conflict resolution, feedback culture and willingness to collaborate and grow.
A closer look into technical interviews
The technical interviews are probably the hardest part of the process — both for the candidates, and for the hiring team. That’s why I think it’s worth re-thinking. I’ve seen, in recent years, a lot of effort being put in from hiring teams to create a very inclusive, understanding and un-bias general process for candidates. Technical interviews, on the other hand, even though they have changed, are still not on-par with the general process. Often times candidates are put in very artificial, unrepresentative and stressful situations and expected to perform. And not only that, they are being compared to their interviewers’ standards of people that have been through that same process multiple times.
Let’s look at the typical options and talk about their usual trade-offs.
Take-home assignment
The good old take-home assignment is a staple in tech hiring. You give candidates an assignment — a toy application or a feature to do on their own time and send it back within some pre-discussed timeframe. Usually it’s a task that should be normally completed within a few hours. It’s more work for candidates compared to other interviews, and naturally some people don’t have a lot of free time in order to complete it very fast. But I still believe it’s a good option. And I’ve never encountered a candidate that couldn’t find a bit of time within 1–2 weeks in order to complete a code challenge. I know people that refuse to do code challenges, but usually it’s not because they are great candidates (at least that’s my experience).
The strengths of this tech challenge are that it allows interviewers to see the candidate in action, while also giving them enough peace and not disadvantage people who find it difficult to focus with other people around. One disadvantage is that you need to make sure candidates do their own solution and didn’t just copy it from somewhere or ask someone else to do it. I haven’t experienced this, but I’d still discuss their solution after the fact and ask a few questions to see if they wrote it themselves.
It’s also really important to come up with a good challenge. I know many companies choose something close to their domain. It should still be obviously a “toy” project — you don’t want to leave an impression that you’re giving candidates your work and expecting them to do it for free. It’s also important to have a generic task that doesn’t assume they are familiar with a particular tech stack of libraries that are not part of the job description. For instance, I think it’s safe to assume that a mobile developer can create some standard UI and consume a RESTful API, but if you ask them to write some really deep graphics code or animations, that might be just something they never encountered before. That would be an unreasonable challenge unless you strictly need someone with a graphics background.
Live coding session
I’ve recently seen more and more companies switch to doing live coding sessions. This is an interview where candidates sit down with an interviewer, either in person or over video, and they are given a task to complete, pair-programming style, together with the interviewer(s). Typically over the course of an hour, they are expected to write actual code, complete a feature or fix a bug. What’s nice about this is that it takes less time for the candidates compared to a typical take-home assignment, it’s way harder to cheat at and it gives the interviewer the opportunity to see how the candidate thinks and approaches a challenge.
However, it puts many candidates at a disadvantage. The problem is that many people cannot think very well while someone is looking over their shoulder. Add to this the intrinsic pressure of being judged and awkwardness of job interviews, and you get people that usually program for a living struggle to write a for loop. I know working under pressure is a beneficial trait, and if you’re interviewing for a news anchor position, by all mean, test for that. But for engineers that a notorious for being loners and working best when given space, I think it’s unfair.
Live code review
Live code reviews are something I used to enjoy a lot in the past. It puts candidates in a standard code review situation, where they are presented a pull request for a feature on a toy project and they are asked to review it just as they would at their job. Here, the devil is in the details, because the interview is only a good as the contents of the PR. Ideally, you want a good selection of anti-patterns and problems in the code that the candidate has the opportunity to notice and propose a solution for. I’d recommend a good mixture of easy and hard problems, with the hard ones leading to good discussion about different approaches and trade-offs. Here’s a list of things I’d add:
[easy] TODOs and FIXMEs — you’d be surprised how many people have learned to ignore these in the code
[easy] Code styling problems (just remember there are different styles)
[intermediate] Coding anti-patterns (e.g. using println instead of logger)
[intermediate] Obvious bugs in the code/logic
[advanced] Inefficiencies in the algorithm
[advanced] Architectural problems
[advanced] Concurrency and synchronisation problems
[advanced] Hard to notice bugs
Designing such a PR is more of an art than science. Especially difficult is to have a good set of issues that most good candidates notice and bad onces don’t. It is a bit of a process to tweak the PR based on feedback. Additionally, I’d advise to use code that at least most developers are familiar with and can relate to (e.g. almost all backend developers should be familiar with creating RESTful APIs with their respective framework, but they might not know too much about GraphQL).
As I said, I was a big fan of these interviews. Until I saw their weakness. I’ve seen cases where a candidate does exceptionally on the code review, only to start the job and show very poor development skills. I guess some people are just good at looking at code, but not so much writing it. That’s why I’d want to see candidates write some code.
System design interview
This one is probably more prominent for backend development and leadership positions. In any case, I’m seeing it more and more. It’s an open-ended discussion about architecting a system and making choices along the way. It could be as easy as saying “If you had to design a movie streaming system, how would you do it?” at the start of the interview and seeing where the candidate takes you. I personally like it as it gives you a sense of how a candidate thinks and if they have a good understanding how different parts of a system work together. It shows more general curiosity and deeper understanding of techology. But obviously, it’s also not suitable for some positions — for instance, you wouldn’t ask a junior developer how they’d build the whole company (or at least you wouldn’t expect a perfect answer). Or a mobile developer might not really have experience with designing databases and scaling traffic (although you could ask them how they’d architect a mobile app that has special requirements). And just like the live code review, I’d personally not like to have this one as the sole technical interview. The fact that you know how to design a system, doesn’t mean you can implement that design well (but it’s a very good start).
Designing a good process for YOUR company
All these interviews outlined above are viable options. Where companies fall short in their hiring is by not designing a process that fits the profile they are looking for. In fact, I’d argue that most companies decide what interviews to have based on their personal preferences, without considering what kind of person they want to hire (or more like what personality and professional traits they want that person to have). Let me give you an example…
As developers, we often talk about what makes a good developer. And obviously, there are different opinions there. Often times we describe a good developer as someone who has deep understanding of how things work together, having experience and a good intuition what works and what doesn’t. Most people would agree that good engineers are language and tool agnostic — they are not too tied to a particular stack. And they have a lot of transferable knowledge that they can apply in a whole plethora of settings.
At the same time, we design code challenges that expect high level of proficiency in a particular language or toolset. Or we expect candidates to, on the spot, come up with a O(n) algorithm for a random, fictitious problem.
What’s the problem with that? In the former, the fact that our team is very proficient in, for instance, Javascript, doesn’t mean each candidate is. And languages can be learned. Quite fast usually, if a developer is good. In the latter example, even though algorithms are important and writing them shows experience and intelligence, we have to ask ourselves — does this job involve algorithms? Or is it a more standard engineering job about building and consuming APIs? Are we interviewing for skills we don’t actually require?
Why do we design such processes?
It’s difficult to judge people’s skills. Especially if a job involves a wide array of topics. And even more so, if it’s as creative and ever-changing as software. So it’s only natural to either go with the majority and do whatever <insert big company name> is doing, or looking at our own experiences and trying to mirror them. And what’s the problem with that?
Other companies have other requirements and priorities
If we give take Microsoft’s hiring process, or we copy the hiring process that our last employer had, we might be hiring the perfect candidate… for someone else. The other company might have had a very specific idea what the new hire would do, and we might need a generalist that can find their place in the team alone. But we would be building a hiring process that would reject such a candidate as not good enough.
We are experts in our own tech stack
The most common reason I see is being too biased towards our own stack. And I speak from experience. In our own teams we use particular tools and we are experts in a particular domain. And we have done that for years. So when we design our hiring process, we design it to look for people that have done the exact same thing and for as long as we have. We ask questions deep into our area of expertise and we expect perfect answers. Because these questions seem easy to us. But they are not… not to everyone. This happens, at least to me. But we need to resist that temptation and realise that it’s a tall order to expect people to know as much as us about a project that WE have worked on for years. And it’s ok for new people to learn as they go.
Always hire people that are better than you?
This is something we hear often. But maybe some of us take it too literally? It’s one thing to have high expectation forcandidates, but at what point are these expectations unfair? Can we expect a candidate to really be better than us at our own game?
What are the key traits we want candidates to have?
This is where the process begins.
There are stronger and stronger voices nowadays that say they hire for ambition, for agency, for urgency, for potential, for mindset and for cultural fit. Notice how these are all softer skills, skills that are difficult to quantify or even measure. These voices say — hire for attitude, skills can be learned.
And yet, too often we judge a tech candidate’s abilities based on how they sort arrays, or what language features they use, or knowing which Java version introduced optionals.
This is because we don’t ask what traits we want candidates to have. We rather ask ourselves — what are technical questions that:
I can answer, and
Are connected to our stack, and
Show deep technical knowledge, and
Many people get wrong, and
…
It’s very understandable that this is a straight-forward and not necessarily bad way of doing things. But we can do better.
So what are the key traits we want?
I’d start off by asking — so we need a generalist or a tight specialist? If we need an expect in a very narrow area, then it DOES make sense to get really deep and really technical in that area. But I also think most positions are not as narrow as we think. Do we need someone more flexible that can tackle a whole range of problems we throw at them? That’s more of a generalists. And generalists tend to fare less well with many traditional interviews. I know that, because I am one of these generalists. One of these generalists that get judged by being a little rusty writing Java code, the ones that don’t know the best way to use your data modelling library. A generalist will never know more than you about the web framework you have been using in the past 10 years. But they might give you a more holistic perspective on the project as a whole. And if you give them a tiny bit of time, they will learn your stack. Because they learn fast.
We also have to ask ourselves — do we need someone deep in the code, or do we need more of an architect that can also think about how components fit together. For a work bee coder, you can also dig deeper into languages and frameworks, but maybe don’t judge them if they don’t know how to choose the perfect database vendor. And vice-versa, an architect might be a bit slow to write vast amounts of working code. But they also know how to create a healthy system.
For creating a well working hiring process, it is critical to test for the traits that we want, not only the tools that we use. And especially not for skills that just came to mind when we were thinking about technical interviews.
What needs to change
We need to be more mindful about what skills are critical for our positions, but are also not easily developed “at runtime”.
You can always teach a developer to use a new library. But it’s not easy to teach them how computers work… or how to be a team player.
So why do we evaluate them by their ability to create a RESTful API with Node.js, with validation and persistency, within 60 minutes and while explaining their solution to 2 other people? This is what needs to change.
How can we change it?
Firstly, we need to accept that a single test will never be representative to a persons expertise as a whole. It’s all too easy to randomly hit either their blind spot, or something that they happen to be experts at.
It’s important to touch a lot of surface area. Probe candidates in a plethora of areas and see where they excel. Not only will you give them a chance to shine in at least one topic, but you are also getting a broader picture of their skillset. Instead of testing a single skill in a vacuum.
We shouldn’t hire a developer just based on their system design skills, or their skills to review code. This job is way more than that.
It’s also important to be more forgiving. It’s all too easy to probe for knowledge we know very well, and be disappointed when a candidate is missing it. Ask yourself if that piece of information is really so important? Does it foreshadow a bigger issue? Is it something that can be easily fixed or learned? Especially when probing in many areas, it’s inevitable that a candidate will not be great in a few. Don’t let a great candidate slip away just because of a minor error.
Tip: Forgiving is not the same as ignoring. If something doesn’t add up, ask. if something is unclear, clear it. Because sometimes behind unclarity hide bigger problems. And since we like to fill in the gaps with things we want to have, we might be in for a nasty surprise.
If we speak concretely about tech interviews, I’d do the following:
As I wrote above, for me it’s imperative to see both practical (coding) and theoretical expertise.
For the theoretical part, I’d choose either a system design interview, or a technical discussion about select topics for the particular job and based on the candidate’s background. Ideally, both can be combined.
For the practical part, I think a live coding session is too discriminative. I wouldn’t have it as a sole option. Some companies give an option between a live coding session and a take-home assignment. I like that. I’m also curious to see if a live code review session, combined with a short(er) take-home task would be a good combination. Unfortunately, I haven’t tried such a thing so far.
What would convince me to hire someone?
Acceptance criteria are important. So why not have them for hiring!? Why not spend some time after designing our hiring process thinking about what we want to see from candidates in order to hire them. That might also avoid some bias in the interviewing process (although these criteria are likely vague enough to not avoid all biases). And let’s not just say candidates need to check all the boxes or they get rejected.
Here’s a very opinionated and abstract list that I personally want to see in candidates:
A candidate needs to show ownership and proactiveness
A candidate needs to be confident, but ideally humble
A candidate needs to show deep understanding of the fundamentals of their job
A candidate should like to learn more and grow professionally and personally
A candidate needs to be a team player and works collaboratively
A candidate shows a lot of experience within the domains they have worked in
Having compiled such a list is a good first step towards designing a hiring process. Having these desired outcomes makes it easier to design “tests” probing for these traits.
The list I provided is naturally very vague and not enough for a tech role. So even if you agree with it, you’d have to supplement it more tangible (and likely more technical) qualities for your particular role.
Takeaways
I’d like to finish this article with the key points I want you to take away with you on your journey to hire your new colleagues.
New hires need to both be able to do the actual work, and also ideally come already with deep understanding of technology. Test for both.
We often talk about diversity and inclusion at the workplace, but we also often don’t allow diversity when evaluating tech skills. And we are not inclusive with the interviews we do
Don’t evaluate skills in a vacuum. Probe for a plethora of skills, and discover where a candidate shines and where they need to improve
Put more emphasis on potential, and less on concrete (and easy to acquire on the job) skills
Be more forgiving — no one is perfect and skills can be learned