new site();
Does this sound familiar?
Round 1: A phone screen with recruiter or HR, featuring some syntax questions and resume verification.
Round 2: A phone screen with team lead or senior dev, featuring mostly syntax questions, one or two conceptual questions and some gotcha questions. (Maybe this is done as a video chat with some programming tasks.)
Round 3: Several one-on-one interviews onsite with peers and managers to assess fitness as a candidate, including a coding challenge and an OO class or database design question.
Round 4: Finally, a formal or informal get-together with the people who interviewed the candidate to get everyone's feedback.
At the end of these steps, or your version of these steps, you have gathered facts about whether or not the candidate:
Additionally, astute observers may have some sense of whether or not the candidate:
Even putting the two lists together, this does not even vaguely resemble the list of qualifications we typically post in a job listing. Worse, this process relies on the intuition of the people interviewing the candidate more than on any facts found during the process, and consequently reflects the biases of those interviewers. At the end we often have several "equally qualified" candidates, even though we know this is basically impossible. Immediately likable candidates are given unspoken boosts for their shortcomings, and candidates who take longer to warm up to but are otherwise skilled are dismissed as "lacking passion" or "presenting a fit problem." It's squishy.
The good news, though, is that it's not all wrong. There's something to be said for the process; for one, it only involves high-cost resources later in the process, so that your expensive managers and dev leads are not stuck weeding out candidates with basic deficiencies. And, especially respecting the second list, you do learn some valuable information. Plus, everyone is doing it this way, so there's no risk of looking like you don't know the right way to assess talent.
The bad news is that you haven't actually measured the person's practical technical ability, especially in the context of team development, and have learned little if anything about how the candidate works under real-world circumstances.
Because offering criticism without suggestion is basically just complaining, here are some things that might help get those last bits of information.
1.) Trial by pairing
Pair your candidate with one of your devs and give them both a challenge to solve together. Because acquiring domain knowledge is a waste of interview time, make the challenge one that is easily understood but not easily solved; we have used CodeEval in the past to supply us with challenge material, but Hackerrank or any of the others should do nicely.
Because neither developer will be privy to the problem, the two will have to collaborate. This differs from the usual programming challenge where the challenger sits with the candidate, parceling out information as needed to keep it moving. Also, using online resources is generally discouraged in a typical programming challenge, but in trial by pairing the goal should be to solve the problem using the resources normally available. That means Stack Overflow too!
What this can help you learn:
2.) Code review time!
This time pair your candidate with two of your devs. Together, have them all go over some code that the candidate has written. Ideally it would be something from their GitHub or BitBucket repos, but if they don't have any publicly available code you can give them a prompt or challenge and have them write something to bring to the session (you could even use what they wrote for the previous challenge). Have the candidate go through the code, explaining what it does, and have the other devs perform a code review as it is typically done at your company.
What this can help you learn:
3.) Code review time take 2!
If you are interviewing the candidate for a leadership role or want to assess the candidate's leadership capabilities, flip it around and have the candidate review someone else's code. Here again it can be something in a public repo, or the code they created in "trial by pairing." The key is for them to assume the role of reviewer and offer suggestions for improvements and/or where they would like to see the code progress to next; ideally, they would also point out good parts of the code, highlighting anything that is particularly elegant. If you believe, as I do, that recognizing good code is harder than recognizing bad code, this part of the exercise should be pretty fruitful.
What this can help you learn:
4.) I see your code and raise you a feature
To get a feel for how a candidate works when intimately familiar with a codebase--in other words, how they will be working for nearly their entire time as your employee--one way is to ask them to add a feature to a piece of software they have already written. My regex modeler, for example, currently doesn't handle lookahead. If asked in an interview to implement this feature, I could demonstrate proficiency through multiple means--showing how code that I have written is extensible and SOLID (or explaining what its deficiencies are, if it isn't), showing how I would gather requirements, how I write unit tests, how I write code, and so on.
A candidate can demonstrate some of these things somewhat doing a traditional coding exercise, but just imagine how much easier it is for them to demonstrate their real-world competence--and for you to evaluate it--under these circumstances.
What this can help you learn: