We’re a very small team with little experience in hiring but got approval for a new engineer. Basically HR will look for people through the usual channels and I think we have a reasonably good job description. Unfortunately the coding challenge (a 30h+ take home) is atrociously difficult and doesn’t really reflect what we do. On the other hand I think the false positive rate would be low. FWIW it’s a Linux application and it might be difficult to only count on experience from the CV.
Any ideas how to build a good challenge from scratch and what time constraints are reasonable?
A 30h+ take home that doesn’t even reflect what you all do is a waste of everyone’s time. I’d think most qualified applicants are going to ghost you when they are tasked with that. You have to keep in mind you’re not the only place they’re applying. Are you sure you want the engineer who has time for a 30h+ coding challenge for a potential job, that might then make a competitive offer?
I interviewed for a position that I was comfortably qualified for. As soon as they mentioned a 3 hour whiteboard interview in person I politely hung up the zoom call.
On the flip side, I had a company give the best interview process of all time. They told you how many people were remaining in the rounds. The programming task was to implement a hugging face model as a FastAPI. There was also a short video interview that took 5 minutes if you had basic ML knowledge. Likely took 1-2 hours tops and it was actually fun.
Short answer: don’t. Ask questions that demonstrate familiarity with concepts that you and your team encounter in a regular basis.
If you insist on a coding challenge, give them an old but difficult problem your team encountered in the job and assess how they would solve it.
I loved our test at my last company.
Do an API exposing a fizzbuzz. Make it “prodable”.
The definition of production ready is completely for the candidates to determine .
The subject is simple, doable in a few hours. What we were looking at was how code was organised, tests, usage of libs, …
You will lose the best candidates with an onerous coding challenge.
Our process, which has been heavily influenced by debate on r/experiencedevs on reddit involves a short phone screen, a 30 MINUTE coding challenge, a tech interview consisting of pair programming, and a non-tech interview with management. Very light.
The coding challenge is a FILTER only. It’s not to evaluate who to hire, but instead it’s to filter who not to continue interviewing.
You learn a lot during pair programming in a short period of time, including personality and team fit. We let them drive and we just watch and discuss. The assignment is to fix a bug, and refactor the code the caused the bug.
Time constraints should be 1 hour maximum and it should be a pair programming thing done over webcam. Why? I refuse to do “take home” assignments. They take far too long (even if it is only 30 minutes) and there’s no guarantee that anyone will look at it. Don’t waste your candidate’s time. You’re going to push good candidates away doing this.
What is your goal for the coding challenge? Right now you’re screening for the most desperate and willing to cheat with a 30hr busywork assignment.
I’ve only ever had one job ask me to do a homework style challenge and I thought it was weird and off putting.
I have a full time job, and a family, and a lot of other shit going on. If you give me homework I’m not going to bother with your company.
I feel like giving out homework in a job interview is a good way to filter out anyone who is busy being gainfully employed already, and leaves you with a weaker pool on average to draw from.
Sure there are some stinkers out there, but if you can’t suss that out in an interview that’s on you.
I am not gonna do a 30hr challenge. I barely have 30 hrs to spare in a month and I’m not gonna spend it for a chance of getting a position. 4 hours max, and not even something too difficult. As another commenter suggested, use it as something to discuss in an interview to get an idea of how they think and make decisions.
You guys pay for that time or do you try to get it free like new code challenges?
On the other hand I think the false positive rate would be low.
Yeah, because any good engineer will noop the hell out of there and anyone left will likely not pass. Cannot have any false positives if you filter everyone out…
1 hour is more reasonable, 2 hour max I would say.
Just a note by setting up a 30 hours home project you effectively removing “people with lives” from your hiring pool. People who can do a 30 hours either have a lot of freetime currently, or code after the job. And if you really want those people in the team then go ahead, but you are missing on 8 to 5 crowd and that is a very good and diverse talent pool. From my experience 8 to 5 minded people are very good in solving tasks in sustainable manner. They just don’t have time to fuck with the system and doing effective “dont-call-me-at-night” solutions.
if you are doing a lot of interviews you need a common set of questions and measures and this take a lot of time and effort to setup.
Personally I would suggest to setup interview as a two parter first ask some theoretical questions and then ask to create a simple code with simple problem related to the questions. This helps to find out if people are really understand what they talking about. This again require a lot of thought to setup an to have small practical tasks relevant to the questions.
For example in most recent interview I asked candidate about algorithm complexity, data structures, garbage collection and then asked them create a simple dictionary to store a hierarchical structure. This helped to see if candidate knows what he is talking about and can use his knowledge in practise. I have seen a lot of people without good theoretical knowledge, but they create a code that is good and working despite their gaps and other way is also correct people have a good theoretical knowledge but fails to apply it in practice.
So figure out who you are searching for. Create an ideal checking solution for their skills and start combing the desert. There is no shortcuts in hiring, sadly.
Thanks for the comment. Yeah good point, I really miss people with the 8-5 mindset. At the moment there are too many people who just throw things over the wall, giving me nightmares once a month. (And users hate it) Already settled for something smaller but that definitely makes me feel better with my decision.
