Technical interviews are generally dreaded, just like every other interview. However, technical interviews include many elements that non-technical folks might find mystifying or even pointless, such as whiteboard problem solving, take-home assignments, design sessions, or even just straight brain teasers. [Erik McClure] went a bit off the beaten path and started using the factory builder game Factorio as a technical interview.
Many point to the intent behind the problems and tricky questions inherent in whiteboard coding exercises and assert that the focus is not to complete the problem, but rather to expose how a candidate thinks and problem solves. Factorio is all problem-solving as you work as a team to slowly scale up a humble production line to a massive factory, which makes it a good candidate for assessing these sorts of skills. We doubt that the fine developers who wrote the game ever imagined it being used as an interview.
In all likelihood, you probably won’t have a Factorio interview anytime soon as [Erik] estimated each interview would take between eight and twenty hours. But we love the idea of reimagining the interview from a tedious set of problems to solve to an evolving cooperative game. Of course, you can also read more about getting the experience necessary for a job and what companies are looking for in an interview.
A trailer for Factorio is after the break.
I didn’t notice it brought up as I skimmed the article, but an 8 to 20 hour long interview had better include pay. I even like playing Factorio, but spending half a working week playing it as a test sounds like work.
Take home assignments are far more useful, you can justify paying the person for it if it’s a real life issue.
You can see how they solve it and if it meets your standards interms of performance, safety, documentation and tests. Then you can ask them to justify their approach.
Worst case scenario they got paid to work a simple problem, best case scenario you have a problem solved and a new team member.
If Factorio was a the take home assignment then the devs failed because they made a system to replace TI which they do not think will be used…
Yeah I came here to say the same thing, either they’re paying me for this or I don’t bother. Plus, meals compensated, of course.
I’ve done many all day 6-8 hour interviews, and a few that required multiple visits. I will admit I was paid … by my employer while I was out looking for a different job.
I would pass on anyone using Factorio for their interview process. It’s a big red flag. It’s usually safe to assume the place is ran by smart people who think they are a lot smarter than they really are. I’ve had my fill of dealing with companies that operate outside of reality.
Based on my Factorio experience I would become the CEO and the president of that company in no time.
That said while a game like that might give a small indication of one’s capabilities it is not sufficiently representative of the real world which is much more that applying simple logic and straightforward planning. Based only on that I would not rate that company very high neither on the potential of working for it neither as a supplier of whatever they sell.
I would never work for a company that asked me to play a videogame to assess me. I wouldn’t be able to take them seriously. It find it degrading.
You haven’t played factorio eh?
No working at a video game company then. ;-)
Luckily there are easier and better paying jobs than being a game developer. You have to love games to dedicate your career to them.
I’m a far simpler creature. I love designing software and bringing up kernels on new hardware. That’s general enough to let me get into multiple industries, and I’ve managed to mostly avoid the game industry. (mostly in that I still got sucked into some console OS development)
Isn’t this the same as IQ tests where IQ tests don’t show you how intelligent you are but actually shows you how good you are at solving IQ tests. I have never played factorio so immediatly I have a disadvantage against someone who is a master at the game.
Ofcourse this would be accounted for but i can’t see it being able to used practically if you want any type of emperical measure of the abilities of a person to be interviewed.
and yet, you can clearly see and understand the difference between a high iq person and a normal one, so they do something right…
I think it more or less would do one thing, indicate what your level of income was growing up as a child. https://www.popsci.com/us-army-world-war-i-era-intelligence-test/
Do you really want to work someplace that has zero respect for a strangers time, or reining in the bizarre conduct of their staff?
All screening processes are designed to filter for specific traits consistent with a company culture. The ones I screened for when I was tasked with such projects:
1. Are you going to cause problems?
2. What were you specifically responsible for on your last project?
3. Is your skill-set useful to the current team?
4. How much time is it going to cost us training you to be useful?
5. Is your ambition greater than your work ethic?
6. Do you still care about technology? i.e. did your time at Facebook/Google/Amazon/Apple/grad-school burn you out, and you secretly now want to be a plumber
7. Discuss three short questions that confirm one or more of the above. i.e. can you communicate with peers under stress, or are you preoccupied with telling us what you think we want to hear.
Each position took about 500 resumes, 30 in-person interviews, and 3 in-person callbacks. After 45 days, if you were not an asset to your team I would jettison the mistakes with extreme expediency.
Met one fellow with a forged copy of my own cv once, was the only time I grilled someone about the ethical conduct part of working in a trusted position. ;-)
I think you forgot: “Are you lying in your CV about your skills?”. If you get a forgery of your own CV, well, you’ll pick that one out quite quickly. :)
To be honest, most people don’t forge their CV. Most people just make their past job look just a bit better, especially in the department of ‘responsibilities I had in my last job’. I filtered out quite a few saying that they were responsible for building almost a whole app, while in reality, they had been doing mainly merge requests and code reviews, and reported to management. Really nice job, but checking out other people’s code and judging it, is not quite the same as producing something from scratch yourself.
Actually never met anyone who downright lied about their skills. But I am in software development, and people would be exposed quite soon. I figure that I do these people a favor by not hiring them, because the other devs would roast them quite soon, and their life would become hell quite soon. They’d end up with even less self-esteem than before. :)
I do think that take-home assignments are not overrated. At the minimum, it tests if a guy *really* wants the job. It’s quite easy to see if they put time and motivation in the assignment or not. But I have seen assignments handed in *textbook perfect*. What??? We give people 5 days, and expect to take the assignment maybe 3 to 4 hours. Seriously, anyone who’s applying for a job, while already having one, doesn’t have the time to make anything textbook perfect. And in the assignment I always write: “It doesn’t have to be textbook perfect, but keep this, and this, and this in mind, these are the basic requirements. Anything extra is just extra.”. It happens once in a while with developers from India.
My bet is that they hire someone to do the assignment for them, and that poor guy then works day and night for 5 days in a row to make it textbook-perfect.
Anyway. In the end, the most important questions are always: is the guy (or gal) willing to learn, does the guy/gal have enough skills to be productive immediately (even if only simpler tasks at first), and: does the guy/gal fit the team. Maybe that last one is even the most important question, because if they fit the team, they will be motivated by the team, and will grow easily.
I should mention that I work for a relatively small company, with a small team, where every member of the team counts, and where every person in the team has a slightly different skill set, and where it’s important to help other team members out if they are struggling with something that doesn’t quite fit their particular skill set.
Not like a huge company, where developers are made solely responsible for a task, but can take whatever time they need (within bounds), as long as it gets done.
And now we know why a history of FOSS development helps.
In the process for interviewing some candidates right now, and this is what I focus on.
Our test is a straightforward Python task that touches on some of the skills we need on the job. When reviewing the results I don’t care so much if they complete it as how they approach the problem and their style. I want to see if they can read the directions given, ask questions if they don’t, write code that others can comprehend, and that they understand the basics of the language.
We’ve hired a number of people who failed to complete the task. Usually they get nervous and ran out of time or spent too much time polishing one part of the program. We follow up in a later session to talk through their solution, and that’s when we get a sense of how they would approach a similar problem on the job.
Those are awful, stupid questions. Okay, specifically they are _random_ questions.
We don’t need “intelligence” or “aptitude” in a general way. And we sure as hell don’t need test-passing skills. And we don’t need “good communication skills”, we need people who are excited to talk about our business domain and what we can do to improve things. We need people who can really build stuff based on our real needs.
There’s only one kind of person you want to hire. This is the person who can just build stuff. And you won’t know that by giving them a take-home assignment or a white-boarding torture. You’ll know that by the outlook they have when you describe the problem. And it’s very important the problem is domain-specific and real to your business. People who actually really solve problems and do work talk differently than people who talk about orthodox solutions and workflow. If you don’t know the difference, you are the problem here.
In an engineering / development position, I don’t think it’s hard to come up with a task for a candidate to do that shows that they can ‘just build stuff’ in the scope of the position you’re hiring for. The important part is to give the candidate a challenge that gives them the opportunity to show you that they can do the work and surprise you with a unique approach rather than expecting them to give you a predetermined solution. There are many ways to ‘just build stuff’, and if you’re hiring people based on their ability to solve a problem exactly one way, then you’re building an inflexible team that’ll crumble as technologies and requirements change.
It should be noted, I often also asked myself if I could tolerate dealing with a person everyday for the next year. As there are plenty of smart people who would also pose a problem for a team dynamic. After meeting >10k people, spotting the folks with problematic personalities fairly quickly gets a bit easier. As a Manager I could see how toxic narcissism affected group stamina, and retention of skilled talent.
Assuming a technical problem scope is small enough for one person to handle in a single lifetime is naive. We rarely hired the Broflakes other companies seemed to prioritize… ;-)
I like both Ben and Changos approach, I’m in different field, controls and automation but like to see what a candidate thinks and why.
If they don’t answer perfectly, if I’m happy they think coherently then they are on the list.
Every job will need somebody training of candidates, if only to the team methodology.
If your ambition isn’t greater than your work “ethic” you will never achieve your full potential. Many of the best innovations and solutions start with biting off more than you can chew. If you’re not willing to do that, you’re doomed to a life of safe, conservative solutions and hard unsolved problems.
Perhaps what you’re getting at there is trying to figure out whether the candidate has an unjustified sense of entitlement? Or maybe I misunderstood and you meant you were looking for that as a positive?
You should strive to hire people who are going to work hard enough, but always have higher expectations.
And you should judge employees by how much they try to accomplish and how much they actually accomplish. There’s nothing worse than an employee with low expectations that tries to make up for their shortcomings by putting in extra hours rather than attempting to better themself.
I( hope this comes out legibly, the cursor is all wonky!)
The next big thing in the tech labor market? Recruiting 2.0 j: Interns are way too expensive. Therefore we have our software designed, implemented and maintained exclusively by job applicants. Our 17-round state-of-the-art hiring process is a reward in itself. Apply now and be the first to succeed!
To do this well they would need to set up a mod that breaks/establishes all new recipes. Anyone who has 100s of hours would know most or all recipes by heart and can just lay down the factories and grow. If you are new to the game then there are other things to learn. Factorio has tons of QOL features that you don’t pick up right away that make building so much faster. A noob won’t know those QOL things but changing the recipes would level the playing field in that one way.
The real problem is can the interviewer gauge how fast the candidate can learn and adapt.
Back in the day I was up for a technical support role and the interviewer came in, gave me a piece of paper and a pen and asked me to work out the number of gas stations in the US. Said I could take the time I wanted and would check in every now and then. About 10 minutes later I had an answer. He didn’t care about the number but wanted to know how I got to it (figure out # of cars in the US, figure out how much they drive, average mileage, gets you # of gallons needed per week, divide by number of gallons in a gas station tank – all complete guesses but hey, I was 25 at the time)
I got the job.
I used to think of interviewers as being the most knowledgable people in the company, perfectly capable to judge me even better than my teachers could judge my exams. I felt like every question from them was to judge me and my skills from the point of view of a wizard judging the apprentice.
Turns out that they are just like everyone else. They have their limited skill set, and are often asking questions that they don’t even know the answer to. But it’s not about the answers for them. It’s about judging how you think, what your character is, and to see if your way of thinking is compatible with the way the team thinks.
When I did my interview for my current job, they started out by showing me around, instead of starting the interview. And the first thing I thought was: wow, these people are so YOUNG! (me being 48 at the time, them being around 27 or so :P). I was hired, so I guess I fit. But that was the first time I considered lending the interview myself, because I was not sure if I could keep up with these young people, and end up hitting the brakes every time (actually it does happen often, not because I can’t follow, but because they tend to jump to conclusions too quickly :)).
A former employer used to drag me into interviews. You are experienced enough to know this, so for others a pro-tip: if the interviewer is competent and has a broad skill set, she probably does not want to be there and does not want to talk to you at any length.
Story time. At the end of a recent contract, the client gave me another job “that should only take 4 or 5 days because we have many qualified candidates.” They needed me to (and this was actually in the SOW for my job) find another engineer like me, only less than 40 ( I am over 60). Approximately 50% of candidates thought that their arduino experience fully qualified them as an embedded systems engineer. No candidates having only a CS degree had any useful hardware experience, so I focused on EEs doing embedded code and called in three candidates for interviews. I did not do any lengthy interviews or assign any at-home projects. I talked to their former and current employers. I asked them three questions about tool chains, power conversion topologies, and software safety principles. Found the (almost) perfect candidate but was rejected by client because she was 41. The language of the SOW allowed me to walk away from this job, but I did keep a copy of her resume and forwarded to another client whom now believes me to be a deity.
Two principles
1. Companies with stupid processes tend to hire less-qualified and more expensive people.
2. Companies that hire marginally qualified people tend to have more stressful working environments.
It’s way easier to hire intelligent people who can learn the specifics of the job than it is to hire people who already know the solutions to the problems you have. Your arduino hackers and CS degree holders with no hardware experience may or may not have been great candidates who could learn the – let’s be completely honest here – incremental knowledge required for building embedded systems relatively quickly when integrated into a team with other already-experienced engineers.
Even when you identify the “right” candidate who already knows everything you need to know, you might not be able to hire that person; those people generally want to be in a technical leadership role rather than being another individual contributor under somebody else with the same qualifications.
The obsession of hiring managers of screening out candidates who haven’t already been there and done exactly that, and recently, is the root cause of many of corporate america’s hiring woes.
When getting creative with interview questions, you need to do some work to be sure you’re *only* asking what you mean to be asking. I tried asking “Write a general function to navigate a maze given only turn_right(), is_facing_wall(), move_forward(), is_at_goal()”. And it turns out more than half of candidates have terrible position-relative 2d spatial skills. I junked that question after three dry runs.
Wonder how many had a background in CAD?
Interviews are just a method to find out why you AREN’T socially qualified which is odd for technical interviews. If they spring this on me, I just get up and walk out because the BS has started and I don’t even work there yet.
Dealing with bias issues in interviews (i.e. trying to eliminate them) is already really hard, I would be concerned that an biases (accidentally) coded into the game would skew the interview, unfairly holding back members of minority groups resulting missing out on hiring potentially great people.
While the problem of bias exists interviews without video games, the video game may introduce a level indirection, making the hiring bias much more ambiguous and harder to identify or fix.
Every possible bias against any genetic trait the computer cannot possibly know will be completely dwarfed by difference between gamers and non-gamers. The difference between gamer and non-gamer is a measure of a largely useless skill set. Knowing WASD controls should not be a good foundation for an interview for a good job.
Unless you are blind because of genetics, that impedes computer use a lot.
Mindustry?
Wow. Honestly, I never thought I’d be in the minority opinion on Hackaday. If you gave me Factorio as an interview question I’d be delighted. You want me to play video games with my potential boss, who ostensibly I could be spending more awake time with than my own child? I’d kill to have that amount of attention from a potential employer. Eight hours of play time is a small sacrifice. This guy is clearly just giving people an opportunity to advertise themselves in a non-confrontational and constructive way.
In the past I’ve spent hours preparing for dumb interview questions that sought to provoke thought processes. If you’ve ever prepped for an interview you’ve probably seen questions like: “why is a manhole cover round?” or “how much does a cloud weigh?” or “Show me how to make a calendar with two six sided die.” I think the real tragedy hiring managers feel the need to ask these types of questions. If I remember correctly, google walked away from these questions in their interview process because these questions don’t really serve any purpose.
I’ve got 40 hours logged on Factorio and I keep finding myself building ultra low pollution builds because I like the aliens in the game. It’s not exactly how the game is supposed to be played, I know. You could even argue that the Engineer is actually the real alien because he crash landed. Perhaps in eight hours I could explain that I want to respect the autonomy of a sentient species, and that despite the challenge, I’m certain there can be a solution.
Anyway I think a better game to use for an ad hoc interview would be Shenzhen I/O. That game comes with a PDF that replicates a microprocessor manual. I highly recommend it.
Why not just play some Tiger Woods golf? Using a specific video game that isn’t the actual work involves some really weird intentional personal bias in the process.
i.e. “I like this game? Do you like this game? No? Maybe you’re not right for the job…”
Interview question:
“I will give you a Stardate, please respond with episode title and guest star.”
That will let us know if they will fit into our team.