developers who can't develop

[Jeanne Boyarsky]

I was talking to some of the moderators here about frustrations in interviewing developers who can't code and was referred to this.

Is there anything we can do about this problem? Or to screen people out faster? It seems like a waste of time to bring in a "developer" for an interview that can't develop.

[Kr Manish]

This Codility is being used by some companies. The run of the mill service companies do not bother to go to this extent but those for whom quality & interview time matters, use this. In India this is rarely used, though. Maybe in US it is used little more than here.

[R. Grimes]

Why not require what I usually bring with me and volunteer to the prospective employer? A sample of recent code. A savvy employer should require a resume AND sample code. It should be further explained to the candidate that the code sample will be discussed in detail, with questions about how the same "problem" could have been tackled through alternative means, and why s/he didn't go that route. This exercise will let you see:

a) how clean is the guy's code
b) how well do they document it
c) how well do they abstract problems
d) how logical is their reasoning processes as they explain to you alternate choices and why they went with one choice over another.

If a candidate knows this is what awaits him/her, that alone will filter out the posers and wanna-be's.

This can be used in conjunction with the debugging "test" described in the article you linked to. However, the problem I have with this is you'll miss some good developers who will simply suffer from a brain freeze when presented with a program and asked "Where's the bug at in this code?" It can be very nerve wracking to have an interviewer staring at you while you look over a program and are expected to spot the bug(s), knowing the job offer depends on you getting this right. In other words, some good developers aren't necessarily good test takers when put on the spot and are expected to find something in short order.

Ron Grimes

[Kr Manish]

R. Grimes wrote:However, the problem I have with this is you'll miss some good developers who will simply suffer from a brain freeze when presented with a program and asked "Where's the bug at in this code?" It can be very nerve wracking to have an interviewer staring at you while you look over a program and are expected to spot the bug(s), knowing the job offer depends on you getting this right. In other words, some good developers aren't necessarily good test takers when put on the spot and are expected to find something in short order.
Ron Grimes

True, someone might be able to do wonderful work in the confines of the office & having some time, but may not be able to crack it then. Yeah, it might result in false negatives.

[Campbell Ritchie]

What do you mean by "false negatives"?

[Jeanne Boyarsky]

Ron,
What if someone else wrote the code and taught the candidate about it?

Kr,
Even if the person missed the bug, the process of explaining what he/she is thinking is the important part. Besides, we don't always have the luxury of time when looking for a bug. When there is a production problem, time is hard to come by. All else equal, I prefer someone who doesn't freeze.

[Gary Marshall]

However, the problem I have with this is you'll miss some good developers who will simply suffer from a brain freeze when presented with a program and asked "Where's the bug at in this code?" It can be very nerve wracking to have an interviewer staring at you while you look over a program and are expected to spot the bug(s), knowing the job offer depends on you getting this right. In other words, some good developers aren't necessarily good test takers when put on the spot and are expected to find something in short order.

Here! Here!. Unfortunately I am one of those individuals, who, while colleagues have given great references about me when asked for them by recruiters as well as department managers I have worked for, I experience this "brain freeze" not only on job interviews, but on timed tests (hello SCJP?). Nerve wracking is putting it lightly. On a recent job interview I was given a test, while being observed by four employees, whereby I was to determine the next logical symbol within a set of symbols. The individual who was administering this test, gave it to me one question at a time and for each question he would hit a timer on his iphone so that he could register how long it took me to come up with an answer. This was nerve wracking. When the test was completed I was told that I got half of questions correct. I mentioned to them that I do not do well on timed exams and the person who gave the test said it wasn't a timed test, to which I replied it was, since he was hitting a timer on each question.

And BTW Jeanne, if you have a production problem that needs to be fixed real quick then perhaps having a resource on hand who can fix the problem quick will be what you need. I agree with what was stated, don't miss out on obtaining a great developer because he/she "freezes" when everybody in the room is waiting for an answer and getting the job depends on coming up the the right answer.

Oh.... and I did get an offer from the company that gave me that "it wasn't a timed test" test....

Thanks
Gary

[fred rosenberger]

I would have a hard time bringing in a sample of recent code. All the code I write for my employer is owned by them, and not mine to show off. I'm pretty sure asking them to let me show off their code to some other potential employer would not go over very well...

[Kr Manish]

Campbell Ritchie wrote:What do you mean by "false negatives"?

By false negative I meant good resources who otherwise would code perfectly but may freeze in the tests(a negative which is false) thus undermining their actual abilities ( of course working under pressure & time constraint would not be one of their qualities.) I know stupid place to use that term

[Campbell Ritchie]

Thank you for explaining what you meant by "false negative".

[Jimmy Clark]

In U.S.A., the Department of Labor should be able to handle this easily in conjuction with state governments. Something similar to the bar exam for practioners of law could be applied to software engineering. Software engineers can be tested and certified by the state. This type of certification program would be very valuable. A certified Software Engineer could then be able to administer the hiring processes of junior positions similar to the Para-Legal concept.

The current certification programs which are controlled by private companies and also serve as marketing programs for the company's products are not effective and are weak measures of an individual's abilities.

[Jeff Langr]

Some people answer technical questions over the phone well; other people don't really fare so well on the phone, even just in general, so when I did a lot of hiring or hiring assistance, I would usually run a follow-up via email:

http://www.langrsoft.com/blog/2005/01/interviewing-email-challenge-ive.html

The email challenge did a good job ferreting out the people who do well in interviews, but whose code stinks. It also allowed people who don't do so well over the phone to shine in a forum more amenable to them (most candidates got the benefit of the doubt). One of the developers I hired (when I was a manager for about a year) about 6 years ago did horribly over the phone, but did an excellent job on the email challenge. He ended up being one of the top developers on the team.

Jeff

[Jeff Langr]

R. Grimes wrote:This can be used in conjunction with the debugging "test" described in the article you linked to. However, the problem I have with this is you'll miss some good developers who will simply suffer from a brain freeze when presented with a program and asked "Where's the bug at in this code?" It can be very nerve wracking to have an interviewer staring at you while you look over a program and are expected to spot the bug(s), knowing the job offer depends on you getting this right. In other words, some good developers aren't necessarily good test takers when put on the spot and are expected to find something in short order.

It takes me only a few minutes of pairing with someone, coupled with a bit of conversation about the code, to determine roughly how good they are. My take: If an employer is willing to offer a lot of money to supposedly smart people, they need to be able to deal with just a bit of pressure by being put on the spot. The problem doesn't have to be tough at all (and really shouldn't be), and shouldn't be so much a 'test' but instead just a casual code-building session. You can even ask the candidate to choose something simple of their own to tackle. Or just do fizz-buzz.

I once paired with a guy (not for hiring) on a small team. He was a senior developer with over ten years' experience, and perhaps a couple of those in Java. It was apparent in about one minute of watching him struggle with the simplest syntactical issues and programming concepts that this guy was a huge productivity sink for the team. Yet he'd been with the team for some time. Why do we let this happen?

With respect to a debugging challenge, finding the bug isn't the most relevant part, it's watching how people go about it. I wouldn't send a candidate after some nightmare bug, but it's certainly fair to toss a small chunk of code at them and get a sense of how they might pinpoint a problem. I once hired a programmer who built decent-looking code, and talked the talk ok, but when it came down to finding defects, she was abysmal, and ultimately had to go.

Given the lame coding abilities of such a huge number of developers out there, it's a huge mistake to not see how someone codes before hiring them.

Jeff L.

[Rachel Maria Howard]

Having a code sample is good, but I have found not enough. I firmly believe that to really tell a person's true skill level or their potential (which should definitely be taken taken into consideration) you need to have someone sit down and do pair programming with the individual. I think its more important to see how someone works through a problem to get the answer than to actually see if they can solve it. Some of the best programmers that I have hired did not actually get the desired answer but the steps they took to get there really showcased their potential and their critical thinking skills. All it took was a couple weeks paired with a more skilled craftsman to get them where they needed to be. If I had just made a decision based on their code sample I might have overlooked them. I have also had the opposite problem where someone's code looked good, but when they were actually paired with one of my developers they failed miserably. I would also recommend having someone spend time in the team room and to get opinions from the team members as to how well the individual would fit in from a cultural fit. Having a cohesive team is one of the more important aspects of a successful agile environment.

[Jimmy Clark]

Geeky programmer types many times have personality challenges or ego-based deficiencies which interfere with the hiring process. When these types become managers things typically become worse and they are unable to hire and/or identify good candidates from both technical and social aspects.

There really is no need to go through the exercise of testing programmer candidates. If a hiring manager posesses sharp skills and knows how to identify and explore individuals through conversation and intellectual debate, then there is no need for all "testing" techniques mentioned in this thread.

If a hiring manager is unsure of their abilities to distinguish a phony from a real programmer, there are more issues here than meets the eye. Interview testing is flawed in many ways and in order to get it right takes significant thought and intellectual design. If you have the skills to do this right, then you most likely do not need to do it this way...

[Jeff Langr]

I'll respectfully disagree. I just now had a conversation with a guy on my team, very sharp developer, able to crank out code. Had I been a manager and had based my decision solely on questions I could ask, I probably would have said yes. Yet I'm not sure I would hire this guy on the basis of seeing his code now--dense, difficult to understand and maintain, and sloppy. It works, but we pay the price for it in maintenance costs and defects. He may be a "real programmer," and I'm able to tell that from conversations with him, but that's missing a huge productivity factor.

The bar exam idea is interesting--it would have to be fairly rigorous to overcome some of the challenges you suggest exist with proprietary testing, and thus costly for all involved (particularly if the government is designing it--bah!). Why create all that when all it takes is five to ten minutes of a good programmer's time to ferret out the people who don't belong?

Every shop that I've been in where pairing was already a factor found a short pairing test the simplest and most effective way to find people they wanted on their team. And I've already avoided a couple big hiring mistakes by a small investment in checking out code first, so why dismiss it out of hand?

[Parag Pathak]

Jimmy Clark wrote:Geeky programmer types many times have personality challenges or ego-based deficiencies which interfere with the hiring process. When these types become managers things typically become worse and they are unable to hire and/or identify good candidates from both technical and social aspects.

There really is no need to go through the exercise of testing programmer candidates. If a hiring manager posesses sharp skills and knows how to identify and explore individuals through conversation and intellectual debate, then there is no need for all "testing" techniques mentioned in this thread.

If a hiring manager is unsure of their abilities to distinguish a phony from a real programmer, there are more issues here than meets the eye. Interview testing is flawed in many ways and in order to get it right takes significant thought and intellectual design. If you have the skills to do this right, then you most likely do not need to do it this way...

World does not look same for everyone. A highly skilled person may call it it a self respect to which some unskilled recuritment agent call ego based difficiency. Remember good and bad are relative terms. Something good for someone may be bad for some other person. Also remember only skilled people have self respect. Other do not.

[Matthew Brown]

Parag Pathak wrote:Also remember only skilled people have self respect. Other do not.

Sorry, but...what?

[chris webster]

Interesting topic and responses...

Not sure about the coding test thing, as I've never had to do that in an interview, thankfully (although the pair-programming approach might be fun and I might actually learn something ). I'm a freelance Oracle developer (only SCJP so don't push me too hard on those Java libraries!), so I have been through a lot of interviews. My hit-rate is pretty good overall, and I usually get offered contract extensions if they're going, so I presume my clients find something acceptable in my work.

In most cases I've found a thorough technical interview is usually a pretty good test of somebody's abilities. It's not just what answers people give, but also what their responses reveal about how they think and the extent to which they can bring in real-world examples of how they might have dealt with similar problems in the past. Of course, you need to have somebody on your team who knows the answers to the questions and can judge the quality of the candidate's response, which is not always the case in some organisations, especially when they're hiring temporary contractors precisely because they don't have the skills in-house. If you're hiring people and don't know how to test their suitability, then you're probably in trouble anyway!

But a coding test wouldn't solve that problem either, as you wouldn't be able to tell if the candidate's solution was any good unless you had some good coders in-house already. In any case, asking somebody to code a prime number checker or whatever is kind of pointless - who needs it and so what if they can do it? I've worked with some very clever computer science graduates who know all kinds of clever stuff about algorithms and writing compilers and so on, but can't actually produce robust maintainable code to meet a real business requirement in anything like a reasonable time. And the same thing applies to obscure APIs etc - the skill is not in learning all those method signatures parrot fashion to regurgitate in interviews, but in knowing when and how to find the answer to a real problem that might require those tools. Most problems I encounter in my daily grind require thinking skills before coding skills.

Some organisations really do need computer science boffins who can perform cute tricks on coding tests, but a lot of places actually just need pragmatic, experienced software craftspeople who can produce reliable, maintainable solutions to real-world business problems. Let somebody else pay the geeks to invent a new sorting algorithm or a prime number generator.

I dunno, maybe it just boils down to taking charge of your recruitment process: whoever you take on, put them on a probationary period and make sure you find their strengths/weaknesses during that period. And make sure the recruitment agency doesn't get paid unless the candidate passes probation.

[Kr Manish]

Parag Pathak wrote:World does not look same for everyone. A highly skilled person may call it it a self respect to which some unskilled recuritment agent call ego based difficiency. Remember good and bad are relative terms. Something good for someone may be bad for some other person. Also remember only skilled people have self respect. Other do not.

Seriously ......... WHAT ?

To answer you logically, a highly skilled individual WITHOUT an attitude conducive to a team environment would fetch nothing substantial to the team he would be hired into; in fact he would prove to be a hindrance. Its not a PhD/R&D program where individual excellence has more weight-age, but a team of collaborating people. If someone is THAT brilliant, he has plenty of other non collaborating places like I mentioned earlier to venture into.

[R. Grimes]

Jeanne Boyarsky wrote:Ron,
What if someone else wrote the code and taught the candidate about it?

Kr,
Even if the person missed the bug, the process of explaining what he/she is thinking is the important part. Besides, we don't always have the luxury of time when looking for a bug. When there is a production problem, time is hard to come by. All else equal, I prefer someone who doesn't freeze.

That is a possibility, but I don't think the person who really wrote it could possibly teach the person about all the questions you might ask. For example, I might ask, "So, why did you choose an ArrayList here instead of a Vector?" Or, "Why did you place this logic in the service class instead of the DAO?" The questions you can ask about any given piece of code are infinite. The idea that I could write code for someone and teach them to fake it is not plausible. A good interviewer would quickly discover this. If you expect the interviewee to be quick on his/her feet, then you should be quick on your feet and be able to look at a piece of code the interviewee brings in and immediately come up with a dozen clever, insightful questions about it that they could not possibly have been prepped for.

As for your response to KR, you may not "always have the luxury of time when looking for a bug", that's true. But, I have yet to have anyone stand in my office staring at me while I perform the process of debugging. And, I've been at this 27 years. So, again, the debugging scenario in an interview does not reflect what you will really expect out of an employee in real life situations.

Ron Grimes

[Jimmy Clark]

In terms of testing candidates, this practice is best applied by the hiring organization. Moreover, test results should be treated highly confidential and private and should never be accessible from the Internet. Recuriters and third-party testing organizations should never be used. Test design and development should be executed by the hiring authority only.

In the best scenario, all test results should be immediately destroyed once a candidate is selected and hired. Preserving this type of data is a risk both to the organization and the candidates. If data is used incorrectly and exposed in the wrong way, there are grounds for lawsuit based on per se slander and defamation. Proceed with caution...o)

[Parag Pathak]

Kr Manish wrote:If someone is THAT brilliant, he has plenty of other non collaborating places like I mentioned earlier to venture into.

Agreed to some extent but it should be there are plenty of other GOOD places. It should not be IT only. Because dimaonds can shine anywhere and grapes are sour with ego.

[Jeff Langr]

chris webster wrote:whoever you take on, put them on a probationary period and make sure you find their strengths/weaknesses during that period.

Absolutely! Great advice--it's for their benefit as well as yours (I'm in a 3-month contract-to-hire probationary period now, and the nice part of that three months is it will allow me to escape this place, with little impact, if I need...).

Jeff

[Bear Bibeault]

I agree. I foolishly refused a contract-to-hire deal at my previous position. It turned out to be a complete nightmare and it would have been much easier to bale sooner if I had taken the contract-to-hire route.

[Sean Landis]

I am late to the discussion and most of the good ideas have been mentioned. Maybe I can add a holistic view to what's been said.

I think it is critical to validate whether the candidate has the skills you require for a position. The typical hiring process goes something like: resume reviews to select potential candidates. Phone interviews to cheaply validate the candidates deserve the investment of an on-site interview, on-site interviews, and then offer or not. Each step in the process builds evidence for decision-making. Each step is an opportunity to move the candidate forward in the process, or reject them. At each step, the hiring team attempts to discover the depth of ability the candidate possesses in those areas most important for the position.

One of the main reasons companies hire people that do not fulfill the job requirements is that they don't know how to discover whether the candidate is competent, at each step of the hiring process. So training is important.

Several good techniques were mentioned on how to validate development skills. When I hire, I insist on having the candidate solve problems - coding problems, design problems, architecture problems - whatever is appropriate for the position. For code, we rely on whiteboards. That sounds primitive but surprisingly there are many reasons why this is good for us. It turns out whatever approach you use: whiteboards, IDEs, debugging a problem, doing it in private or in front of the interviewers, taking a written test, sitting with the team for a day, they all have strengths and weaknesses. They all select for certain things, and hide others. They all have psychological and logistical challenges.

I think the important thing is to choose techniques that are strongly related to the position, your environment, your culture, and that are likely to shine a light on the things that are most important for your situation.

[Jeanne Boyarsky]

Lots of good comments in here. I'm happy to see so much discussion.

Jimmy: are you proposing not having the candiate do anything hands on? How do you differentiate a developer from a manager who used to develope, still speaks about it well and can no longer code?

[Jeff Langr]

Jeanne Boyarsky wrote:How do you differentiate a developer from a manager who used to develope, still speaks about it well and can no longer code?

Hi Jeanne,

:-) It's like a bicycle, isn't it?

One time, I went away to band camp, and couldn't code for a whole week, but when I came back to it, it still fit like a glove. No wait... I mean... One time, I was a manager for a while, and ... wait, I still coded regularly even then. Well, happily, I don't think I've gone longer than a few weeks without programming, for the past ~30 years. Probably shouldn't ask me. Maybe I should've just kept my mouth shut. :-)

I worry about the folks who can speak well about it but never could code. I do suspect that people who were good in the first place were good because (a) they cared enough to be good and (b) had good problem-solving skills. I think those things don't go away easily.

Jeff

[Jimmy Clark]

First, a sharp developer that evolves into a manager will never magically forget how to write code. They may not have detailed knowledge of a particular programming language's syntax, but the core principles of all computer programming languages are the same for all languages and haven't changed in 50+ years. If somehow they can no longer code, there are other issues here and maybe they never really understood what they were doing as a developer.

[Sean Landis]

Jimmy Clark wrote:First, a sharp developer that evolves into a manager will never magically forget how to write code. They may not have detailed knowledge of a particular programming language's syntax, but the core principles of all computer programming languages are the same for all languages and haven't changed in 50+ years. If somehow they can no longer code, there are other issues here and maybe they never really understood what they were doing as a developer.

It is like riding a bicycle. But if you were a bike racer, and take two years off of riding a bike, you aren't going to hop back on the bike and enter race. I took two years off from daily coding to management and went back to coding. I still knew how to code but I performed at a much lower level for some time. As that time off increases, I bet the 'recovery' becomes more challenging. Depending on the individual, recovery may be hard enough to become unreachable. When I hire a developer, I don't usually want to take on the cost of 'rehabbing a former manager' :-)

[Jimmy Clark]

A sharp developer that evolves into a manager will typically possess an intrinsic passsion for software engineering and will most likely stay somewhat "tuned-in", while managing and mentoring his/her staff. Circumstances will vary, but this type of developer will never take "time off." It is not a wise career decision and can have negative effects in the future, e.g. $$$.

[chris webster]

Jimmy Clark wrote:First, a sharp developer that evolves into a manager will never magically forget how to write code. They may not have detailed knowledge of a particular programming language's syntax, but the core principles of all computer programming languages are the same for all languages and haven't changed in 50+ years. If somehow they can no longer code, there are other issues here and maybe they never really understood what they were doing as a developer.

As a contractor, I have had some long spells out of work between jobs, before being expected to hit the ground running pretty fast when I find a new job. I think you can re-adjust to life at the code-face pretty quickly, if you make an effort to stay in practice while you're not actually coding e.g. I try to pick up new languages/tools and read about different aspects of the business while I'm stuck on the bench. And it's fun too, of course.

As a qualified translator as well (don't ask...), I find coding skills are bit like advanced language skills: you may be a little rusty if you haven't used a language at a high level for a while, but it will come back pretty quickly once you're using it every day - usually a lot more quickly than somebody with lower skills would be able to improve to your own level.

In any case, as I said above, an important part of being a good developer is not just coding ability, but thinking ability, and as far as I can tell you don't lose those skills very quickly. Recruitment agencies don't see things that way, of course, because they tend to rely on buzzword-bingo to identify candidates with the "right" skills and automatically ditch people who have been out of work for any length of time, which is why it can be very hard for developers to break back into development work once they've been out for a while.

But next time you are recruiting for your own organisation, it might be worth asking yourself if you would be better off recruiting a slightly rusty but experienced developer who can get up to speed again pretty quickly and bring lots of practical experience to your project, or a developer with up-to-date coding experience but not much of it. IMHO, a good development team usually needs a healthy mix of cool new coding skills and deeper/broader development experience: it's up to you to find the right balance for your own organisation.

[chris webster]

Jeanne Boyarsky wrote:How do you differentiate a developer from a manager who used to develope, still speaks about it well and can no longer code?

These days it seems people move so quickly into management (or out of the industry altogether) that they never have time to become good developers in the first place!

[Jeanne Boyarsky]

Chris,
Thatis an excellemt point.

Jimmy,
I'm talking about someone who isn't a techie at heart that codes in his/her free time.

[Jeanne Boyarsky]

On the question Chris poses: if i had a way to tell if a developer was rusty but good as opposed to bad, it would be good to consider. That said, a good developer doesn't get that rusty. On specific areas yes, but not on coding in general and writing simple algorithms.

[Sean Landis]

chris webster wrote:

Jeanne Boyarsky wrote:How do you differentiate a developer from a manager who used to develope, still speaks about it well and can no longer code?

These days it seems people move so quickly into management (or out of the industry altogether) that they never have time to become good developers in the first place!

Hi Chris,
That might be an over-generalization, but certainly true in some places. I guess the good news is that one reason people are moved too quickly because there's a demand for good people. The flip side might be that companies promote too quickly because they have high attrition.

[R. Grimes]

Jeanne Boyarsky wrote:How do you differentiate a developer from a manager who used to develope, still speaks about it well and can no longer code?

I think that's an oxymoron. I have yet to meet a manager who is a good developer. And, if you're a good developer, they can't afford to let you go into management because so few can truly write solid code and develop good systems. My experience at 12 companies, over the past 27 years, is that management is a good place to put people you like, and want to keep around, but you don't want them coding.

Ron Grimes

[Bear Bibeault]

R. Grimes wrote:I have yet to meet a manager who is a good developer

You need to get out more.

Yes, I'm management, and yes, I consider myself a good developer.

[R. Grimes]

Bear Bibeault wrote:

R. Grimes wrote:I have yet to meet a manager who is a good developer

You need to get out more.

Yes, I'm management, and yes, I consider myself a good developer.

So does my boss. He thinks he's as good as I am, and all he knows is one language: RPG IV on the AS/400. A procedural language, no less.

[Jeff Langr]

I've met a lot of managers, and I honestly haven't met a lot of good ones... managers that is.