Technical Screening Questions.

[Andy Bober]

The company I work for uses a Java trivia test as part of a technical screening when searching for Java consultants. The questions on the test are core Java questions similar to what's found on the Java Programmer certification test.

Some examples of questions are:

"In Java are objects passed by value or by reference?"

or

"What must be done to an object to insure timely garbage collection."

or

"If I have an ArrayList of String objects named l, and I use l.get(i) method to get an item in the list. What is returned and what must be done to what is returned before it can be used?"

Is there any real value to these types of questions when looking for senior candidates? (consultants)
[ February 15, 2005: Message edited by: Andy Bober ]

[Ben Souther]

These sound more like questions you would start out with when interviewing for a full time Jr Developer position (a rudimentary understanding of core Java).

There is an excellent chapter in Joel Spolky's book "Joel On Software" on interviewing. His #1 rule is ALWAYs have applicants write code during their interview. He has them write code on paper. He lists examples. They're mostly algorithms, data structures, and problem solving in c. After they write the answer, he pushes them to optimize them, or debug them.
He'll tell them that he sees a bug (even if there isn't one) and pays attention to how they react. He looks for intelligence and coding skill -- not the ability to memorize and regurgitate facts. This is the approach I would take for a potential employee (for hire).

You said this was for consultants though....
These days, companies interviewing consultants are usually looking for experts in a particular area like Servlets, Swing, EJB, etc..

You pay a lot, up-front, for a consultant and in this economy, you don't want to pay for them to ramp up on a particular subject. If you're building a Swing app with an Oracle database and web-service hooks to a legacy system written in Cobol, you should be looking for someone who can demonstrate competency in every one of those technologies. If you can't find an exact fit, (say, no Cobol), look for someone who has connected a Swing app with some legacy app via web services. There are enough good people out there that you shouldn't need to compromise very much.

I think you've already formed your opinion of the questions.
Calling them Trivia sums it up quite well.
[ February 15, 2005: Message edited by: Ben Souther ]

[Johanna Rothman]

Ben's right. Technical questions that someone relatively junior can answer are not useful in an interview. (They may be useful in a phone screen, just to screen out the people who can spell Java, but who don't have experience.)

The most useful technique is an audition, where people use the language (or whatever you want to audition them about) to solve a problem just like they would at work. Auditions need to be related to the job, for you to get the most out of them.

Johanna

[Anselm Paulinus]

How many people have been to an interview before and where asked to write codes? Is the interviewer going to inform one upfront he has to write code or is one going to spit out the code like he is using a human language?

[Linda Pan]

This reminds me of one time I had to qualify for a technical position at a bank. They had a test that is given to all applicants regardless of what technical level or experience. I forgot the name of the test. There were I think 6 or 7 questions. Each question was as long to read as to figure out the solution. And if I remembered correctly there were no set answers. These were not specific to a language or technology. But I guess to determine some sort of aptitude. Has anyone taken such a test? Johanna, do you know what I'm referring to?

[Ben Souther]

Originally posted by Anselm Paulinus:
How many people have been to an interview before and where asked to write codes? Is the interviewer going to inform one upfront he has to write code or is one going to spit out the code like he is using a human language?

Does it really matter if you know ahead of time?
If you put down three years experience with Servlets and JDBC, you should be able to write a servlet that queries a database and presents the results.

Yes, I've written code during interviews.
I've also had to answer pre-written questions asked by a non-technical interviewer who couldn't really tell if I was answering them correctly or not unless I worded the answer just like what was on his answer sheet.

I'd rather write code.
[ February 16, 2005: Message edited by: Ben Souther ]

[Jeanne Boyarsky]

Anselm/Linda, I have gone to interviews where they have aptitude tests (informed in advance) and coding tests (told when I got there) when interviewing for my first (netry-level) job.

As a slight variant of this, what's the difference to a candidate between a verbal or written technical screening question? When I ask a candidate a question or ask them to write a line of code, I generally consider it to be the same type of question.

[Johanna Rothman]

Linda, I know the insurance and banking industries used to use some form of aptitude testing extensively. Maybe that's what you took? I haven't taken one in a long time, so I've forgotten. (Sorry.)

Anselm, I don't always warn people I'm planning an audition. It's part of the interview. I don't expect perfect code (or perfect tests or perfect project plans or perfect anything). I do expect to discuss the person's approach, and to see if they want to revise their work product once we've discussed it. I also find the candidate's comments useful. One developer asked me once "Who will review my code? I really like my code reviewed." Way cool.

Jeanne, you asked "what's the difference to a candidate between a verbal or written technical screening question?" When I was a developer, I preferred the written question, because I frequently forgot stuff in my excitement to answer the question. I liked having the time to think about the answer, write it down, and check it.

Johanna

[Linda Pan]

Johanna, I'm in consulting, so projects vary and time to time my skills may not match a particular project but you learn and get it done. My resume isn't updated yet but if it was, I would have worked on various technologies and only specialized on one. If I was targeting for a job which utilizes a different technology, would I be an unlikely candidate? What do hiring managers think when they see such a resume? Assuming that I am looking to work in IT/IS at a company (not a consulting company).

[Ben Souther]

Originally posted by Linda Pan:
Johanna, I'm in consulting, so projects vary and time to time my skills may not match a particular project but you learn and get it done. My resume isn't updated yet but if it was, I would have worked on various technologies and only specialized on one. If I was targeting for a job which utilizes a different technology, would I be an unlikely candidate? What do hiring managers think when they see such a resume? Assuming that I am looking to work in IT/IS at a company (not a consulting company).

The answer to that depends almost entirely on the economy.
I consulted from 99 to 02 and watched a general shift in attitude among employers. When I started, people were happy to get intelligent, all around, coders who could ramp up quickly. If they also happened to be seasoned with the exact technology that they were using that was a plus.
As the market shifted, employers, especially ones looking for consultants started looking more an more for specialization. This is understandable.
They went from waiting months for a recruiter to get SOMEONE in the door who was willing to code for them to having hundreds of resumes show up in the first few days after an add hit the job boards.

It's a simple case of supply and demand.
It makes no sense what-so-ever for anyone to pay someone to learn the technology they're using when there is a huge surplus of talented developers already familiar with it.
[ February 17, 2005: Message edited by: Ben Souther ]

[Mark Herschberg]

Originally posted by Ben Souther:

The answer to that depends almost entirely on the economy.

I disagree. I think the answer depends almost entirely on the company/hiring manager. Now in many cases, the manager does look to the economy. In '99, finding people was difficult, and many managers would make do with what they could get. Thse days those same managers want to find a perfect match, figuring the economy is in their favor.

Other managers, myself included, focus on smart people. If I'm hiring for a year long contract position, I'll usually take a smarter person and burn the first month or two while thy do no real work, but a lot of learning. I find those people superior to mediocre candidate who have prior knowledge in the technology. Where the line is drawn between direct experience and intelligence depends on many factors including: nature of work, intelligence utility curve to the job function (i.e. how much more valuable is 1 additional "IQ" point), experience utility curve to the job function (i.e. how much more valuable is one additional month of experience), cost of finding another candidate, duration of contract, capabilities of other team members, etc.

--Mark

[Ben Souther]

"Smart people" was a given.

With the current state of things, unless you're working with some really obscure technolgy, you don't need to choose between a smart candidate with general knowledge and a mediocre candidate with a specific skillset.

I do agree that the length of contract is a big factor -- the longer the contract the less specific the skillset would need to be. In a year long contract, 3 or 4 weeks to ramp up would be tolerable.

[Anselm Paulinus]

Originally posted by Johanna Rothman:
Linda, I know the insurance and banking industries used to use some form of aptitude testing extensively. Maybe that's what you took? I haven't taken one in a long time, so I've forgotten. (Sorry.)

Anselm, I don't always warn people I'm planning an audition. It's part of the interview. I don't expect perfect code (or perfect tests or perfect project plans or perfect anything). I do expect to discuss the person's approach, and to see if they want to revise their work product once we've discussed it. I also find the candidate's comments useful. One developer asked me once "Who will review my code? I really like my code reviewed." Way cool.

Jeanne, you asked "what's the difference to a candidate between a verbal or written technical screening question?" When I was a developer, I preferred the written question, because I frequently forgot stuff in my excitement to answer the question. I liked having the time to think about the answer, write it down, and check it.

Johanna

What happened to the logic of hire for the culture and train for the skills or what ever quote you had in your book?

In my current job I am using the Tapestry framework; I got hired for the job despite the fact that I had no prior knowledge o Tapestry, neither was it even mentioned in the job description, I wonder what would have happened if I was asked to write Tapestry code as part of the screening exercise.

Does it really matter if you know ahead of time?
If you put down three years experience with Servlets and JDBC, you should be able to write a servlet that queries a database and presents the results.

Yes, I've written code during interviews.
I've also had to answer pre-written questions asked by a non-technical interviewer who couldn't really tell if I was answering them correctly or not unless I worded the answer just like what was on his answer sheet.

I'd rather write code.

Yes if you put down three years of servlets and JDBC you should be able to do that; what would happen if you last did Servlet three years ago, Would it not be a different ball game? Does that mean you wont be able to do Servlet again if confronted with a real situation?

[Ben Souther]

Originally posted by Anselm Paulinus:


Yes if you put down three years of servlets and JDBC you should be able to do that; what would happen if you last did Servlet three years ago, Would it not be a different ball game? Does that mean you wont be able to do Servlet again if confronted with a real situation?

In this market, there is a good chance that there are 10 other applicants who've been coding servlets within the last six months. If you haven't touched them in 3 years and haven't brushed up on them lately then you probably aren't familiar with Servlet Spec 2.4 or 2.3.

If it comes down to you and someone else who seems equally intelligent, with the same track record of getting things done, but demonstrates that she is ready to sit down and start coding servlets (especially for a short contract) who do you think will win? Who should win?

If I could shoot us back to the dot-com era, I would in a second.
This is the reality of today's market.
Employers want all of the above.

[Mike Gershman]

It's a simple case of supply and demand.
It makes no sense what-so-ever for anyone to pay someone to learn the technology they're using when there is a huge surplus of talented developers already familiar with it.

Of course, this huge surplus of talented developers magically disappears when the big employers go to Washington to raise the H1B quota. Suddenly, there is a critical shortage of qualified applicants.

If I could shoot us back to the dot-com era, I would in a second.
This is the reality of today's market.
Employers want all of the above.

We don't need a time machine, we just need to require employers to hire US programmers with legacy experience and current certifications and "burn the first month or two" while they become productive, before they bring in H1B workers based on an alleged critical skill shortage. I think a one year H1B freeze should do it. Issuing green cards to existing H1B holders would be even better as that would quickly drive the H1B slave markets out of business.

[Jeanne Boyarsky]

Originally posted by Anselm Paulinus:
Yes if you put down three years of servlets and JDBC you should be able to do that; what would happen if you last did Servlet three years ago, Would it not be a different ball game? Does that mean you wont be able to do Servlet again if confronted with a real situation?

Anselm,
But you are going to the interview knowing that they want servlets and JDBC. That's a pretty good clue to review it. Even if the last time you touched it was three years ago, it comes back quickly if you really worked with the technology.

[Mark Herschberg]

Originally posted by Ben Souther:
With the current state of things, unless you're working with some really obscure technolgy, you don't need to choose between a smart candidate with general knowledge and a mediocre candidate with a specific skillset.

Wanna bet? How smart is smart enough? Chances are two candidates aren't equally smart and there's always som trade-off. I still have trouble finding smart candidates.

--Mark

[Linda Pan]

In consulting you really do need someone who can get up to speed or at least it would benefit the company because you need to be 'billable'. So far the new hires after myself in my company always start in the beginning to be in learning mode and not billable so it can vary from person to person. Some are not billable for some months.

I would assume a 'smarter' candidate would be someone well-rounded and able to roll with the punches, learn quick and get things done. Or defined as someone geeky and knows everything? And possibly have a gadzillion certifications? By the way did anyone mention certifications?

[Anselm Paulinus]

Originally posted by Jeanne Boyarsky:


Anselm,
But you are going to the interview knowing that they want servlets and JDBC. That's a pretty good clue to review it. Even if the last time you touched it was three years ago, it comes back quickly if you really worked with the technology.

We might argue this till the next coming of Christ but brushing up might not be enough to write codes in an interview but it might be enough to get you through questions and answers session, how many is one going to brush up with the inuendo of technology that human resources list in Job applications. In my current Job; the most vital part of the technology "Tapestry Framework" needed for the job was missing in the job requirement. I guess I would not have applied if I knew they needed it. For those in consulting we have to learn and relearn in virtually every job. I guess employers of labor are real people who understand the fact that a chap that did Struts last three years ago might be able to speak Struts but might not be able to write codes without adequate preparations. I guess there are a lot of factors they put into consideration in their hiring decisions.

[Ben Souther]

Originally posted by Mark Herschberg:


Wanna bet? How smart is smart enough? Chances are two candidates aren't equally smart and there's always som trade-off. I still have trouble finding smart candidates.

--Mark

We could continue splitting hairs on this one all week.
The bottom line is, if I'm interviewing two candidates who both seem very intellegent, can both breeze through the core Java questions or coding requests, but on one has been working on a Swing project for the last three years and can't demonstrate that s/he has stayed on top of web issues and can't write a "hello, world" servlet but the other has can sit down and code a Servlet, a context listener, and a filter, I'm going with #2.

There are always other factors to be considered (over qualified, location, salary requirements, etc...) and I never suggested that this would be the only consideration. Otherwise, a PC could do all the interviews for you.
The truth is, it's a buyers market and the interviewer has a duty to get the best candidate for the position. A good manager is not going to make his company pay to train someone if there is someone else who already possesses the needed skillset.

This is especially true for consultants. The shorter the contract, the more important it will be that they are ready.

[Mark Herschberg]

Originally posted by Ben Souther:

The bottom line is, if I'm interviewing two candidates who both seem very intellegent, can both breeze through the core Java questions or coding requests, but on one has been working on a Swing project for the last three years and can't demonstrate that s/he has stayed on top of web issues and can't write a "hello, world" servlet but the other has can sit down and code a Servlet, a context listener, and a filter, I'm going with #2.

Well duh. :-p

If you say A & B are the same except for experience it's not worth the energy to post the question, since we all know the answer. The way we break differently is that I will hirer the smarter candidate except in extreme cases and others prefer experience.

--Mark

[Ben Souther]

My point was, that in this economy, you are going to get lots of smart people. Lots and lots. In this climate, it makes sense that employers are also expecting very specific skill sets.

In a different economy, or if you are looking for a full time employee that might need to wear different hats, then I agree with you. Get the smartest person you can and you know s/he will be able to learn whatever needs to be learned.

[Mark Herschberg]

Originally posted by Ben Souther:
My point was, that in this economy, you are going to get lots of smart people. Lots and lots. In this climate, it makes sense that employers are also expecting very specific skill sets.

Right, my point is that you're not going to get lots and lots of smart people in this, or any, economy.

Smart people are in short supply. They had no problems the last few years, even in a down economy, because they are so hard to find. In an up economy, they are equally hard to find. Demand for smart people is relatively inelastic.

--Mark

[pchen]

Try taking the GOOGLE EMPLOYMENT TEST for any open position at Google in the U.S. India, etc. I'm trying to prepare for it. It's NOT on the web.

[Mike Gershman]

Try taking the GOOGLE EMPLOYMENT TEST for any open position at Google in the U.S. India, etc. I'm trying to prepare for it. It's NOT on the web.

Can you tell us more about it? How do you know what to study?

Thanks

[Jesse Torres]

Here is GLAT Google Labs Aptitude Test.

Enjoy,

[Sania Marsh]

I was recently asked to write a code at home as a first round of interview, before face-to-face. Then on the interview I was asked few questions on the strategies I choose.
I thought it was fair for each side.
I didn't waste 2 hours of their time writing code during the interview and writing it at home I was calm enough to write same quality code as I would write normally.