Employer looking 4 online C++ aptitude tests; recommendations?

M

Matt

Hello,

I'm a hiring C++ developer employer looking for existing, online C++
aptitude tests.

I have not yet extensively researched this yet, but as an example, I
thought this test looked pretty good:

http://expertrating.com/c++test.asp

I'm curious if anyone can offer any other recommendations? If so,
could you please offer your experience with your recommended resource
and why you recommend them...or might recommend others?

Fyi, My philosophy falls in line with this post:

http://groups-beta.google.com/group/comp.lang.c++.moderated/msg/f840d9074af466c1

-Matt
 
G

Gianni Mariani

Matt said:
Hello,

I'm a hiring C++ developer employer looking for existing, online C++
aptitude tests.

I have not yet extensively researched this yet, but as an example, I
thought this test looked pretty good:

http://expertrating.com/c++test.asp

I find it a bit offensive to have a candidate pay.
I'm curious if anyone can offer any other recommendations? If so,
could you please offer your experience with your recommended resource
and why you recommend them...or might recommend others?

Fyi, My philosophy falls in line with this post:

http://groups-beta.google.com/group/comp.lang.c++.moderated/msg/f840d9074af466c1

I just went through the process myself in creating a bunch of questions.

Nothing beats creating the questions that you feel work for you.

What are important skills for this position ?
 
M

Mark

my opinion on this is that , if you are looking for the best staff,
a simple C/C++ test is not enough. You should create your own
based on your own requirements / what you think is important for
a programmer to know.

The best test I was ever given was to be sat in a room in front of an
(unfamiliar) C compiler and given an hour to write a program to
read in a proprietary file containing graphics shapes and draw it to
the screen. I did'nt quite finish the exercise, but it showed them
what they needed to know. Could I code, and what was approach to
solving the problem re: design.

I consider myself an intermediate C++ programmer, I still get caught
out by syntactic sugar , the sort of thing a C++ tests are designed around
but I don't consider myself any less a programmer for it.

Hope this helps.

Mark
 
M

Mark

my opinion on this is that , if you are looking for the best staff,
a simple C/C++ test is not enough. You should create your own
based on your own requirements / what you think is important for
a programmer to know.

The best test I was ever given was to be sat in a room in front of an
(unfamiliar) C compiler and given an hour to write a program to
read in a proprietary file containing graphics shapes and draw it to
the screen. I did'nt quite finish the exercise, but it showed them
what they needed to know. Could I code, and what was approach to
solving the problem re: design.

I consider myself an intermediate C++ programmer, I still get caught
out by syntactic sugar , the sort of thing a C++ tests are designed around
but I don't consider myself any less a programmer for it.

Hope this helps.

Mark
 
J

Jerry Coffin

Matt wrote:

[ ... ]
I have not yet extensively researched this yet, but as an example, I
thought this test looked pretty good:

http://expertrating.com/c++test.asp

It's not particularly great. I probably shouldn't have wasted my money
on it, but I decided to look at it, and noticed questions like this:

-> Question:
-> State which of the following are true:
->
-> a. You can use a goto statement to jump into a __try statement block
-> or a __finally statement block
-> b. You can nest an exception handler or termination handler inside a
-> __finally block
-> c. You can use a goto statement to jump out of a __try statement
-> block or a __finally statement block
-> d. All of the above are false

This was supposed to be a test about C++, wasn't it? Where did this
nonsense about __try and __finally come from? WRT to C++, even D isn't
really correct, because none of them really qualifies as truly true or
false, but merely irrelevant. Even if the question were rephrased to
deal with C++, it's still downright stupid anyway! Given a limited
number of questions, which subject should be included: goto or (for
example) partial specialization?

As another example:

-> Consider the sample code given below and answer the question:
->
-> class A{
-> public:
-> A() {}
-> ~A(){ cout << "in destructor" << endl;}
-> };
->
-> void main(){
-> A a;
-> a.~A();
-> }
->
-> How many times will the destructor be called in the above code?
-> a. 0
-> b. 1
-> c. 2
-> d. There is an error because destructors cannot be called directly


I suppose they probably want C -- but they're wrong. The correct answer
is that the code has undefined behavior. Nothing more and nothing less
-- any arbitrarily chosen result is equally correct for this input. D
is wrong, because explicit dtor invocation is allowed, and sometimes
even useful (usually in conjunction with placement new).

Yet another question asks:

-> Base class members are made public for a derived class and private
-> for rest of the program by:

Well, there simply is no such thing -- presumably they're trying to
refer to "protected", but protected doesn't make anything public or
private. Rather, it makes the it accessible or inaccessible.

I won't belabor the point, though this certainly isn't a complete list
of even the obvious errors. The bottom line is that while I've seen
worse, this test is still sadly inaccurate.

I'll admit that writing a good test (i.e. selecting questions whose
answers really _mean_ something) is difficult. If you settle for
testing stupid things, writing the test should be easy, but they've
managed to get even that wrong! Oh well, I guess I'll get off my
soapbox and return you to your regularly scheduled flaming...
 
M

Matt

Thanks all for the feedback. I'm specifically looking for
alternatives to expertrating.com; can anyone offer suggestions?

Here's my current list:

http://brainbench.com
http://teckcheck.com
http://proveit.com
http://qwiz.com
http://expertrating.com

To follow up previous points:

We do not plan to charge candidates for the tests; the fact that
expertrating.com does so is unfortunate. I would look for a way for
them to charge me instead when administering the test, or reimburse
the candidate.

We are a startup company, and I play the role of software-development
manager. I have heavy development and systems-engineering background,
as well as requirements development, marketing, program-management,
etc. Unfortunately, my immediately-previous jobs were lean on the
software and heavy on the marketing/training, and my swdev skills are
quite rusty, including my C++ knowledge, tools, and syntax. I think
my requirements and design skills are quite good. I'm looking for
someone to fill a "C++ language-and-usage expert" hole to start. More
positions to fill later as we get up to speed.

We have developed other test material to cover general logic and other
swdev skills. Now I want material to cover C++ specifically. I like
an in-room test as well, but I also want something I can administer
prior to a face-to-face interview. To reiterate, I'm also looking for
a C++ language expert, for while I will hopefully become something
akin to that in the coming months, I don't have to have to page
through manuals to learn everything in the STL by myself, etc etc.

Thanks for any help,
-Matt
 
A

Andrew Koenig

Thanks all for the feedback. I'm specifically looking for
alternatives to expertrating.com; can anyone offer suggestions?

My concern with mechanized tests of any kind is that they don't really tell
you much beyond how good the candidate is at taking tests. Moreover, the
knowledge contained in such tests is often irrelevant to actual job
performance.

Another problem with such tests is that the alternatives offered often
exclude the correct answer, which is: "I don't know, because I'd never write
code like that, and if I ever encountered it, I know where to look it up."

If I were interviewing someone for a programmer job, I think I would start
with three questions:

1) Tell me about a piece of technical work you did that you're
particularly proud of, and explain what you did that makes you proud.

2) Tell me about a particularly bad design decision you made, and
explain what you learned from it.

3) Make up an interview question, answer it, and explain why you think
it's a good question.

My guess is that if someone's trying to snow you with inadequate
qualifications, you'll know it for sure before the candidate is done with
the first question.
 
G

Gianni Mariani

Matt said:
Thanks all for the feedback. I'm specifically looking for
alternatives to expertrating.com; can anyone offer suggestions?

Here's my current list:

http://brainbench.com
http://teckcheck.com
http://proveit.com
http://qwiz.com
http://expertrating.com

To follow up previous points:

We do not plan to charge candidates for the tests; the fact that
expertrating.com does so is unfortunate. I would look for a way for
them to charge me instead when administering the test, or reimburse
the candidate.

We are a startup company, and I play the role of software-development
manager. I have heavy development and systems-engineering background,
as well as requirements development, marketing, program-management,
etc. Unfortunately, my immediately-previous jobs were lean on the
software and heavy on the marketing/training, and my swdev skills are
quite rusty, including my C++ knowledge, tools, and syntax. I think
my requirements and design skills are quite good. I'm looking for
someone to fill a "C++ language-and-usage expert" hole to start. More
positions to fill later as we get up to speed.

We have developed other test material to cover general logic and other
swdev skills. Now I want material to cover C++ specifically. I like
an in-room test as well, but I also want something I can administer
prior to a face-to-face interview. To reiterate, I'm also looking for
a C++ language expert, for while I will hopefully become something
akin to that in the coming months,

I remember once thinking that. C++ is a huge language. A few months
won't cut it. I still run into grey areas and I also find issues where
the standard itself (or implementations) are ambiguous.

There are multiple paradigms for programming in C++. You can find
yourself with a very compentant language guy with all the wrong experience.

C++ as it stands is just a tool. You need to have much more knowledge
about how you would layer on top of C++ for your particular problem. I
suggest that you can find 5 very qualified candidates with 5 very
different opinions. How are you going to choose the right one ?

I don't have to have to page
 
M

Matt

To reiterate:

A C++ "language" test is not the only test and/or interviewing
process/stage/gate we are using. It's simply one among many.

Questions like these:
1) Tell me about a piece of technical work you did that you're
particularly proud of, and explain what you did that makes you proud.

2) Tell me about a particularly bad design decision you made, and
explain what you learned from it.

3) Make up an interview question, answer it, and explain why you think
it's a good question.

are good ones and are already part of an existing, pre-interview
qualification test that we are currently using. Further, we probe
much more background during phone and face-to-face interviews (eg,
things that cover more of the nature of our business and technology
focus).

Now I want more nitty-gritty C++ questions to gauge their true C++
experience and capability. I don't want to hire what seems to be a
capable candidate and then find out I can run C++ circles around said
candidate. Please note: this will be one data point among several to
be considered for application qualition; it will be far from the only
thing.

This still seems to me like a valid endeavor to me; maybe I'll learn
otherwise in the future. Until then, I'd be interested to know if
anyone considers *any* online tests helpful for *any* reason. What
I'm hearing thus far is: "you should consider using other
interviewing techniques instead." From what I can tell, we are using
some of these techniques already...and I'd like to investigate
more-technical test administration in some fashion or another, whether
I have to write these things or not. I've taken them before as an
applicant, and I'm very glad said employers issue them; I felt these
past tests that I've taken helped separate the "wheat from the chaff"
rather quickly.

Regards,
-Matt
 
L

Laurence Finston

I'm a hiring C++ developer employer looking for existing, online C++
aptitude tests.

I'm a programmer (C, C++, and LISP) looking for work, and I refuse to take
tests. After my last job I took the time to write a package and
publish it. I use this to demonstrate my ability to program
in C++. (No one seems to care about C anymore, except for embedded
applications, and nobody seems to care about LISP anymore at all, more's
the pity.)

The problem is, there are things that are easy to test, but have no value
for actual programming in real life. For example, I haven't memorized the
rules governing the priority of operators. I know some rules just by
using operators all the time, but the compiler doesn't care if I put in
parentheses if I'm unsure. It also makes my code more readable for people
who haven't memorized the rules, either.

A couple of tests potential employers gave me for C involved
expansion of preprocessor macros.
Using inscrutable preprocessor macro constructions in programs is about
the most stupid programming technique I can think of. But it's easy to
test. It's also an easy way to make someone fall on his or her face,
which is what happened.

I don't carry the language definitions of C and C++ or the calling
conventions of all of the functions in the STL around in my
head. I don't think it would make me a better programmer if I did.
I know the things I use often and I can look the others up.
If you ask me about something I don't use a lot, it's quite possible I
won't know the answer.

I applied for a job recently and was asked to learn how to use that
company's software and write a short program for some purpose! I think
I can use my time better by applying to other companies. By the same
token, I never fill out companies' "special on-line application forms".
I hope I find a company that's willing to take the trouble to
evaluate my work instead of making a decision based on a somewhat
arbitrary test. And any test will be somewhat arbitrary.

Laurence Finston
 
G

Gernot Frisch

-> a. You can use a goto statement to jump into a __try statement
block
-> or a __finally statement block
-> b. You can nest an exception handler or termination handler
inside a
-> __finally block
-> c. You can use a goto statement to jump out of a __try statement
-> block or a __finally statement block
-> d. All of the above are false

This was supposed to be a test about C++, wasn't it? Where did this
nonsense about __try and __finally come from? WRT to C++, even D
isn't
really correct, because none of them really qualifies as truly true
or
false, but merely irrelevant. Even if the question were rephrased to
deal with C++, it's still downright stupid anyway! Given a limited
number of questions, which subject should be included: goto or (for
example) partial specialization?

I don't understand this:

can you do:

goto a;
__try
{

}
__catch(...)
{
a:
goto c;
}
__finally
{
__try
{
}
__catch(...)
{
}
goto c;
}

c:
;


-> class A{
-> public:
-> A() {}
-> ~A(){ cout << "in destructor" << endl;}
-> };
->
-> void main(){
-> A a;
-> a.~A();
-> }
I suppose they probably want C -- but they're wrong. The correct
answer
is that the code has undefined behavior.

Why? Is it not allowed to call the d'tor explicitly for an object not
created with 'new'?


But I agree, stupid tests questioning C++ do not help getting you a
good employee. Main intererst should be: how does he/she approach new
problems, what's the coding style, what if something does not work?

I'd let them hand in a nice 100 line source project and explain what
it does. From a code listing of someone you can get a very good
implression of what he/she's like.

Maybe you'd better make sure you're watching while the code get's
written, so there's no danger of cheating.

-Gernot
 
J

Joseph Turian

Why do you need an online aptitude test?

Couldn't you design some interview questions that are a much better at
detecting the skillset you are looking for in an employee?
 
K

KCs

I think that it isn't very fortunate to hire programmers testing their
knowledge of a specific language. A better approach is to test their
algorithmic skills, their thinking. A language is just a tool and it's
usage can be learned quite fast.
 
F

Francis Glassborow

-> Consider the sample code given below and answer the question:
->
-> class A{
-> public:
-> A() {}
-> ~A(){ cout << "in destructor" << endl;}
-> };
->
-> void main(){
-> A a;
-> a.~A();
-> }
->
-> How many times will the destructor be called in the above code?
-> a. 0
-> b. 1
-> c. 2
-> d. There is an error because destructors cannot be called directly


I suppose they probably want C -- but they're wrong. The correct answer
is that the code has undefined behavior.

It has undefined behaviour before it tries to call a dtor (it has the
wrong return type for main(). This kind of thing says far more about the
testers than it does about the tested. If the examples you give are in
anyway typical I would want my money back because they claim that every
question has at least one correct answer, that claim is demonstrably
false.
 
A

Andrew Peter Marlow

I'm a hiring C++ developer employer looking for existing, online C++
aptitude tests.

brainBench seems quite good to me.
It draws from a very large database, making it harder
for agents to cheat, and although it is multiple-choice
the choices have subtle differences. Also the responses
have a time limit.
 
C

Carlos Moreno

Jerry said:
-> Question:
-> State which of the following are true:
->
-> a. You can use a goto statement to jump into a __try statement block
-> or a __finally statement block
-> b. You can nest an exception handler or termination handler inside a
-> __finally block
-> c. You can use a goto statement to jump out of a __try statement
-> block or a __finally statement block
-> d. All of the above are false

This was supposed to be a test about C++, wasn't it? Where did this
nonsense about __try and __finally come from? WRT to C++, even D isn't
really correct, because none of them really qualifies as truly true or
false, but merely irrelevant.

Actually, d) is correct, IMHO -- If there is no such thing
as a __try statement block in C++, then a statement that
says you can do whatever thing into a __try statement block
is false: it is false because there is no such thing as
a __try block, and therefore NO, you can NOT use a goto to
jump into a __try block.

(same reasoning applies to the other two)

The sad reality is, I would bet one year's salary that
according to them, d) is NOT the correct answer... :-(
Even if the question were rephrased to
deal with C++, it's still downright stupid anyway! Given a limited
number of questions, which subject should be included: goto or (for
example) partial specialization?

150% agree with that!
-> void main(){
-> A a;
-> a.~A();
-> }
->
-> How many times will the destructor be called in the above code?
-> a. 0
-> b. 1
-> c. 2
-> d. There is an error because destructors cannot be called directly


I suppose they probably want C -- but they're wrong.

Not to mention that beautiful void main()...

Cheers,

Carlos
 
H

Hyman Rosen

Gernot said:
I don't understand this: can you do:
goto a; __try {...

C++ does not contain constructs named __try, __catch, or __finally,
so it makes no sense for a claimed C++ test to ask about them.
Why? Is it not allowed to call the d'tor explicitly for an object not
created with 'new'?

It is undefined behavior to cause a d'tor to be called more than once
on an object. A program which executes undefined behavior, as this one
does, has no required behavior according to the standard. Therefore none
of the choices is correct.
 
M

Maciej Sobczak

Hi,
I'm curious if anyone can offer any other recommendations?

I may be biased, but this served me well (I'm on the other side):

http://www.brainbench.com/

(click around patiently to dig out the C++ test)


As for the C++ test itself, it is rather focused on syntax issues
instead of idioms and techniques, so don't rely on a *single* test to
select your candidates. If you can combine such tests with *other* ways
of assessing candidates, then those tests can certainly help you.
 
I

Ivan Vecerina

Matt said:
I'm a hiring C++ developer employer looking for existing, online C++
aptitude tests.

I have not yet extensively researched this yet, but as an example, I
thought this test looked pretty good:

http://expertrating.com/c++test.asp

I've never tried this one. An alternative I am familiar with is
available at www.brainbench.com.
I'm curious if anyone can offer any other recommendations? If so,
could you please offer your experience with your recommended resource
and why you recommend them...or might recommend others?

I cannot rate the expertrating test, so I can't compare. Personally,
I'd probably not be fully satisfied with the coverage or quality
of either test.

I agree, but keep in mind that these multiple-choice online tests
have a limited scope. They can test for knowledge of the language,
but not coding quality, proficiency, etc. I'd say that being able
to pass the test can be a necessary condition (i.e. for screening),
but not sufficient (to my hiring standards).

As part of the interview process, I will go further and give an actual
coding assignment. A company I worked for had a standardized assignments,
asking candidates to maintain and evolve an initial program through
multiple iterations of requirements. When properly reviewed by a
competent senior developer, I find that the result of such a test will
give you much better insight -- and feel less deprecatory to the candidate.

I hope this helps,
Ivan
--
http://ivan.vecerina.com/contact/?subject=NG_POST <- email contact form
Brainbench MVP for C++ <> http://www.brainbench.com
Full disclosure: I'm not affiliated with Brainbench other than for
having free access to their tests and online materials in exchange of
reviewing some of their test questions and posting this occasional link.
 
G

Gary Labowitz

Matt said:
To reiterate:

A C++ "language" test is not the only test and/or interviewing
process/stage/gate we are using. It's simply one among many.
<<big snip>>
As I've said here before, you are opening yourself up to lawsuit from an
alert candidate. If the candidate can show that he/she did well on the
written exam and that the exam is not predictive of performance, not hiring
that candidate would (allegedly) show a bias in not hiring. The argument
goes that if you use a test and the candidate passes that test, then if you
don't hire it's because of some other determination. Why bother using the
test?

On the other hand, if the candidate fails the test (whatever that means) you
have to prove that everyone who works there can pass the test, and the test
asks questions that are predictive of successful work at your place. This is
very hard to do. You might start by seeing if you can pass the test.

What can be done is to administer testing after hiring to determine
experience and/or training that might need to be given. This should be
presented before hiring as a condition of employment, however.

Besides which, and more to the point, your questions are (so far) flawed
and/or meaningless. Predictive exams are hard to come by, and most on-line
quizes are pinpoint-limited to a very narrow range of syntax and language
topices --- hardly predictive of performance. I'd recommend you give up the
idea of a test and stick with (admitedly) subjective determination, stated
this way: the candidate appears to have the experience based on resume and
references and an attitude which fits our way of operating. A trial
employment will determine if the candidate can do the job within the normal
range of expertise we require. Then you conditionally hire for 30 days.

Sorry to sound so negative, but I've been down this road for almost 44
years. (My first experience was with the Programmer's Apptitude Test given
by IBM, in 1961). It still bothers me.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,769
Messages
2,569,580
Members
45,055
Latest member
SlimSparkKetoACVReview

Latest Threads

Top