blinnov’s blog

Dear Madam/Sir,

I appreciate you going into the trouble actually visiting this page.  This post represents this blog owner’s point of view on so-called “Online C++ tests” that are used by recruitment agents to select candidates and explains why its author does not take any of these tests anymore. The sole purpose of the post is to be referred to when he is asked to sit one of these to save him the time explaining his position on this issue over and over again. By writing this post, the author did not intend to insult anyone nor pay revenge of any kind to anyone.

Almost every good software developer can find themselves discussing a new exciting career™ opportunity with the potential employer or their agent. Nine from ten, the candidate will be asked to sit a test. A very easy one, should not be too much trouble, person of such experience will not have any problems completing it in no time, they are told. And this is what is terribly wrong about it. You will have problems. No matter how good you are.

I agree that some initial screening may be required. For example, company needs a junior software developer and they need to be sure that candidates understand basic concepts of certain programming language. In this case test will help. It simply must be well prepared, 100% accurate, 100% correct, 100% meaningful and also short, if not too much trouble. The problem is that most of the tests out there don’t have any of this characteristics. The bigger problem is that they are usually put together by those who have never worked in the software development. And the killer problem is that these tests are filled with inconsistencies, discrepancies or simply full of errors (I wanted to use more colorful language here, but this post is going to be a part of my CV, so I better don’t).

About a year ago I was asked to sit an online C++ test.  I had no objections then. I also was interested in the position. The first question was quite obvious. The second had 4 answers and three of them were correct, but according to the question, only one of them was supposed to be correct. I lost all the interest momentarily. I wanted to stop it right then, but I decided to continue. I don’t regret. At least, now I have the proof how bad these tests are . That’s what question number 4 was:

Anyone who knows C++ will see immediately that everything here is totally screwed. At first, Box::returnpointer() returns reference. But the question is “how to return the address”, which means pointer. I don’t know how this rubbish got into the test and I don’t really care, the fact is that the person who really knows C++ will never ever mistake reference for pointer and vice versa. The question makes no sense nor any of the answers is correct.

The next question brought me a joy of guessing what did they really had in mind. The question that came after that, was a masterpiece. Obviously, somebody simply copied and paste the code directly into HTML, so all template definitions simply disappeared.

It’s so screwed I don’t even want to explain anything. Of course, this is a technical error, but what I was supposed to do? Made a guess? Click randomly? Thank you very much, If I wanted to gamble, I’d go and play pokies.

In fact, I did a nice thing. I took those pictures, randomly clicked through the rest of questions (what was the point answering if the test was already compromised?), expectantly failing it. Then I went into the trouble of putting a big email together with those “screen shots” and detailed explanations what was wrong with the test. Surprisingly enough, I was invited to an interview, but at the end of the day it did not work out, but it is an entirely different story.

You may disagree with me – there are a lot of tests that are checked thoroughly. And you will be correct, well known Brainbench tests don’t usually have stuff-ups like above in them. But they are still stuffed, just in different way. They are still made by people who obviously don’t quite understand what software engineer job is about, and half of questions have no relations to how programming language is used in the real life. The second half of Brainbench questions are “find what’s wrong with this piece of shit code”. As a developer, I have to deal with not-so-great code on a regular basis, but what you’ll see in BB tests is beyond any reasonable limits. When commercial software is considered, such code just does not get checked in into the repository. The code review simply stops at the sight of it and the author gets a chance to get rid of it. To top it off, BB has a bad habit of giving code snippets in images. In which code is printed using proportional font. Like this:

int z= (*(pPtr))(i+m/k);

In programming, any single symbol matters. That’s why all programming IDEs use monospace fonts:

int z= (*(pPtr))(i+m/k);

The first snippet is much harder to read. Also, it is not the case of “it is just the test, it worth the effort, so stop bitching and read the bloody code”. It does not worth the effort. In the industry, only monospace fonts are used for the code. Every symbol matters, therefore all symbols must be of exactly the same width. To read the code the is printed in proportional font, you really have to look so hard it hurts.

I should also mention meaningless questions. Such as when you are asked something like “what is the standard name for basic_filebuf<char>”. No C++ guru will be able to answer this, except by accident. As it is not something you have to remember to be a C++ programmer. And chances are, you’ll never have deal with that basic_filebuf template.

There also is another type of questions that their authors tried used to evaluate something completely irrelevant. For example, some time ago I did that BB test and I was asked the question above. It was the only question (of 40) that had any C++ templates in them. At the end of the test I was told that I did not have enough competence in templates. How could they figure this out I don’t know? Trust me, I know C++ templates. My colleagues sometimes tell me I know them a little bit too well.

I could go for ages (I did not even mention virtual inheritance questions yet!), but I think I have already said enough. It’s time to summarize.

Imagine yourself in my shoes. You are not a graduate. You are a professional that uses this programming language since you can’t even remember when. You speak that language. You know its tricks. You know its best practices. And you are required to take a test, which:

• Created by people that are far less competent then you are
• Created by people that, probably, never worked in software development
• Has no relation to the real job you are doing now
• Is unrelated to what you will be doing on that job
• Is inconclusive
• Contains errors in either questions or answers. Or both
• Has bugs in implementation

All it means that you have pretty good chances of failing this test, no matter how brilliant you are. Simply because you click on the answer that you know is correct, but test thought it was incorrect.

Your time is wasted, and your application is marked “incompetent in C++”. And this mark is set by someone who doesn’t event know the difference between a reference and a pointer! And, in most cases, you have no chances to prove your innocence, because nobody listens to you.

I can’t speak up for everyone, but I don’t want to risk my reputation in that way. I don’t want to watch my experience and my knowledge being flushed down the toilet. This is why I don’t take any online tests.

If you are an agent who get here by the link in my resume, please note: if sitting such test is your requirement or your customer requires it, I will not be interested. I know my business. The software I developed works in a lot of places. It is unlikely that you will ever notice  it, but you have pretty good chances of being served by some of the systems I build. I won’t name them here (you have my resume, it is all there). I am not afraid of challenge. I love challenge. I desire challenges. That’s what I love most in my work. But the test that you are likely to offer is not a challenge. It’s a Russian roulette. And I won’t play it. Chances of losing playing it do not depend on how good I am, and I do not take such risks. I better lose this exciting opportunity this time.