+972 52-548-6969
Comparing developers
Recently I had to try to explain to a non technical person how I rate the developers that I work with. In technical terms, it is easy to do:
int Compare(devA, devB, ctx)
But it is very hard to do:
int Compare(devA, devB);
var score = Evaluate(dev);
What do I mean by that? I mean that it is pretty hard (at least for me), to give an objective measure of a developer with the absence of anyone to compare him to, but it very easy to compare two developers, but even so, only in a given context.
An objective evaluation of a developer is pretty hard, because there isn’t much that you can objectively measure. I’m sure that no reader of mine would suggest doing something like measuring lines of code, although I wish it was as easy as that.
How do you measure the effectiveness of a developer?
Well, to start with, you need to figure out the area in which you are measuring them. Trying to evaluate yours truly on his HTML5 dev skills would be… a negative experience. But in their areas of expertise, measuring the effectiveness of two people is much easier. I know that if I give a particular task to Joe, he will get it done on his own. But if I give it to Mark, it will require some guidance, but finish it much more quickly. And Scott is great at finding the root cause of a problem, but is prune to analysis paralysis unless prodded.
This came up when I tried to explain why a person spending 2 weeks on a particular problem was a reasonable thing, and that in many cases you need a… spark of inspiration for certain things to just happen.
All measurement techniques that I’m familiar with is subject to the observer effect, which means that you might get a pretty big and nasty surprise by people adapting their behavior to match the required observations.
The problem is that most of the time, development is about things like stepping one foot after the other, getting things progressively better by making numerous minor changes that has major effect. And then you have a need for brilliance. A customer with a production problem that require someone to have the entire system in their head all at once to figure out. A way to optimize a particular approach, etc.
And the nasty part is that there is very little way to actually get those sparks on inspiration. But there is usually a correlation between certain people and the number of sparks of inspiration per time period they get. And one person’s spark can lead another to the right path and then you have an avalanche of good ideas.
But I’ll talk about the results of this in another post 
.
Comments
The variance between developers is huge, sometimes you can't estimate it and not even predict. I know a developer who finish a task in 4 hours, while for other developer it took 1 week (real story).
sometimes there is a nasty bug that takes a lot of time to figure out or just to reproduce, sometimes it's easy. I think most of the brilliant developers tend to be curious, explore, learn and think of ways to solving problems.
Good insight. All developers are necessarily specialized.
That said programming has a general skill component that is not dependent on particular technologies or project types. I'm good at .NET but I could pick up Java extremely quickly and be a top-level dev in a month or so. The same is true for almost all languages and technologies that I can think of. That's because I know how to program in general. The principles are always the same.
"The principles are always the same." of course, rest, c#, java, javascript, c++, aseembly, sql is the same it's just a matter of syntax ! And what is syntax if not just letter, and we all know all the god damn letters ! Never say this in an interview, never.
It is exactly not about the letters and the syntax. That is an unimportant detail that can be learnt in a week.
It's about structuring programs. Indeed, c#, java, javascript, c++ are very similar and very easy to cross learn. The same program structuring principles apply. In C# you write a class, in JavaScript the same concept exists but it must be emulated. Structurally it is the same.
SOLID, for instance, applies everywhere. Immutability as a principle is useful in many contexts.
The language is just a detail. If you know all programming languages in the world you still can't program.
"The language is just a detail." this is just wrong, you might be able to do a 2k lines program in whatever language you choose, but a language comes with frameworks, environments, operating systems, requirements, implementation details, bugs, culture, documentation, performance ... and we all know that devil is in the detail, and the easier you fight the devil the best you are.
Tobi, There is a big difference between being able to write C# in JS and being able to write idomatic JS.
True. It doesn't take long to learn, though, if you take an active learning stance. If you slave away for 30 days and type in whatever seems right then you will not pick up an idiomatic style. That's too short of a period of time. But if you spend a few days reading books, blogs and open source code you pick up all those little things quickly. For example it might not occur a newbie to use
<hr>usingbut the web is littered with advice about it and he'll see that eventually.Both of you made true statements but they do not detract from the fact that being a productive programmer is little about the language and other misc stuff. That's good to have and certainly required. But the real productivity jump comes from being able to structure code and run bigger projects. A newbie can't do that and he can't learn it in 30 days.
Although I do agree that most of programming skills do not depend on the programming language (it's also convenient to think so), reaching excellence in a specific programming language takes time. And the examples that you find in books and internet are usually simplified to the point that it doesn't follow any good practices, for the sake of simplicity. It takes time (and probably multiple projects) to start to "feel" the language, let alone the frameworks that go with it.
I've seen terrible code that was written by people with years of experience in one language/platform and then switching to another one. Ever seen subscribing to C# events using reflection, instead of using "+=" operator? God knows how and where they have picked this up, but I've seen many examples like that.
Everything depends on what kind of developer you need. Some developers are great consultants and have good client facing skills, which is very valuable when you do projects for customers. Other are not so good at talking to people, but can handle complex implementation tasks. So there's no single best method for selecting best candidates and you probably need several different species of developers to form a good team. But any programmer imho should be able to actually write code and possess some basic knowledge that's taught at computer science courses - too many people have no idea what they're doing and can be elliminated by practical programming tests and general algorithms/data structures questions.
Evaluating developers has been on my mind for a while now, as I transitioned from 95% developer/5% manager to 95% manager/5% developer. While I couldn't measure all of the aspects, we created a skill progression matrix for the ones that did seem more measurable. We have 3 areas: technical skills, depth of knowledge in our application, and general development skills (troubleshooting, support, staying up to date with the field, teamwork, etc). We define what we think entry level looks for each skill, then define what we think the next level is from there, and so on until we get one level past whoever is best at it on the team (when someone is at the top, they should have some idea on how they would become even better). The intent is to find the people that are the best ones at any point in time to do training or code/design reviews, on one side, and for the individual devs (including me) to be able to see progression in those skills as they learn more on the job and see a path on how to improve further.
It doesn't cover everything. There isn't a box for the person that regularly has that extra intuitive spark that gets the ticket closed faster. We didn't quantify how quickly people could pick up new programming languages. It meets some of our goals for it, but I also look forward to Oren's next post :)
That being said, I disagree that there are two different types of developers. Even introverted developers can enjoy talking to customers. There are definitely a number of developers that really dislike it or need to learn a lot more humility before I would put them in front of a customer, but I've found there are a large number of developers that don't like speaking to customers simply because they haven't been allowed to do so, are unfamiliar and nervous about it, and find it easier to stick to areas they are already strong in. I hire smart developers that can talk to customers (we screen and interview for it).
You're absolutely right. It's all about context. A good developer understands the needs of the team to deliver quality software and doesn't waste a lot of time doing low value activities, within context.
Tobi, you say: "The same is true for almost all languages and technologies that I can think of. That's because I know how to program in general. The principles are always the same."
I think "I can think of" is actually the most important part. If you really can't think of anything else, try to program e.g. in Haskell, Prolog und Lisp. If, after that, you still think the same, then you are either a genius or a goat. ;)
Comparing would be so easy if done subjectively, or by looking at some numbers at the end of the month, but a developer's work is not just numbers. Numbers can help for sure, but I guess that in the end some subjectivity will make its way in determining who...you know!
Comment preview