Sunday, March 4, 2018

Why is it so hard to hire software engineers?

As an Engineering Manager at one of my previous employs, we would submit postings looking for something called a 'full-stack employee'. Ideally, this would be someone with expertise in exactly the stack that we had, who has worked in the entire stack before, enough to get proficiency at every single level. For a Microsoft stack, this would be something like .NET, SQL Server, Windows 10, Linux = LAMP (showing my age a bit), or more recently MAMP.

On it's face, it seems impossible. It's highly unlikely that we'd ever find someone, aside from a previous employee, perhaps, who matches technology-for-technology. I've given and received many interviews and can safely say that this requirement is one of the first things to go (unless its heavily prescribed from the top-down, which can be a bit of a problem). More recent coding interview trends use algorithmic knowledge as a metric, or solving pet problems in real time (which I argue is probably less a test of individual programming skill, and more a test of ability to memorize and dedication to training).

If we do it by the numbers, let's say n is the number of layers in your stack. Then 2^n is the combination of potential experiences with different technologies of your stack (assuming 1 technology per layer - an invalid assumption). You want exactly 1 combination of these 2^n possibilities when you want a 'full-stack developer' in your stack. Traditional 3-tier architecture means 1:8 chance of meeting your requirement. Add to that stacks, queues, message queues, batches and so on, and it's not unreasonable to get to the 1:32767 (1:2^7 - and again showing my age - can you guess how?).

If we loosen up on the specific technology requirement (and say, want someone who has *any* experience building an API, UX, and DB), then we can stay at 1:8. The numbers don't lie - you could assume that 1:8 candidates is what you're looking for here too. What I've seen lately is a spike in the number of UX developers, and a dwindling in the number of API developers. That's driven me to the totally-anecdotal analysis that API developers are harder to find than UX developers generally. Just a thought.

Suppose your candidate satisfies that test. You still don't know if the individual can code, or considers system design when they do. Now you've got to tease that out. Hence, the super-hateful 2 hour coding-and-design session that nobody wants to administer, and nobody wants to experience. Somehow, we've come to the conclusion that at the end of the day, there's nothing that proves someone knows how to work on enterprise-level scalable technology than a whiteboard and a toy problem. What could go wrong?

Finally, assuming *that* works, then you've still got to figure out if the individual is a good organizational fit, or is one of those 'brogrammers' who you really don't want influencing your corporate culture. Sorry - it's a problem. Get past that, and finally you have a potential candidate. If you're counting, you're now down to 1:64 of your candidates (roughly completely made up estimate).

All of us engineers know that after that, we're supposed to hire the 3rd, best candidate (Oxford comma, beware). No problem - we'll just wade through the next 128 candidates...

I don't see a good way out of a lengthy exam of some sort, but I would suggest an annual bar exam, kind of like what lawyers have to do to practice law. This would give engineering HR a known baseline (which we kind of use the BS in Science as right now). The difference between this and a BS would be that an engineering bar can change over time as technology does. It would include a technology-proficiency part, and a data-structures and algorithm component. With that in place, organizations can focus more on corporate culture and recent work experience,instead of lengthy, compressed, coding and design tests.