How Come Good Engineering Managers Aren't Just Hard to Find, They Don't Seem to Exist?

Bigfoot. The Loch Ness Monster. Good engineering managers, especially in Silicon Valley. Why are they all reportedly so hard to find?

For the sake of brevity - and sanity - we will ignore the first two. But the issue of finding 'the right' employees is one that plagues the economy and around which certain myths appear to have grown, some due to ideological predispositions and some because organizations become wedded to the wrong management processes, compensation schemes and personal incentive structures.

We can start with the premise that there are a fair number of pretty good engineers out there. Perhaps not enough as the demand for those with technical degrees and competence increases across all industries. This number has increased as nationality, race and gender have become less of a barrier to those who dont conform to traditional stereotypes of what an engineer 'should be.' Necessity will do that to you.

The problem, as the following article explains, is that many engineers dont want to be managers - they like working with colleagues on coming up with really cool solutions to seemingly intractable problems. The hassles of writing reports and sucking up to superiors even further along the value chain is not how they want to spend their time. And the people who do want to be managers are often ill-suited to the task by dint of personality and competence.

So enterprises design elaborate schemes to induce otherwise happy and productive engineers to become managers. All too often, the managers disappoint. Themselves and their superiors. This is generally because it is not something they wanted to do in the first place, they are better suited to working on the bench, in the lab or wherever else the real work is found - and the incentives end up rewarding behavior that may make sense to the bureaucracy or to the consultants who got paid to invent it but dont produce the sorts of outcomes required by those focused on real deadlines, real products and real customers.

The answer probably lies in reinventing the notion of management from a top-down imperative to a collaborative exercise in group achievement. Lots of enterprises dont want to face the disruption such an approach to work might cause. But the successful ones - and there are plenty of them - do figure it out. Eventually. JL

Juan Pablo Dellarroquelle comments in VentureBeat:

Everyone talks about how difficult it is to hire great software engineers in the Valley. And it is. But nobody’s talking about how hard — really hard — it is to find good engineering managers
While good engineers might be famously tough to hire, at least they’re easy to find. Good engineering managers, on the other hand, are practically impossible to find.
This is a problem I’ve had to grapple with a lot lately, and I’ve spent a lot of time thinking about the cause. Why is it so difficult to find these people? If there’s a lot of demand for them by so many companies, shouldn’t the supply gradually be going up — lots of talented people wanting the job? For the longest time, I just assumed I was looking in the wrong places. But recently I’ve come to a different conclusion: They don’t exist.
Why? Well, the implicit aim of most functions in a company is to get further up the hierarchy. To have more influence. To become the “boss”. To lead people. But engineers are… different. Unlike virtually every other function in a software company, engineers — particularly the good ones – don’t want to move up. This means that the people who want the engineering manager role are unlikely to be very good at it; and those who could be good at it don’t want anything to do with it at all.
Let’s dig into this. When you think about where engineering managers can come from, you divide the pool into two.
Pool one is made up of those engineers that want to become engineering managers because they want to “move up.” The irony is, these are the people that you don’t want. They fail to understand that going from engineer to engineering manager isn’t a decision to move up — it’s a decision to move down. Becoming an engineering manager means that you now work for your engineers — not the other way around. The engineers in this pool are the ones that have never been heard. That were never the leaders inside their teams. They think management is their opportunity to make decisions. Now, they can “call the shots.”

All the wrong reasons

There are a lot of managers like this out there. How did they become managers in the first place? Well, particularly in big organizations, they did so by being organized, by keeping track of what everyone was working on, by focusing on deadlines. I interview a lot of these individuals. They tend to be weaker technically, and have a condescending way of referring to the engineers they “manage.” They wear the number of engineers in their organizations with pride: “At one point, almost 100 engineers reported up to me!”
In pool two, you have the rest of the engineers. If you’re lucky, you have good ones. Within this group hides a breed of engineers that has the potential to be great engineering managers. They tend to be very technically competent. They exert their leadership almost unconsciously through code, through conversations, and by helping others in the team and across the organization. They have a sense of ownership that is broader than their daily tasks. They cannot stand others not caring in the same way. They get pissed when they see people not trying hard — or going through the motions without understanding why they’re doing what they’re doing.
More than managers, they’re leaders.
Even though they don’t have much formal authority, engineers like this have a lot of influence. That dynamic means they have very little motivation to want to move up in an organization. Taking on the role of a manager means giving up time doing what they love — solving challenging technical problems — in exchange for what they see as taking out the trash every night. Meetings, performance reviews, managing other people’s deadlines.
So why would they want to do this? Or better yet, how do you help them realize that management can be fulfilling, and not just busywork? Good question.