Excellent Pay, Interesting Work: So Why Is There a Severe Data Scientist Shortage?

Um, misperceptions which potential employers have utterly failed to disperse, that it's boring, socially meaningless back-office toil for grunts and grinds?  JL

Justin Gage comments in Venture Beat:

McKinsey says that by 2018, the demand for data scientists will outpace supply by 60%. Accenture noted that 90% of its clients were looking for data talent. And the median starting salary for a data scientist can be double that of a programmer. Big data can be part of the big, world-shaking ideas that students want. If you’re a company looking to generate interest in data science, emphasize what potential societal impacts can have as data scientists.
In the sage words of Marc Andreessen, “Data is eating the world.” Er, something like that. Big data has been and continues to be one of the hottest areas in tech, constantly covered in the news. According to CB Insights, it’s also an exciting area for venture funding, with almost $8 billion in funding last year, a number that decreased recently but had been steadily growing since 2011 (around 20% year over year). Yet despite the popularity, most companies seem to agree on one, major problem: There aren’t enough people who want to work with data for a living.
The statistics (perhaps ironically) are pretty convincing. Summarized in an article at Datanami, McKinsey says that by 2018, the demand for data scientists will outpace supply by 60%. Accenture noted that 90% of its clients were looking for data talent, and 40% cited a lack of it as a major problem. And to top it off, Glassdoor found that the median starting salary for a data scientist can be almost double that of a programmer. Everybody’s looking to hire and pay (well) for data people, but they can’t seem to find them.
I’m a data science student at NYU’s Business School, so I see and talk to fellow students all the time about the careers they’re interested in, and unsurprisingly, data science is never at the top. Along those lines, and considering what drove me to choose data science as my major, I think that there are two key issues holding students back from being interested. If you’re a business looking to poach data talent straight out of college, you should read this.

Big data can be a societal and economic good

College students think big – they’re passionate about and interested in big ideas that have the potential to change the world and make people’s lives better. It’s why we’re so involved in and excited about politics. Programming is exciting – companies like Google and Facebook (=programmers) are doing amazing things, like Google’s Loon and Facebook’s Internet. But data sounds boring, doesn’t it? Facebook talks about how its engineers are programming and creating the products of tomorrow; its data science careers page is … less exciting.
Fortunately, big data does have the potential to change the world and make people’s lives better. For a few ideas, check out Harvard’s page on the topic. An example is using data science to increase crop yields, and eventually prevent food shortages. Big data is powerful and can be part of the big, world-shaking ideas that college students want. If you’re a company looking to generate interest in data science at schools, emphasize what potential societal and social impacts students can have as data scientists at your company.

Big data is intuitive and clear

Data science, at least on the surface, can be pretty cloudy, and most people (and students) don’t even know what it means to be a data scientist. This was once a challenge for software engineering – but companies like Codecademy and CodeMonkey, along with initiatives like Scratch, have been part of a movement that democratized coding and made it simpler and more accessible, at least at the beginner level. An easy code editor in Codecademy has shown students (plenty of my friends use it) what coding is, how you generally do it, and why it’s exciting. Bottom line: if you’re a college student, chances are you know what programming is, and you might have even tried it a little bit.
That’s not true for data science. Students generally have heard of the term but don’t know what a data scientist does, or the tools he/she would use at the job. The bulk of basic data science work – numpy, matplotlib, and pandas, if you’re using Python – isn’t easy, but it’s often intuitive and powerful. Students need a clear walkthrough and framework for what data science is, how they do it, and what cool things they can find and create with it. Something interactive like Codecademy for a relevant data set, like world population or GDP, could help get students interested.
I think that these are the core issues, but they aren’t the only solutions offered. Amy Gershkoff believes that more formal education and frameworks are needed. But I’m taking a different route – I think it’s more about the big picture potential and practical hands-on experience. These two topics – the potential positive impact of data science and the cloudiness surrounding it – definitely overlap. But solving them could be the breakthrough needed to get students interested and majoring in the field. Let’s get to it.