STEM (Science, Technology, Engineering and Mathematics) shortage claims are claims that there is a current or projected shortage of STEM workers in the United States and sometimes worldwide. These claims are promoted by large employers of STEM workers in private industry, academia, and the government. In the last few years the claims tend to be focused on a particular subset of STEM workers: programmers, software engineers, and other “technology” workers, where “technology” is implicitly equated with “computer technology.”
A high profile example of these claims can be found in venture capitalist Marc Andreesen’s widely cited Wall Street Journal article “Why Software is Eating the World” (August 20, 2011):
Secondly, many people in the U.S. and around the world lack the education and skills required to participate in the great new companies coming out of the software revolution. This is a tragedy since every company I work with is absolutely starved for talent. Qualified software engineers, managers, marketers and salespeople in Silicon Valley can rack up dozens of high-paying, high-upside job offers any time they want, while national unemployment and underemployment is sky high. This problem is even worse than it looks because many workers in existing industries will be stranded on the wrong side of software-based disruption and may never be able to work in their fields again. There’s no way through this problem other than education, and we have a long way to go.
(Emphasis added)
Andreesen is far from an isolated instance of these claims. For example, in his testimony to the House Committee on Science and Technology in 2008, former Microsoft CEO Bill Gates claimed:
I know we all want the U.S. to continue to be the world’s center for innovation. But our position is at risk. There are many reasons for this but two stand out. First, U.S. companies face a severe shortfall of scientists and engineers with expertise to develop the next generation of breakthroughs. Second, we don’t invest enough as a nation in the basic research needed to drive long-term innovation.
(Emphasis added)
Ironically, Microsoft, a highly profitable company, announced several thousand layoffs of its highly qualified and presumably difficult to replace employees a few months later. Both Bill Gates and Microsoft have been prominent in claiming shortages of qualified technology workers since 2009 even as Microsoft has announced a series of major layoffs of presumably highly qualified technology workers. Microsoft announced another round of about 3,000 layoffs a few weeks ago (July 2017).
Rising Above the Gathering Storm
In 2005, the COMMITTEE ON PROSPERING IN THE GLOBAL ECONOMY OF THE 21ST CENTURY produced a widely cited report “Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future” under the auspices of the National Research Council (NRC) promoting similar claims. The committee members were:
NORMAN R. AUGUSTINE (Chair), Retired Chairman and CEO, Lockheed Martin Corporation, Bethesda, MD
CRAIG R. BARRETT, Chairman of the Board, Intel Corporation, Chandler, AZ
GAIL CASSELL, Vice President, Scientific Affairs, and Distinguished Lilly Research Scholar for Infectious Diseases, Eli Lilly and Company, Indianapolis, IN
STEVEN CHU, Director, E. O. Lawrence Berkeley National Laboratory, Berkeley, CA
ROBERT M. GATES, President, Texas A&M University, College Station, TX
NANCY S. GRASMICK, Maryland State Superintendent of Schools, Baltimore, MD
CHARLES O. HOLLIDAY, JR., Chairman of the Board and CEO, DuPont Company, Wilmington, DE
SHIRLEY ANN JACKSON, President, Rensselaer Polytechnic Institute, Troy, NY
ANITA K. JONES, Lawrence R. Quarles Professor of Engineering and Applied Science, University of Virginia, Charlottesville, VA
JOSHUA LEDERBERG, Sackler Foundation Scholar, Rockefeller University, New York, NY
RICHARD LEVIN, President, Yale University, New Haven, CT C. D.
(DAN) MOTE, JR., President, University of Maryland, College Park, MD
CHERRY MURRAY, Deputy Director for Science and Technology, Lawrence Livermore National Laboratory, Livermore, CA
PETER O’DONNELL, JR., President, O’Donnell Foundation, Dallas, TX
LEE R. RAYMOND, Chairman and CEO, Exxon Mobil Corporation, Irving, TX
ROBERT C. RICHARDSON, F. R. Newman Professor of Physics and Vice Provost for Research, Cornell University, Ithaca, NY
P. ROY VAGELOS, Retired Chairman and CEO, Merck, Whitehouse Station, NJ
CHARLES M. VEST, President Emeritus, Massachusetts Institute of Technology, Cambridge, MA
GEORGE M. WHITESIDES, Woodford L. & Ann A. Flowers University Professor, Harvard University, Cambridge, MA
RICHARD N. ZARE, Marguerite Blake Wilbur Professor in Natural Science, Stanford University, Stanford, CA
Nearly all of the committee members were current or former top executives, frequently the CEO, of major employers of STEM workers, public and private.
The committee followed up with another report in 2010 “Rising Above the Gathering Storm, Revisited: Rapidly Approaching Category 5.” Category 5 is a reference to the Saffir-Simpson hurricane wind scale in which the highest classification Category 5 is reserved for extreme storms with winds exceeding 156 miles per hour.
Rising Above the Gathering Storm, like most reports of this type (there are many), called for more STEM teachers, more STEM students, more visas for STEM worker immigrants and guest workers strongly implying a major shortage of STEM workers in the United States.
Ironically the report starts with a claim that appears grossly inconsistent with this, a quote from Nobel Laureate Julius Axelrod (Rising Above the Gathering Storm, Preface, Page ix):
Ninety-nine percent of the discoveries are made by one percent of the scientists.
Julius Axelrod, Nobel Laureate
It is manifestly unclear why more scientists and more funding for science is needed if ninety-nine percent accomplish almost nothing. Why indeed not eliminate the nearly useless 99 percent and the 99 percent of funding that they consume? Federal R&D funding is over $100 billion per year. Why not free up over $99 billion to fund other more productive activities? 🙂
STEM shortage claims have a long history
STEM shortage claims predate the acronym STEM by many decades. STEM shortage claims date back at least to the early days of the Cold War, when much of the focus was on physics and physicists. Then, as now, the STEM shortage claims often involve an alleged existential threat to the nation.
Professor David Kaiser of MIT, a physicist turned historian of science, has written a number of articles and given a number of presentations on the Cold War physics and STEM claims, notably “TOIL, TROUBLE, AND THE COLD WAR BUBBLE: PHYSICS AND THE ACADEMY SINCE WORLD WAR II” at the Perimeter Institute in 2008.
In recent years, the STEM shortage claims tend to focus on computer science and software engineering rather than physics, although claims of this type are common for almost all forms of STEM work.
STEM shortage claims have many highly qualified critics
The claims have been questioned and challenged by a large number of academics, journalists and others for many years including Michael S. Teitelbaum (Senior Research Associate at the Labor and Worklife Program at Harvard Law School), Norman Matloff (Professor of Computer Science at UC Davis), Peter Cappelli (George W. Taylor Professor of Management, Wharton Business School, University of Pennsylvania), Paula Stephan (Professor of Economics at Georgia State University), Ron Hira (Associate Professor, Howard University), Patrick Thibodeau (a Senior Editor at Computerworld), Robert N. Charette of the IEEE and author of “The STEM Crisis is a Myth,” and many others.
I have written many critical articles on the claims including “STEM Shortages, Purple Squirrels, and Leprechauns,” “STEM Shortage Claims and Facebook’s $19 Billion Acquisition of WhatsApp”, “The Corinthian Colleges Scandal, STEM Shortage Claims, and Minorities.” The last includes a lengthy discussion of Microsoft’s numerous layoffs in the comments section. UPDATE (May 16, 2018) Also see my article: “Microsoft Layoffs and STEM Shortage Claims (2008-2017)”.
STEM shortage claims are closely connected to, although logically separate from, calls for increased immigration and guest worker visas such as the controversial H1-B visa. The claims are also closely connected to, though again logically separate from, claims that education in the United States is poor both in absolute terms and compared to other nations such as Finland and calls for “school reform” often promoted by extremely wealthy individuals such as former Microsoft CEO Bill Gates, Facebook CEO Mark Zuckerberg, and others.
STEM shortage claims are confusing
STEM shortage claims are surprisingly difficult to pin down. The crux of the issue is what exactly constitutes a qualified STEM worker (software engineer, scientist,…)?
Many claims seem to imply a shortage of STEM workers with critical basic skills taught at the K-12 level such as basic arithmetic, algebra, AP Calculus, basic programming skills taught in AP Computer Science and other introductory CS courses (or for that matter learned programming a game on your laptop in Python or Java, a popular activity among STEM students who never take AP Computer Science).
Obviously, the tens of thousands of highly qualified engineers and other STEM workers laid off by Microsoft since 2008 have these K-12 skills in spades. Indeed many of the highly qualified engineers laid off by Microsoft were from Finland which has consistently scored top or near the top in the international comparisons of K-12 skills frequently cited in STEM shortage claims. So apparently this is not the STEM shortage referred to by Microsoft and Bill Gates.
Similarly, many older — over thirty-five, even over thirty sometimes — software engineers and other STEM workers report surprising difficulties finding jobs, a fair number leaving the STEM fields every year. Again there is little question these candidates have the K-12 level STEM skills and much more.
When pressed about these obvious inconsistencies, spokesmen for STEM employers will generally begin to claim they mean a shortage of very specific skills such as years of paid experience developing first person shooter apps for the iPhone (IOS) in Objective C (C++ on Android won’t cut it!) and often that they mean a shortage of the very best STEM workers — along the lines of the elite one percent in the Axelrod quote from the Rising Above the Gathering Storm report above. Often, years of specialized experience in narrowly defined skills and being the very best are implicitly conflated in these revised STEM shortage claims.
What do the STEM employers really want?
Yet, do the employers actually want either the candidates with years of specialized experience or the very best or both as they claim? There are some high profile rejections of candidates who would seem to meet these criteria such as Facebook’s infamous turndown of Brian Acton who went on to found WhatsApp which Facebook then acquired for $19 billion.
In recent years, many employers are noted for quizzing candidates about introductory data structures and algorithms taught in college CS courses rather than advanced specific skills learned on the job. This has spawned a large number of interview practice books, courses and programs such as Gayle Laakmann McDowell’s Cracking the Coding Interview.
It is difficult to see how these introductory questions would reliably identify the specialized skills learned on the job such as iPhone app programming that are often listed in job descriptions and cited in defenses of STEM hiring practices.
UPDATE April 30, 2021: These algorithmic coding interviews test knowledge of the implementation of introductory data structures and algorithms that those of us who develop advanced algorithms such as video compression, speech recognition, touch processing, or data analysis rarely implement in the real world. Most modern programming and languages such as C++, Python, R, MATLAB, and many others include data structures such as linked lists, trees, hash tables (also known as associative arrays or dictionaries) as built in language features or components of standard libraries always included with the language. Similarly the searching and sorting algorithms often used in coding interviews are built into modern programming languages, libraries and tools. Experienced developers reuse code and rarely re-implement these data structures and algorithms which date to the 1960’s or earlier.
Can these tests really identify the very best candidates either? More likely they identify candidates who have spent many hours drilling on the questions in books like Cracking the Coding Interview.
STEM shortage claims are highly questionable. For sure, there is no shortage of K-12 level STEM skills in the United States and probably world wide. Indeed, the actual hiring practices of STEM employers suggest they are often not interested in the specialized skills they claim to seek when confronted about refusing to hire, laying off, or firing seemingly highly qualified engineers and other STEM workers.
Is the real problem a STEM worker shortage or excessively picky, irrational, discriminatory and ultimately costly hiring and employment practices?
(C) 2017 John F. McGowan, Ph.D.
Credits
The picture of Ken Thompson and Dennis Ritchie is from Wikimedia Commons and is in the public domain.
About the Author
John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech).