Shale 2.0: The Coming Big-Data Revolution in America’s Shale Oil Fields

With petroleum prices down 50 percent over the past year, many analysts and pundits are predicting the end of America’s shale oil boom. Recent headlines include: “Oil Price Fall Forces North Dakota to Consider Austerity” (New York Times);  “Oil Price Drop Hurts Spending on Business Investments” (Wall Street Journal); “The American Oil Boom Won’t Last Long at $65 per Barrel” (Bloomberg Business); and “The Shale Oil Revolution Is in Danger” (Fortune).

High prices, shale skeptics argue, created a bubble of activity in unsustainably expensive shale fields. As shale-related businesses contract, consolidate, and adjust to the new price regime, a major shale bust is inevitable, they add, with ghost towns littering idle fields from Texas to North Dakota.

It is true that the oil-price collapse was caused by the astonishing, unexpected growth in U.S. shale output, responsible for three-fourths of new global oil supply since 2008. And as lower prices roil operators and investors, the shale skeptics’ case may seem vindicated. But their history is false: the shale revolution, “Shale 1.0,” was sparked not by high prices—it began when prices were at today’s low levels—but by the invention of new technologies. Now, the skeptics’ forecasts are likely to be as flawed as their history. Continued technological progress, particularly in big-data analytics, has the U.S. shale industry poised for another, longer boom, a “Shale 2.0.”

John Shaw, chair of Harvard’s Earth and Planetary Sciences Department, recently observed: “It’s fair to say we’re not at the end of this [shale] era, we’re at the very beginning.” He is precisely correct. In recent years, the technology deployed in America’s shale fields has advanced more rapidly than in any other segment of the energy industry. Shale 2.0 promises to ultimately yield break-even costs of $5–$20 per barrel—in the same range as Saudi Arabia’s vaunted low-cost fields.

The shale industry is unlike any other conventional hydrocarbon or alternative energy sector, in that it shares a growth trajectory far more similar to that of Silicon Valley’s tech firms. In less than a decade, U.S. shale oil revenues have soared, from nearly zero to more than $70 billion annually (even after accounting for the recent price plunge). Such growth is 600 percent greater than that experienced by America’s heavily subsidized solar industry over the same period.

Shale’s spectacular rise is also generating massive quantities of data: the $600 billion in U.S. shale infrastructure investments and the nearly 2,000 million well-feet drilled have produced hundreds of petabytes of relevant data. This vast, diverse shale data domain—comparable in scale with the global digital health care data domain—remains largely untapped and is ripe to be mined by emerging big-data analytics.

Shale 2.0 will thus be data-driven. It will be centered in the United States. And it will be one in which entrepreneurs, especially those skilled in analytics, will create vast wealth and further disrupt oil geopolitics. The transition to Shale 2.0 will take the following steps:

1. Oil from Shale 1.0 will be sold from the oversupply currently filling up storage tanks.

2. More oil will be unleashed from the surplus of shale wells already drilled but not in production.

3. Companies will “high-grade” shale assets, replacing older techniques with the newest, most productive technologies in the richest parts of the fields.

4. And as the shale industry begins to embrace big-data analytics, Shale 2.0 begins.

In recent decades, developed nations have spent hundreds of billions of government dollars trying, and failing, to invent a cost-effective replacement for petroleum. Yet without taxpayer largesse, American entrepreneurs invented a new method to extract astounding quantities of oil from rock, upending the global hydrocarbon trade in the process. In a world where oil still powers 95 percent of air and ground miles and will remain dominant for decades, this represents a very positive development.

Compared with 1986—the last time the world was oversupplied with oil—there are now 2 billion more people living on earth, the world economy is $30 trillion bigger, and 30 million more barrels of oil are consumed daily. The current 33 billion-barrel annual global appetite for crude will undoubtedly rise in coming decades. Considering that fluctuations in supply of 1–2 MMbd can swing global oil prices, the infusion of 4 MMbd from U.S. shale did to petroleum prices precisely what would be expected in cyclical markets with huge underlying productive capacity.

While sellers naturally prefer higher prices, the dramatic recent oil-price slump has set the stage for America’s upcoming Shale 2.0 revolution. Given petroleum’s continued economic and geopolitical importance, what policies should the U.S. pursue to maximize the benefits that it secures from Shale 2.0?

Legislators have yet to recognize and incorporate into law the far-reaching implications of how the energy landscape has fundamentally changed. U.S. energy law remains anchored in the decades-old paradigm of insatiable U.S. demand and resource shortages. The modern reality has instead utterly reversed, with de minimis growth in U.S. oil demand, exploding global demand, and an ascendant second era of American petroleum production. Congress should thus undertake a comprehensive review and rewrite of the antiquated 1974 Energy Policy & Conservation Act, which enshrined the now-antiquated paradigm. In the meantime, Congress can:

1. Remove counterproductive rules prohibiting U.S. companies from selling crude oil and shale gas overseas.

2. Remove the 1920 Merchant Marine Act’s constraints on transporting domestic hydrocarbons by ship. This will require finding a more cost-effective solution to the national security interests associated with subsidizing a domestic shipbuilding industry.

3. Avoid inflicting further regulatory burdens on the already heavily regulated shale industry. Ominously, the U.S. Bureau of Land Management has announced plans for new standards that will affect about one-fifth of shale-based hydrocarbon production.

4. Open up and accelerate access to exploration and production on federally controlled lands. This would boost domestic economic opportunities and send a powerful message about America’s oil export ambitions—rivaling, in inverse, the announcement of the 1973 Arab oil embargo.

Further, while Congress works to unshackle a shale industry adjusting to lower prices, hydrocarbon firms might consider launching a “Shale Ready Vets” program—a private alternative to President Obama’s new Solar Ready Vets—offering instruction in vital, high-paying shale-related jobs, from machinery and supply-chain logistics to big-data analytics. At a time of growing scarcity for skilled labor in such fields, such a program would help prepare the U.S. workforce for the emergence of Shale 2.0.

This is excerpted from Shale 2.0: Technology and the Coming Big-Data Revolution in America’s Shale Oil Fields, released by the Manhattan Institute on May 20.

Mark Mills is a senior fellow at the Manhattan Institute. You can follow him on Twitter here.

Interested in real economic insights? Want to stay ahead of the competition? Each weekday morning, e21 delivers a short email that includes e21 exclusive commentaries and the latest market news and updates from Washington. Sign up for the e21 Morning eBrief.