A growing number of analysts say oil-price trends can no longer be explained simply through supply and demand. While energy analysts still see those factors as the foundation of the oil market, they also view oil investor behavior as a factor in recent prices. Increasingly speculative behavior by a more diverse set of investors outside the oil industry–including hedge funds, pension funds, and investment banks–has made oil-market trends harder to predict, say analysts. Many believe speculative investments from financial firms contributed to record-high global oil prices seen in early 2008, and that a selloff by firms contributed to the subsequent massive price decline later in the year. As major political events rock the Middle East, analysts also worry about market speculation in 2011, which has already experienced the largest increase in oil’s history.

A 2010 U.S. financial reform law was intended to curtail trading by non-industry players to lessen volatility, but attempts to craft new regulations under the law could be hampered by federal budget pressures and other issues.

The past few years have seen sharp volatility in oil prices. During 2008, oil prices set record highs during the early part of the year, jumping to a high of over $147 per barrel in July. Oil prices then fell sharply in 2008 at the outset of the global financial crisis, plunging to less than $40 a barrel by December 2008. Prices started rising again gradually over the next two years, and were more than $90 a barrel at the beginning of 2011. Protests in the Middle East and North Africa, particularly in Libya, caused an even steeper jump in prices as investors grew concerned about the region’s stability. The rise in prices was followed by a “massive sell-off” (WSJ) of oil in May, which some analysts say was partially caused by concerns demand that is falling and the U.S. economic recovery is slowing down.

Supply and demand remain among the most influential components of oil-market behavior. Unlike in most other markets, though, drastic changes in oil price do not necessarily kindle changes in demand. “Prices can fall a long way without stimulating demand,” says Tim Evans, an energy analyst at Citigroup. While lessening demand does have an impact on price, the continuing increase in oil consumption by developing countries coupled with steady high demand from the United States mean that demand is not likely to ease in the near future, analysts say.

Supply issues, on the other hand, can have considerable impact on oil prices. Geopolitical events that threaten oil supplies, such as troubles between Venezuela and the United States or Turkey and Kurdish Iraq, can spook investors and lead to price volatility. Concerns that a major supply disruption could result from ongoing unrest in the Middle East in early 2011 drove up prices within a few weeks and led to estimates that oil could reach above $200 per barrel (CNBC) within the year. “Two factors determine the price of a barrel of oil: the fundamental laws of supply and demand, and naked fear,” the Economist wrote in March 2011.

However, evidence suggests the price increases of 2008 did not reflect major changes in market fundamentals or political instability. Saudi officials noted that in the year between 2007 and 2008, when oil prices more than doubled, increases in oil supplies outstripped the rise in oil demand (PDF), according to a January 2010 report prepared for the International Energy Forum.

Oil markets also do not entirely act like other commodities markets. Evans, who calls the market “a volatile beast,” says the Organization of the Petroleum Exporting Countries (OPEC) cartel is a major reason the oil market is not truly competitive like other markets. Up until the mid-1980s, OPEC set crude oil prices. Now it simply influences the market by adjusting production levels for its members, which supply about 40 percent of the market. Such actions can have a significant impact on prices. OPEC supply cuts in 2003 and 2007 resulted in gradual but significant increases in prices in the following months. But when crude prices rose in 2007, OPEC officials often contended that the market had enough supply and blamed rising prices on oil-market investors.

Energy expert Philip Verleger argues market unpredictability is not the fault of investors. Instead, he says oil analysis has become “lazy” and has not adapted to the fact that the oil market now behaves more like other financial markets where supply and demand factors prevail. Evans agrees somewhat, but says increased “financially driven oil trading has given the price a greater independence to swing further away from a fair market or equilibrium price for a longer period of time.”

During previous times of market turmoil, large producers such as the United States and Saudi Arabia would raise production and stabilize prices. CFR’s Michael Levi says to expect more volatility in oil markets in part because there is no longer a big producer step in. “That means prices have to swing far to balance supply and demand,” he says.

As with other commodities markets, there are two sets of oil prices: a spot price, for oil delivered immediately, and a futures price, for oil delivered at some specified time in the future. When the spot price for oil is less than the futures price, this is called a “contango” market. When the futures price for oil is less than the spot price, this is called a “backwardation” market. Contango markets encourage the stockpiling of oil supplies; people will buy spot-price oil and hold it, waiting for the higher price in the future. In backwardation markets, there is an incentive for oil holders to sell their stocks in the short term, thus adding supplies to the market.

If a contango market implies that oil inventories currently are abundant, a backwardation market implies that oil inventories are currently tight. In a backwardation market, there is an incentive for investors to simply sell the near-term futures held at the higher prices and buy futures further out at the lower rate, thus making a profit on the difference. Verleger says, “Cash from investors adds liquidity to the market, and liquidity is a good thing.” But Evans notes investors also increase volatility, which may not be so benign.

Large-supply holders such as OPEC have in the past played a big role in moving the spot price and futures markets to suit their aims. Experts say OPEC created a backwardation market by significantly curtailing production in the late 1990s to drive oil prices up from $10 per barrel. Backwardation can be self-perpetuating; investors sell in the short term because they believe the price will go down in the future, which keeps inventories low.

Investors closely follow oil inventories to gauge supply and demand. But complicating oil markets are new kinds of investors (PDF), including corporations, hedge funds, and institutional investors like pension funds. Non-industry oil speculation, as seen in the last few years, masks the real supply and demand of present oil in favor of the supply and demand of oil futures. For example, by some estimates, speculators in 2011 own futures for six times more oil on the West Texas Intermediate exchange than can be physically stored (Fortune) at the WTI facility in Cushing, Okla.

An August 2009 report (PDF) from the James Baker III Institute for Public Policy found that noncommercial players now constitute about 50 percent of the U.S. oil futures market, compared to an average of about 20 percent before 2002. “The speculative fervor is so remarkable that the big trading firms now have nearly twice as many long contracts open as they did in 2008, when oil spiked to $147 in the summer, a development that either foreshadowed or caused the global economic meltdown,” wrote Fortune’s Colin Barr in March 2011.

But others, such as NASDQ blogger Phil Flynn, contend the oil buying driving up prices is due to some refiners and European countries being “fearful they won’t be able to get oil next week,” not because of non-industry speculators.

D. Barry McKennitt, executive director of the U.S. National Association of Petroleum Investment Analysts, says that none of this moneymaking would be possible unless supplies were tight, but speculation on this scale magnifies price volatility. “When speculators make up too large a share of the futures market, they have the potential to upset the healthy tension between consumers and producers and resulting adherence of prices to market fundamentals,” writes the University of Maryland’s


Michael Greenberger in a January 2010 paper (PDF). The 2009 Baker Institute report found that non-oil industry traders tend to be bullish, which encourages prices increases.

Despite new investor behavior, contango markets remain more stable because of higher inventory, but backwardation markets spur growth in the oil industry for infrastructure and exploration–considered very good for the industry. One reason oil supplies are tight is because a lack of investment in exploration and infrastructure in the 1990s, deterred by low oil prices. Giacomo Luciani, director of the Gulf Research Center Foundation, notes that commodities investors tend to profit from volatility and argues that the way to reverse the ballooning of prices through speculation is for OPEC to reassume its role as “price maker” (ArabNews).

Other energy experts say OPEC is now in a delicate position for addressing market dynamics. While high oil prices can encourage much-needed investments in oil infrastructure and new exploration, some oil producers remain wary of spending their windfall due to worries that the market will return to the prices of the 1990s. But most analysts believe oil supplies will remain tight and prices high.

Different qualities of crude oil supplies play a crucial role in market prices. Crude oil is classified by density and sulfur content. Refiners consider light and sweet crude (containing less than 0.5 percent sulfur) the best because it takes little refining to produce high quality products, such as gasoline. Sour crude contains 1 percent sulfur, a problem for refining since sulfur is both corrosive and toxic. The higher viscosity of heavy crude also requires extra efforts to refine. Since the world has limited refining capacity for sour and heavy crude, these varieties sell at prices less than sweet and light varieties, sometimes significantly so. Crudes can be blends of heavy sweet or light sour as well.

A considerable portion of OPEC production falls into the heavy-sour category. Many non-OPEC nations also have started to produce more heavy-sour crude due to older oil wells. This in turn has created a tight supply of light-sweet crude. Some analysts believe heavy-sour production is going to significantly exceed light-sweet production as time goes on, which could continue to help increase volatility on light-sweet crude prices until refining capacity catches up.

Platts, an energy news service, says of the two lower-grade types, sour or heavy, “it is the sour crude that is gradually assuming more importance” in pricing. However, sour crude has become less attractive since the United States and Europe recently mandated greater use of low-sulfur diesel for air quality reasons. The International Energy Agency says the first option for refiners to meet low-sulfur targets is to use low-sulfur crudes (PDF). This has created greater demand for light-sweet crude, leading to tighter supplies in the past year. Under such conditions, when major suppliers like Saudi Arabia seek to boost production to ease prices, they sometimes have trouble finding buyers because their surplus production is mainly lower-quality crude. Light-sweet crude prices jumped by $20 to $25 per barrel from August 2007 to November 2007, Verleger notes, while at the same time Saudi Arabia was cutting the price of its sour crude. There is also some concern that a Saudi pledge in 2011 bolster losses in production–mostly sweet light crude–by countries like Libya will not be adequate, since Saudi replacements will largely involve sour crude.

OPEC in November 2007 said the shift toward lighter oils and stricter fuel-quality standards will have “substantial consequences” on supply and demand. The organization stressed the need to upgrade refining for desulphurization capacity by eighteen million barrels per day in the next fifteen years. Citigroup analyst Evans says refineries just need to spend the money to upgrade refineries. He says as long as poorer-grade crudes sell at a discount and light crude prices remain high, there will be an incentive to invest in upgrades.

OPEC and others have called for greater controls on energy markets–particularly on futures trading. The G20 is considering new rules to address volatile commodity markets. A U.S. financial reform law enacted in July 2010 gave the U.S. Commodities Futures Trading Commission (CFTC) more power to regulate the futures market, migrating many trades from face-to-face deals to financial exchanges to allow for more transparency. “End user” businesses–such as oil companies–would be exempt for transactions used to hedge risk from producing or consuming commodities. There are also plans to limit the amount of trading by non-industry traders such as banks.

But a February 2011 CFR Working Paper on improving international energy markets from Daniel Ahn contends that setting limits may be counterproductive for “restraining excessive speculation and may be useful only for the narrower purpose of preventing market manipulation.” Ahn goes on to argue that policymakers in the United States and in the G20 countries “should prioritize those areas that are better understood to unambiguously improve market efficiency, particularly better physical and financial transparency.”

New rules will not be set until 2012 and face challenges from a variety of quarters. Republican lawmakers–many opposed to the financial reform law–have proposed major cuts to the agency’s budget (CNBC), which could hamper new rulemaking. Others are concerned that even if the rulemaking proceeds, trader pressure is attempting to ensure it will be largely toothless (BusinessInsider).