At the beginning of the 20th century, refined oil was used primarily for lighting. This use was soon eclipsed by the needs of automobile and aircraft, making oil a more significant fuel than coal by 1920. Liquid petroleum was easier to transport, and was a much more concentrated and flexible form of fuel than anything previously available. Soon, dependence on crude oil made it a raw material of immense economic value and international political significance. Eventually, oil refining innovations would allow engineers to tailor the products of crude oil to suit the market, thus marking the true beginning of the petrochemical industry and the development of modern plastics.

Petroleum-based fuels transformed the world landscape as they increased agricultural productivity, provided the means for distributing industrial and farm products, and furnished the personal mobility that defines 20th century technology. Petrochemicals have also had an enormous impact, providing everything from aspirin to zippers, including pharmaceuticals, medical devices, synthetic fabrics, fertilizers, pesticides, building materials, and cosmetics, among them.

Initially, the demand for convenient and affordable fuels drove the industry. During the war years and later, concern shifted to finding substitutes for increasingly scarce natural commodities such as rubber. The resulting products, such as sulfa drugs and vitamins, were useful in fighting infectious diseases. Later in the century, emphasis shifted to reducing the environmental consequences of oil. Gasoline, for example, was reformulated to eliminate lead.

Until 1900, refining consisted of a fairly simple batch process whereby oil was heated until it vaporized, and the various fractions were separated by distillation. The first big advance came in 1913 with the introduction of thermal cracking. This process took the less volatile fractions after distillation and subjected them to heat under pressure, thus cracking (breaking) the heavy molecules into lighter molecules for producing gasoline, kerosene, and light industrial fuels. The introduction of catalytic cracking in 1936 further manipulated the molecules of the hydrocarbon raw material, providing a higher-octane product. Cracking of petroleum yields light oils, heavier oils, and gases such as methane, ethane, ethylene, propane, and propylene. These gases are the starting points for the production of compounds that constitute five major groups of end products: synthetic rubber, plastics, textiles, detergents, and agricultural chemicals.

Edwin Drake drilled the first well specifically for oil in Titusville, Penn., in 1859. The success of this well, drilled close to an oil seep, soon led to similar exploration elsewhere. Within a short time, inexpensive oil from underground reservoirs was being processed at already existing coal-oil refineries, and by 1900, oil fields had been discovered in 14 states. During the same period, oil fields were found in Europe and East Asia as well. In 1900, crude oil production worldwide was nearly 150 million barrels. Half of this total was produced in Russia, and most of the rest was produced in the U.S. Annual production surpassed 1 billion barrels in 1925 and 2 billion barrels in 1940.

Since World War II the demand for light products (gasoline, jet, and diesel fuels) has grown, while the requirement for heavy industrial fuel oils has declined. In 1947, a process called "platforming" introduced platinum as a catalyst in the refining process. This resulted in fewer emissions, removed much of the sulfur and other contaminants, and generated significant amounts of hydrogen and other raw materials used to manufacture plastics. The availability of hydrogen was one of the most far-reaching developments of the refining industry in the 1950s. Since 1980, hydrogen processing has become so prominent that many refineries now incorporate hydrogen manufacturing plants in their processing schemes.

By the last decade of the 20th century, there were almost 1 million wells in more than 100 countries producing more than 20 billion barrels per year. Every day, more than 60,000,000 barrels of oil are produced. The rate of growth has been stunning. The first 200 billion barrels of world oil were produced before 1968 — since that time, world oil production rates have stabilized at a rate of about 22 billion barrels a year.

The challenge of retrieving this resource, however, often places humans in hostile and volatile environments. The vast majority of petroleum deposits lie trapped in the pores of natural rock at depths from 500 to 25,000 feet below the surface of the ground. As a general rule, the deeper deposits have higher internal pressures and contain greater quantities of gaseous hydrocarbons.

The Middle East is thought to have an estimated 41 percent of the world's total oil endowment, with Saudi Arabia thought to have the largest original oil endowment of any country. North America is a distant second. Eastern Europe, because of the large deposits in Russia, is also well endowed with oil. Most of Western Europe's oil is buried below the North Sea.

Cumulatively, the U.S. has produced more oil than any other country, but is still considered to have a significant remaining undiscovered oil resource. Prudhoe Bay, which accounted for approximately 17 percent of U.S. oil production during the mid-1980s, is in decline. This situation, coupled with declining oil production in the contiguous states, has contributed to a significant drop in domestic oil output.

With an estimated 77 percent of the world's total recoverable oil having already been discovered, the remaining 23 percent, mostly located in smaller fields or in more difficult environments, is expected to become ever more expensive to find and to recover. The offshore oil and gas industry has produced some of the world's largest and most unique structures, as well as the large floating construction vessels required to build them. Some are located hundreds of miles offshore, enduring formidable forces — hurricanes in the Gulf of Mexico, typhoons in the South China Sea, earthquakes off California shores, and icebergs that roam the Canadian North Atlantic. More than 11,000 work-years were required to construct the largest of the North Sea gravity platforms, making capital costs per daily oil production as much as 40 times the costs in the Middle East. A guyed tower constructed in more than 300 meters of water in the Gulf of Mexico has been estimated to produce oil at about 65 times the production cost of the Middle East. As oil exploitation moves into deeper waters or under Arctic ice, the cost will further escalate.

As the world has become aware of the impact of industrial pollution on the environment, the petroleum-refining industry has had to take remedial action. Refiners added hydrotreating units to extract sulfur compounds from their products and began to generate large quantities of elemental sulfur. Effluent water and atmospheric emission of hydrocarbons and combustion products have been reduced, and techniques have been developed for manufacturing high-quality gasoline without employing lead additives. For example, the catalytic converter, developed in the 1970s, removes nitrous oxide and sulfur dioxide from vehicular emissions. Additives such as ethanol were placed into fuel to make it burn cleaner. By 1990, substantial investments in the complete reformulation of transportation fuels helped to minimize environmental emissions. Petroleum refining has become one of the most stringently regulated of all manufacturing industries, expending a major portion of its resources on the protection of the environment.

For these and other reasons, much research has examined alternatives to petroleum-based fuels. Since the crude oil shortages of the late 1960s and early 1970s, the natural gas formed alongside or near oil deposits (which was for many years flared or burned off at the wellhead) became itself an important world energy source. Because natural gas burns completely, carbon dioxide and water are normally formed, and the combustion of gas is relatively free of soot, carbon monoxide, and the nitrogen oxides associated with the burning of other fossil fuels. In addition, sulfur dioxide emissions, another major air pollutant, are almost nonexistent with natural gas.

Throughout the 19th century the use of natural gas remained localized because there was no way to transport large quantities of it over long distances. An important breakthrough occurred in 1890 with the invention of leakproof pipeline coupling, which still had limited use for anything more than 100 miles from a source of supply. Long-distance gas transmission became practical during the late 1920s, and from 1927 to 1931, more than 10 major transmission systems were constructed in the U.S. alone. Since the early 1970s, the longest gas pipelines have been built in Russia.

The engineering efforts in the petroleum and gas industries continue to resolve environmental issues, dealing with shortages and spills alike. If, as anticipated with any limited resource, oil reigns as the dominant source of energy for a mere transitory period of 100 years or so, then it will have done so with enormous influence on the quality of life worldwide.