Page 291

Building the Next-Generation Biological-Information Infrastructure.

John L. Schnase
Center for Botanical Informatics, LLC. St. Louis, MO
(Current address: NASA Goddard Space Flight Center, Greenbelt, MD 20771)
Meredith A. Lane
The Academy of Natural Sciences, 1900 Benjamin Franklin Parkway, Philadelphia, PA 19103
Geoffrey C. Bowker
Susan Leigh Star
Graduate School of Library and Information Sciences, University of Illinois at Urbana-Champaign, Champaign, IL
(Current address: Department of Communication, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0503)
Abraham Silberschatz
Information Sciences Research Center, Lucent Technologies-Bell Laboratories, 600 Mountain Avenue, Murray Hill, NJ 07974

A grand challenge for the 21st century is to harness the accumulating knowledge of Earth's biodiversity and the ecosystems that support it. To accomplish that, we must mobilize biological information—assemble it, organize it, and deliver it with dramatically increased capacity. We must elevate the global biological-information infrastructure to a new level of capability—a “next generation”—that will allow people to share on a worldwide basis the knowledge created by biodiversity and ecosystems research.

Recognizing the urgency of the task, the President's Committee of Advisors on Science and Technology, through its Panel on Biodiversity and Ecosystems, recently coordinated a review of the US National Biological Information Infrastructure (NBII) (PCAST 1998). Over a 6-month period in 1997, people from a broad cross section of the public and private sectors contributed their insights, experiences, concerns, and hopes. What emerged was a renewed understanding of the importance of biological information to all aspects of human society. It became clear that much remains to be done to ensure that this information is complete and usable. Although the purpose of the review was to develop recommendations to build capacity in the United States, many of the panel's findings address global concerns of relevance to biodiversity research wherever it occurs. In this paper, we provide a summary of the panel's report, a view of what a next-generation

Page 292

biological-information infrastructure might encompass, and suggestions about how it might be achieved.

Background

In the United States, NBII is the primary mechanism whereby biodiversity and ecosystem information is made available to all sectors of society. It is the biological component of the National Information Infrastructure and is the framework that connects US activities to the global biodiversity and ecosystem research enterprise. Its meaning is expansive and intended to convey the idea that an information infrastructure comprises not only computers, networks, and the like, but also the information, policies, standards, and people who use it. Initiation of the NBII was one of the primary recommendations made by the 1993 National Research Council report A Biological Survey for the Nation (NRC 1993).

Because our fate and economic prosperity are so completely linked to the natural world, information about biodiversity and ecosystems—as well as the infrastructure that supports it—is vital to a wide range of scientific, educational, commercial, and government uses. Most of this information now exists in forms that are not easily accessed or used. From traditional paper-based libraries to scattered databases and physical specimens preserved in natural-history collections throughout the world, our record of biodiversity and ecosystem resources is uncoordinated, and large parts of it are isolated from general use. It is not being used effectively by scientists, resource mangers, policy-makers, or other potential client communities (National Performance Review 1997; NRC 1997).

Research activities are being conducted around the world that could improve our ability to manage biological information. In the United States, the Human Genome Project is producing new medical therapies and developments in computer and information science. Geographic information systems (GISs) are expanding the ability of federal agencies to conduct data-gathering and data-synthesis activities more responsibly and creating opportunities for commercial partnerships that can lead to new software tools. The National Spatial Data Infrastructure (http://nsdi.usgs.gov) is improving the management of geographic, geological, and satellite datasets; the Digital Libraries (http://www.cise.nsf.gov/iis/dli/home.html) projects are beginning to produce useful results for some information domains; and the High-Performance Computing and Communications Initiative (http://www.hpcc.gov) has enhanced some computation-intensive engineering and science fields.

But little attention has been paid to computer and information science and technology research in the biodiversity and ecosystem domain. We must produce mechanisms that can efficiently search through terabytes of Mission to Planet Earth satellite data and other biodiversity and ecosystem datasets, make correlations among data from disparate sources, compile those data in new ways, analyze and synthesize them, and present the results in an understandable and usable manner. Despite encouraging advances in computation and communication performance in recent years, we are able to perform these activities on only a very small scale. We can, however, make rapid progress if the computer and

Page 293

information science and technology research community becomes focused on the needs of the biodiversity and ecosystem research community (Robbins 1996).

Managing Complexity

Knowledge about biodiversity and ecosystems is vast and complex. The complexity arises from two sources. The first is the underlying biological complexity of the organisms themselves. There are millions of species, each of which is highly variable across individual organisms, populations, and time. Species have complex chemistries, physiologies, developmental cycles, and behaviors resulting from more than 3 billion years of evolution. There are hundreds, if not thousands, of ecosystems, each comprising complex interaction among large numbers of species and between those species and multiple abiotic factors.

The second source of complexity in biodiversity and ecosystem information is sociologically generated. The sociological complexity includes problems of communication and coordination—among agencies, among divergent interests, and among groups of people from different regions and different backgrounds (academe, industry, and government) and with different views and requirements. The kinds of data that humans have collected about organisms and their relationships vary in precision, in accuracy, and in numerous other ways. Biodiversity data types include text and numerical measurements, images, sound, and video. The range of other databases with which biodiversity datasets must interact is also broad, including geographic, meteorological, geological, chemical, and physical databases. The mechanisms used to collect and store biological data are almost as varied as the natural world that they document. In addition, biological data can be politically and commercially sensitive and can entail conflicts of interest. Users' skill levels are highly variable, and training in this field is not well developed.

Because of those complexities, humans still play a crucial role in the processing of biological data. Biological information is not as amenable to automatic correlation, analysis, synthesis, and presentation as many other types of information, such as that in radioastronomy, where there is more coherent global organization and the problems being studied are often conducive to automatic analysis. In biodiversity research, people act as sophisticated filters and query processors—locating resources on the Internet, downloading datasets, reformatting and organizing data for input to analysis tools, then reformatting again to visualize results. This process of creating higher-order understanding from dispersed datasets is a fundamental intellectual process, but it breaks down quickly as the volume and dimensionality of the data increase. Who could be expected to “understand” millions of cases, each having hundreds of attributes? Yet problems on this scale are common in biodiversity and ecosystem research (Schnase and others 1997).

For a biological-information infrastructure to be effective, it must provide the means to manage complexity. It must allow scientists to extract new knowledge from the aggregate mass of information generated by the data-gathering and synthesis activities of other scientists. It must use the power of computers to facilitate the queries, correlations, and processing that are impossible for humans to

Page 294

perform alone. And it must deliver this functionality within a physically and intellectually accessible framework. That means developing ways of delivering information to a wide array of users with differing skills, ages, and investment in the material.

We are only beginning to develop a vocabulary to describe these large-scale, synthetic, information-processing activities. Some sociologists use the term distributed cognitive system to emphasize the role of humans in a synergistic information-processing network (Hutchins 1995). Data mining is often used by the data base community. Whatever the name, the activities form only a part of a process of knowledge discovery that includes the large-scale, interactive storage of information (known by the unintentionally uninspiring term data warehousing); the cataloging, cleaning, preprocessing, transformation, verification, and reduction of data; and the generation and use of models, evaluation and interpretation, personal communication, the evolution of sophisticated user interfaces, and finally consolidation and use of the newly extracted knowledge. Those processes will become increasingly important if we are to use what we know and expand our knowledge in useful directions.

At present, the NBII provides little support for these activities. At best, it can be used to access information in databases held by federal agencies and other institutions around the country. Once the information is accessed, however, the task of organizing, integrating, and interpreting it remains, for the most part, a laborious, manual process. The development of computational tools for the biodiversity and ecosystem enterprise lags behind other sciences. Important classes of information are missing (information on fewer than 1% of the specimens in our natural-history collections has been entered in databases!), and databases are uneven in the types of information that they hold. It is difficult for individual scientists to publish their data electronically in useful ways. Standards for information exchange have not been widely adopted. We have no mechanism for archiving data over generations of use and generations of technologies. And the power of communication networks to build communities remains largely untapped. In summary, the NBII is neither a system nor an infrastructure: it is a cumbersome and brittle patchwork that presents as many obstacles to scientific work as it does opportunities. It clearly is time to transform it into a coherent and empowering capability.

The Next Generation—NBII-2

We envision a “next generation” National Biological Information Infrastructure, NBII-2, that would address many of the concerns described above. The overarching goal of NBII-2 would be to become a fully accessible, distributed, interactive digital library. NBII-2 would provide an organizing framework from which scientists could extract useful information—new knowledge—from the aggregate mass of information generated by various data-gathering activities. That would be accomplished by using the power of computers and communication networks to augment the processing activities that now require a human mind. It would make analysis and synthesis of vast amounts of data from multiple datasets easier

Page 295

and more accessible to a variety of users. It would also serve management and policy decision-making, education, recreation, and industry by presenting data to each user in a manner tailored to that user's needs and skill level.

We envision NBII-2 as a distributed facility that would be considerably different from a “data center,” considerably more functional than a traditional library, considerably more encompassing than a typical research institute. Unlike a data center, NBII-2 would have the objective of automatic discovery, indexing, and linking of datasets rather than collection of all datasets on a given topic into one facility. Following the best practice of traditional libraries, this special library would update the form of storage and upgrade information content as technologies evolve. Unlike a typical research institute, it would provide services to research going on elsewhere, and its own staff would conduct biodiversity and ecosystem research and research in biological informatics. The facility would offer “library” storage and access to diverse constituencies.

The core of NBII-2 would be a “research library system” that would comprise at least five regional nodes sited at appropriate institutions (national laboratories, universities, museums, and so on) and connected to each other and to the nearest telecommunication providers by the highest-bandwidth network available. In addition, NBII-2 would seamlessly integrate all computers—laptops, workstations, fileservers, and supercomputers—capable of storing and serving biodiversity and ecosystem data via the Internet. The providers of information would have complete control over their own data but have the opportunity to benefit from (and the right to refuse) the data-indexing, cleansing, and long-term storage services of the system as a whole.

NBII-2 would be

• the framework to support knowledge discovery for the nation's biodiversity and ecosystem enterprise and to involve many client and potential-client groups;

• a common focus for independent research efforts and a global context for sharing information among those efforts;

• an accrete-only, no-delete facility from which all information would be available on line—24 hours a day, 7 days a week—in a variety of formats;

• a facility that would serve the needs of (and eventually be supported by partnership with) government, the private sector, education, and individuals;

• an organized framework for collaboration among federal, regional, state, and local organizations in the public and private sectors that would provide improved programmatic efficiencies and economies of scale through better coordination of efforts;

• a commodity-based infrastructure that uses readily available, off-the-shelf hardware and software and the products of digital-library research wherever possible;

• an electronic facility where scientists and others could “publish” biodiversity and ecosystem information for cataloging, automatic indexing, access, analysis, and dissemination;

• a place where intensive work on how people use large information systems would be conducted, including studies of human-computer interaction, the

Page 296

sociology of scientific practice, computer-supported cooperative work, and user-interface design;

• a place for developing the organizational and educational infrastructure that will support sharing, use, and coordination of massive datasets;

• a facility that would provide content storage resources, registration of datasets, and “curation” of datasets (including migration, cleansing, and indexing);

• an applied biodiversity and ecosystem informatics research facility that would develop new technologies and offer training in informatics; and

• a facility that would provide high-end computation and communication to researchers and institutions throughout the country.

The facility would not be a purely technical and technological construct but, would also encompass sociological, legal, and economic issues within its research purview. These would include intellectual-property rights management, public access to the scholarly record, and the characteristics of evolving systems in the networked information environment. The human dimensions of the interaction with computers, networks, and information will be particularly important subjects of research as systems are designed for the greatest flexibility and usefulness to people.

The research nodes of NBII-2 must address many needs, including

• new statistical pattern-recognition and modeling techniques that can work with high-dimensional, large-volume data;

• workable data-cleaning methods that automatically correct input and other types of errors in databases;

• strategies for sampling and selecting data;

• algorithms for classification, clustering, dependency analysis, and change and deviation detection that scale to large databases;

• visualization techniques that scale to large and multiple databases;

• metadata encoding routines that will make data mining meaningful when multiple distributed sources are searched;

• methods for improving connectivity of databases, integrating data-mining tools, and developing better synthetic technologies;

• methods for improving large-scale project coordination and scientific collaborations;

• continuing, formative evaluation, detailed user studies, and quick feedback between domain experts, users, developers, and researchers;

• methods for facilitating data entry and the digitization of large amounts of irregularly structured information; and

• ways of engaging society in the pursuit of global information-sharing.

None of those problems is peculiar to biodiversity research. However, there is an urgent need to address them in the biodiversity domain because research has demonstrated that there can be no domain-independent solutions. We cannot “borrow” discoveries wholesale from other disciplines; we must work through these problems ourselves (Star and Ruhleder 1996). To comprehend and use our

Page 297

biodiversity and ecosystem resources, we must learn how to exploit massive datasets, learn how to store and access them for analytical purposes, and develop methods to cope with growth and change in data. NBII-2 as envisioned here can be the enabling framework that unlocks the knowledge and economic power lying dormant in the masses of biodiversity and ecosystem data that we have on hand now and will accumulate in the future.

Infrastructure Requirements

The total volume of biodiversity and ecosystem information is almost impossible to measure. We do know that whatever the total, only a fraction has been captured in digital form. Our natural-history museums, for example, contain at least 750 million specimens, the vast majority of which have not been recorded in databases. The same holds for the published record, where most biodiversity and ecosystem information still resides in paper-based journals, books, field notes, and the like. Clearly, one of the most important infrastructure issues is to move the biodiversity and ecosystem enterprise into a digital world—to create the content for the NBII-2 digital library—by digitizing the existing corpus of scholarly work on a large scale.

The NBII-2 digital library will place challenging demands on network hardware services and on software services related to authentication, integrity, and security. Needed are both a fuller implementation of current technologies, such as digital signatures and a public-key infrastructure for managing cryptographic key distribution, and consideration of tools and services in a broader context related to library use. For example, the library system might have to identify whether a user is a member of an organization that has some set of access rights to an information resource. As a national and international enterprise that serves a large range of users, the library must be designed to detect and adapt to various degrees of accessibility of resources connected to the Internet.

A fully digital, interactive library system, such as NBII-2, will require substantial computational resources, although little is known now about the precise scope of the necessary resources. In many aspects that are critical to digital libraries, such as knowledge representation and resource description or summarization and navigation, even the basic algorithms and approaches are not yet well defined, so it is difficult to project computational requirements. Many information-retrieval techniques are intensive in their computational and input-output demands as they evaluate, structure, and compare large databases in a distributed environment. Distributed-database searching, resource discovery, automatic classification and summarization, visualization, and presentation are also computationally intensive activities that are likely to be common in the NBII-2 digital library.

Finally, NBII-2 will require enormous storage capacity. Even though the library system we are proposing would not set out to accrue datasets to become the repository of all biodiversity data—many other federal agencies have their own storage facilities, and various data-providers will want to retain control over their own data—large amounts of storage on disk, tape, optical media, and other future storage forms will still be required. As research is conducted to produce new ways to

Page 298

manipulate large datasets, these will have to be sought out, copied from their original sources, and stored for use in the research. And in serving its long-term curation function, NBII-2 will accumulate substantial amounts of data for which it will be responsible, including redundant datasets that will have to be maintained in case of loss.

Research Agenda

New approaches to managing information must be developed in the context of NBII-2. Massive datasets can lead to the collapse of traditional approaches in database management, statistics, pattern recognition, personal-information management, and visualization. For example, a statistical-analysis package assumes that all the data to be analyzed can be loaded into memory and then manipulated. What happens when the dataset does not fit into main memory? What happens if the database is on a remote server and will never permit a naive scan of the data? What happens if queries for stratified samples cannot be accepted, because data fields in the database being accessed are not indexed and the appropriate data therefore cannot be located? What if the database is structured with only sparse relations among tables or if the dataset can be accessed only through a hierarchical set of fields?

Furthermore, challenges often are not restricted to issues of scalability of storage or access. For example, what if a user of a large data repository does not know how to specify the desired query? It is not clear that a structured query language (SQL) statement—or even a program—can be written to retrieve the information needed to answer a query like, “Show me the list of gene sequences for which voucher specimens exist in natural-history collections and for which we also know the physiology and ecological associates of those species.” Many of the interesting questions that users of biodiversity and ecosystem information would like to ask are of this type: they are “fuzzy,” the data needed to answer them must come from multiple sources that will be inherently different in structure and conceptually incompatible, and the answers might be approximate.

Major advances are needed in methods for knowledge representation and interchange, database management and federation, navigation, modeling, and data-driven simulation; in approaches to describing large, complex networked information resources; and in techniques to support networked information discovery and retrieval in extremely large-scale distributed systems. In addition to near-term operational solutions, new approaches are needed to longer-term issues, such as the preservation of digital information across generations of storage, processing, and representation technology. Traditional information-science skills, such as thesaurus construction and indexing, must be elaborated on and scaled to accommodate large information sources. We need to preserve and support the knowledge of library-science and information-science researchers and help to scale up the skills of knowledge organization and information retrieval.

Also much needed are software applications that provide more-natural interfaces between humans and databases than are now available. For example, a valuable data-cleansing activity might be to “show the data related to all specimens

Page 299

in our natural-history collections whose likelihood of being mislabeled exceeds 0.75.” Assuming that some cases in the database can be identified as “labeled correctly” and others “known to be mislabeled,” a training sample for a data-mining algorithm could be constructed. The algorithm would build a predictive model and retrieve records matching that model rather than a structured query that a person might write. This is an example of a much needed and much more natural interface between humans and databases than is currently available. In this case, it eliminates the requirement that the user adapt to the machine's needs rather than the other way around. We must refine and augment the interactions between people and machines, expand the role of agentry in information systems, and discover more-powerful and more-natural ways of navigating the scientific record.

In return, research in computer and information science and technology in the biodiversity and ecosystem domain is likely to yield discoveries of value to other fields (Spasser 1998). Nowhere do we find the problems of heterogeneous database federation more challenging than in the life sciences. A fully implemented digital library for biology would include everything from ideas to physical objects and enormous amounts of information in every medium type imaginable. Research on global climate change, habitat destruction, and the discovery of species is among the most distributed of our scientific activities and creates extraordinary opportunities to learn about computer-mediated project coordination and communication. At almost every turn, scale, complexity, and urgency conspire to create a particularly wicked set of problems. Working on these problems will undoubtedly advance our understanding and use of information technologies, perhaps more than in any other circumstance.

Action Plan

We have laid out the case for building a fully digital, interactive, research-library system for biodiversity and ecosystem information and the basic requirements of and goals for the library and its research and service. But how much will it cost, and how long will it take to build?

We estimate that each of the regional nodes that will form the core of NBII-2 will require an annual operating budget of at least $8 million—probably more. Minimally, supporting five such nodes would require at least $40 million per year, an amount that is a small fraction of the funds spent nationwide each year to collect data (conservatively estimated at $500 million for federal government projects alone). As with the Internet itself, the federal government should provide the “jump start” for this new infrastructure by investing heavily in its formative stages. Part of the investment should be devoted to developing incentives for the participation of private-sector partners. Gradually, support and operation of the infrastructure should be shared by nongovernment participants, as has happened with the Internet.

The planning and request-for-proposals process should be conducted within a year. Merit review and selection of sites should be complete within the following six months. The staffing of the sites and initial coordination of research and

Page 300

outreach activities should take no more than a year after initial funding is provided. The “lifetime” of each facility should not be guaranteed for more than 5 years, but the system must be considered a long-term activity so that data access is guaranteed in perpetuity. Evaluation of the sites and of the system should be regular and rigorous, although the milestones whereby success can be measured will be the incremental improvements in ease of use of the system by students, policy-makers, scientists, and others. In addition, an increasing number of public-private partnerships that fund the research and other operations will indicate the usefulness of accessible, integrated information to commercial and government interests.

Conclusion

In the 21st century, work will depend increasingly on rapid, coordinated access to shared information. Through the shared digital library of NBII-2, scientists and policy-makers will be able to collaborate with colleagues who are geographically and temporally distant. They will use the library to catalog and organize information, perform analyses, test hypotheses, make decisions, and discover new ideas. Educators will use its systems to read, write, teach, and learn. In traditional fashion, intellectual work will be shared with others through the medium of the library—but these contributions and interactions will be elements of a global and universally accessible library that can be used by many different people and many different communities. By increasing the effectiveness of information, NBII-2 is likely to lead to scientific discoveries, advance existing fields of study, promote disciplinary fusions, and enable new research traditions. And most important, it could help us to protect and manage our natural capital so as to provide a stable and prosperous future.

References

Hutchins E. 1995. Cognition in the wild. Cambridge MA: MIT. 381 p.

National Performance Review. 1997. Access America: reengineering through information technology. Report of the National Performance Review and the Government Information Technology Services Board. Washington DC: GPO. 97 p.

NRC [National Research Council]. 1993. A biological survey for the nation. Washington DC: National Acad Pr. 205 p.

NRC [National Research Council]. 1997. Bits of power: issues in global access to scientific data. Washington DC: National Acad Pr. 235 p.

PCAST [President's Committee of Advisors on Science and Technology]. 1998. Teaming with life: investing in science to understand and use America's living capital. Report to the President of the United States from the PCAST Panel on Biodiversity and Ecosystems. Washington DC: GPO.

Robbins RJ. 1996. Bioinformatics: essential infrastructure for global biology. J Comp Biol 3(4):465–78.

Schnase JL, Kama DL, Tomlinson KL, Sánchez JA, Cunnius EL, Morin NR. 1997. The flora of North America digital library: a case study in biodiversity database publishing. J Network Comp Applica 20:87–103.

Spasser MA. 1998. Articulating collaborative activity: design-in-use of collaborative publishing services in the Flora of North America Project. Proceedings of ISCRAT '98 (Århus, Denmark, June 1998).

Star SL, Ruhleder K. 1996. Steps toward an ecology of infrastructure: design and access for large information spaces. Info Syst Res 7(1):111–34.