Post-occupancy evaluation (POE) is a process of systematically evaluating the performance of buildings after they have been built and occupied for some time. POE differs from other evaluations of building performance in that it focuses on the requirements of building occupants, including health, safety, security, functionality and efficiency, psychological comfort, aesthetic quality, and satisfaction. “Lessons learned” refers to programs aimed at collecting, archiving, and sharing information about successes and failures in processes, products, and other building-related areas for the purpose of improving the quality and life-cycle cost of future buildings. Ideally, the information gained through POEs is captured in lessons-learned programs and used in the planning, programming, and design processes for new facilities to build on successes and avoid repeating mistakes.

In 2000 the Federal Facilities Council, a cooperative association of 21 federal agencies with interests and responsibilities for large inventories of buildings, funded a study to look at the state of the practice of POEs and lessons-learned programs in federal agencies and in private, public, and academic organizations both in the United States and abroad. The primary purpose was to produce a report that identified successful post-occupancy evaluation programs (those that have worked well in terms of impact, longevity, and user satisfaction) and lessons-learned programs in federal agencies and the private sector. Specific objectives were to identify:

• an industry-accepted definition of POEs;

• methods and technologies used for data collection;

• the costs of POE surveys;

• the benefits of conducting POEs and capturing lessons;

• organizational barriers to conducting POEs;

• a standardized methodology that could be used within agencies to assure consistency in data gathering and allow for cooperative development of benchmarks and best practices; and

• performance measures for POE programs.

To produce this report the Federal Facilities Council commissioned a set of papers by recognized subject matter experts, conducted a survey of six federal agencies with POE programs, and held a forum at the National Academy of Sciences on March 13, 2001.

The next sections of Chapter 1 summarize the findings contained in the authored papers, the survey questionnaires, and the forum presentations as they relate to the study objectives. In Chapter 2, “The Evolution of Post-Occupancy Evaluation: Toward Building Performance and Universal Design Evaluation,” Wolfgang Preiser reviews the historical development of POE programs, their uses, costs, and benefits; describes an integrative framework for building performance; and outlines a new paradigm for universal design evaluation. Chapter 3, “Post-Occupancy Evaluation: A Multifaceted Tool for Building Improvement,” written by Jacqueline Vischer, discusses the historical basis for POE programs; identifies the discrepancy that exists between reasons for doing POEs and the difficulties of implementing them; describes successful POE pro-

grams employing the building-in-use assessment system; and makes recommendations for an unobtrusive POE process. Chapter 4, “Post-Occupancy Evaluation Programs in Six Federal Agencies,” summarizes the survey questionnaire findings and describes current and emerging POE practices in those agencies. In Chapter 5, “Post-Occupancy Evaluations and Organizational Learning,” Craig Zimring and Thierry Rosenheck identify the elements necessary for organizational learning; explore how 18 organizations have used POEs successfully for organizational learning; and discuss the lessons-learned role of POEs. Chapter 6, “The Role of Technology for Building Performance Assessments,” authored by Audrey Kaplan, identifies technologies that have been used for POE processes; explores the possibilities of cybersurveys for improving the response rate and lowering the costs of POE data collection and analysis; and discusses Web survey design considerations, sampling techniques, publicity, data collection, and responses.

In Appendix A, “Functionality and Standards: Tools for Stating Functional Requirements and for Evaluating Facilities,” Francoise Szigeti and Gerald Davis discuss how the ASTM Standards on Whole Building Functionality and Serviceability (ASTM, 2000) can be used to evaluate the quality of the performance delivered by a facility in relation to the original expectations. Appendix B, “A Balanced Scorecard Approach for Post-Occupancy Evaluation: Using the Tools of Business to Evaluate Facilities,” written by Judith Heerwagen, outlines a performance-based approach that could provide an analytical structure to the entire process, from original concept through lessons learned. Appendixes C-F contain supporting materials. The bibliography is a compilation of references cited in the text and additional references submitted by the authors.

Post-occupancy evaluation is based on the idea that better living space can be designed by asking users about their needs. POE efforts in Britain, France, Canada, and the United States in the 1960s and 1970s involved individual case studies focusing on buildings accessible to academic researchers, such as public housing and college dormitories. Information from occupants about their response to buildings was gathered through questionnaires, interviews, site visits, and observation; sometimes the information was linked to the physical assessment of a building. The lessons from these studies were intended to convey what design elements work well, what works best, and what should not be repeated in future buildings.

POE efforts in the United States and abroad were primarily focused on government and other public buildings from the 1960s to the mid-1980s. Private sector organizations in the United States became more actively involved with POE after the release of Using Office Design to Increase Productivity (Brill et al., 1985), which linked features of the office environment with worker productivity. As corporations downsized and reengineered their business processes to focus on core competencies, chief executive officers began to think of their buildings as ways to achieve such strategic goals as customer satisfaction, decreased time to market, increased innovation, attraction and retention of high-quality workers, and enhanced productivity of work groups. A number of organizations have since used POE as a tool for improving, innovating, or otherwise initiating strategic workspace changes.

As POEs have become broader in scope and purpose, POE has come to mean any activity that originates out of an interest in learning how a building performs once it is built (if and how well it has met expectations) and how satisfied building users are with the environment that has been created. POE has been seen as one of a number of practices aimed at understanding design criteria, predicting the effectiveness of emerging designs, reviewing completed designs, supporting building activation and facilities management, and linking user response to the performance of buildings. POE is also evolving toward more process-oriented evaluations for planning, programming, and capital asset management.

As a consequence, there is no industry-accepted definition for POE; nor is there a standardized method for conducting a POE. Even the term POE has come under scrutiny. Academics and others working in the field have proposed new terms, including environmental design evaluations, environmental audits, building-in-use assessments, building evaluation, facility assessment, post-construction evaluation, and building performance evaluations in an effort to better reflect the objectives and goals of POEs as they are practiced.

Traditionally, POEs are conducted using questionnaires, interviews, site visits, and observation of building users. Over time, more specific processes, levels of surveys, and new technologies have been developed to better fit stakeholders’ objectives and budgets. Shortcut methods have been devised that allow the researcher or evaluator to obtain valid and useful information in less time than previously.

Use of the Web and other technologies could substantially change the methods for conducting POEs and for analyzing the data generated. Web-based cyber-surveys may become the primary survey instrument, owing to their convenience, low cost of distribution and return, ability to check for errors and receive data— including rich-text replies—in electronic format, and ease with which respondents can receive feedback. Two U.S. federal agencies have already begun moving in this direction. The Public Buildings Service of the General Services Administration is working with the Center for the Built Environment at the University of California, Berkeley, to develop a set of POE surveys that can be administered over the Web. Different surveys are directed to different key personnel to help determine if GSA is meeting a number of key management indicators. The Naval Facilities Engineering Command is modifying its database to integrate corporate management systems and to Web-enable its POE survey. The POE survey will draw information from the management system and alert individuals when the surveys should be administered.

For organizations seeking to link facility design and business goals, a POE approach could be used that combines assessment of the physical condition of the building and its systems with assessment of user comfort on such topics as indoor air quality and ventilation rates, lighting levels and contrast conditions, building (not occupant) noise levels, and indoor temperature (thermal comfort). Results from subjective or instrument measures could be plotted on floor plans using geographical information systems. The data could then be analyzed individually or as overlays showing the spatial distribution of a range of factors. For example, ratings of thermal comfort could be assessed with temperature data and spatial location. Occupants’ perceptions of interior environments could also be linked with data from building control systems, local weather conditions, or facility usage as recorded by building-access smart cards. The Disney Corporation and the World Bank both have linked POE data to their geographic information systems for future planning and design purposes.

Depending on the type of survey conducted and the level of analysis used, the cost for a POE survey can range from a few thousand dollars per facility to U.S. $2.50 or more per square foot of space evaluated. Federal agencies have reported costs ranging from $1,800 for a simple standard questionnaire that could be completed in one hour to $90,000 for an in-depth analysis, including several days of interviews, the use of multidisciplinary teams, site visits, and report writing. Today the range of methods for conducting POEs allows an organization to tailor the technique to its objectives and available resources (time, staff, and money). Web-enabled surveys are emerging, and these may provide another technique that can be used at a relatively low cost.

Stakeholders in buildings include investors, owners, operators, designers, contractors, maintenance personnel, and users or occupants. A POE process that captures lessons can serve many purposes and provide many benefits, depending on a stakeholder’s goals and objectives. These include the following:

• support of policy development as reflected in design and planning guides. The validity of underlying premises used in recurrent designs can be tested and evolutionary improvements to programming and design criteria can be identified and incorporated into standards and guidance literature.

• provision to the building industry of information about buildings in use by improving the measurement of building performance by quantifying occupant perceptions and physical environmental factors.

• testing of new concepts to determine how well they work in occupied buildings.

• generation of information needed to justify major expenditures and inform future decisions. Information generated by POEs can be used for decision-making in the pre-design phase of a new

project to avoid repeating past mistakes. It can also be used to educate decision makers about the performance implications of design changes dictated by budget cuts and to improve the way space is used as determined by stakeholders or documented standards.

• improvement of building performance throughout the life cycle. POEs can be used to identify and remediate such problems associated with new buildings as uncontrolled leakage, deficient air circulation, poor signage, and lack of storage. For facilities that incorporate the concept of adaptability, where changes are frequently necessary, regularly conducted POEs can contribute to an ongoing process of adapting the facility to changing organizational needs.

• making design professionals and owners accountable for building performance. POEs can be used to measure the functionality and appropriateness of a design and establish conformance with explicit and comprehensive performance requirements as stated in the functional program. They can also serve as a mechanism to monitor a building’s quality and to notify decision makers when the performance of a building does not reach an agreed standard.

• aiding communications among stakeholders such as designers, clients, facility managers, and end users. Through active involvement in the evaluation process, the attitude of building occupants can be improved and proactive facility management that responds to building users’ values can be facilitated.

Despite these benefits, only a limited number of large organizations and institutions have active POE programs. Relatively few organizations have fully incorporated lessons from POE programs into their building delivery processes, job descriptions, or reporting arrangements. One reason for this limited use is the nature of POE itself, which identifies both successes and failures. Most organizations do not reward staff or programs for exposing shortcomings. In addition, relatively few organizations have created appropriate conditions for learning (i.e., conditions that allow the organization to constantly improve the way it operates under routine conditions and to respond to change quickly and effectively when the need arises).

Additional barriers to more effective use of POE and lessons-learned programs include the following:

• the difficulty of establishing a clear causal link between positive outcomes and the physical environment. This lack of a clear link can make it difficult for POE proponents to convince decision makers that the benefits received will justify the expenditure of time and money on the evaluations.

• reluctance by organizations and building professionals to participate in a process that may expose problems or failures or may be used as a method to focus (or deflect) blame. For federal agencies, senior executives may be concerned that identifying problems may be considered a weakness by Congress or the inspector general.

• fear of soliciting feedback from occupants on the grounds that both seeking and receiving this type of information may obligate an organization to make costly changes to its services or to the building itself.

• lack of participation by building users. In some cases the reluctance to participate can be attributed directly to uncertainty about senior management’s commitment to the program, which may be manifested by lack of resources or visible endorsement of the program.

• failure to distribute information resulting from POEs to decision makers and other stakeholders.

• pressure to meet design and construction deadlines, which can create a time barrier to sustained POE activity. Staffs may be so focused on future projects and ongoing construction that POEs for completed and occupied projects are given lower priority.

• lack of in-house staff having the wide range of skills and technical expertise needed to direct and manage the results of evaluations and to communicate the information so that it is useful and non-threatening. Organizations may be reluctant to hire consultants to conduct and analyze POEs if resources are limited and there is a lack of executive-level commitment to such programs. For federal agencies it may be difficult to obtain or earmark the funding needed to conduct POEs regardless of whether they are using consultants or in-house staff.

• organizational structures can create barriers when responsibilities for POE administration and

lessons-learned database development are assigned to different offices, thereby creating a need for interoffice collaboration and blurring the lines of accountability.

Despite the barriers mentioned above, POE has continued to grow as a practice. Some organizations have been able to effectively integrate the lessons of POEs into strategic planning and decision-making processes for facility delivery and acquisition. Notable examples include the following:

• the U.S. Army Corps of Engineers program of participatory programming and evaluation during the 1970s: The effort, undertaken after research indicated that aging facilities were an impediment to recruiting and retaining soldiers for the all-volunteer Army, resulted in design guides for facilities ranging from drama and music centers to barracks and military police stations.

• the U.S. Postal Service program: In the 1980s the newly reorganized U.S. Postal Service made extensive use of POEs to gather information about buildings to develop retail-focused postal stores to better compete with private sector companies. The program proved successful in meeting its objectives. Over time the survey methodologies have been modified to support new objectives, but the Postal Service program remains active.

• the Disney Corporation, which uses three evaluation programs and corresponding databases to explore optimal conditions and develop predictors of its key business driver, intention of the customer to return: The databases are used extensively in the design and renovation of buildings because they allow Disney to make direct links between inputs, such as proposed number of people entering gates, and outputs, such as the width of Main Street.

• the Massachusetts Division of Capital Planning and Operations, which links POE with pre-design programming of public buildings: POEs have been used to develop and test prototype concepts for state police stations, child care centers, and vehicle maintenance and repair stations, resulting in savings of cost and time in the programming, design, and construction of new facilities.

• Bell Canada and the World Bank: Both organizations have attempted to use POE as an asset management tool for space planning. Both companies collected large amounts of data from occupants and calculated baseline scores on seven comfort factors across all buildings. Real estate staff could then identify which spaces exceeded baseline scores and which fell below. These factors are considered in budgeting for maintenance and repair, space planning, and the reconfiguration of space. The World Bank also made an effort to link its database with computer-aided design drawings so that baseline scores of buildings, or floors of buildings, could be used as indexes of quality in its space-planning process.

One objective of this study was to identify a standardized methodology that could be used by federal agencies to assure consistency in data gathering and allow for cooperative development of benchmarks and best practices. As outlined above, POEs can serve a variety of purposes and the methods used for POE surveys can be tailored to the specific purpose and available resources. It is not clear that a standardized methodology for POEs that could be used for benchmarking across federal agencies would be effective or even desirable. However, based on the information in the following chapters, it is evident that organizations seeking to establish or restructure POE programs need to make a number of key decisions in the early planning stages and develop and incorporate several key components in their program if they are to be successful, regardless of the POE purpose or methodology. These decisions and components are identified below.

• Develop a clear statement about what the organization wants to achieve by conducting and applying POEs. The links between evaluations and stated requirements have to be explicit and easy to trace.

• Identify the resources available to carry out the POE, matching data collection and analysis activities to the available time and budget.

• Identify the likely users of POE results and determine how they need the results communicated.

• Garner support from high levels of the organization to signal the importance of the project or program to people throughout the organization.

• Determine if it is to be a one-of-a-kind case study or a standardized approach that allows building professionals to collect modest amounts of comparable data from a variety of buildings on a comparative basis over time.

• For a standard approach, capture and present the information that is fed forward from such activities in comparable formats. Develop accepted terminology, standard definitions, and normalized documentation at the outset to make such comparisons easier.

• Have the questionnaire designed and analyzed by someone skilled in survey research. Determine what indicators will be selected for measurement. Evaluate such measures for usefulness, reliability, validity (they should measure what they purport to measure), efficiency, ability to allow small changes to be noticed, and balance. The entire set of measures should include both quantitative and qualitative measures, as well as direct and indirect measures.

• Consider POE techniques that avoid direct questioning of users, for example, using data generated by building control systems, observations, and expert walk-throughs. The evaluation process will fail if occupants are reluctant to participate or if staff resources are insufficient to help with the organizational data gathering or for other measures.

• Decide whether user survey data will be made available to building occupants and, if so, in how much detail and for what purpose.

• Inform facility occupants about the purpose of their involvement in providing feedback and how the data will be used. They need to be assured that their own input will be kept confidential and should be informed if immediate correction of problems is not envisioned. Occupants are more likely to be engaged in the process if they are involved in helping design the measurement plan and if they see a benefit from participation.

Issues related to applying lessons learned from POEs are only partly technical. Tools for creating Web sites and databases are now widely available and relatively inexpensive. Optimizing the value of POEs requires the initiative to collect the information, the time to make sense of it, and the will to share it. To successfully incorporate the lessons from POEs into capital asset management processes and decision making, organizations need to do the following:

• Gain the support and long-term commitment of senior management.

• Create broad opportunities for participation and reflection. Encourage the direct participation of building owners, decision makers, designers, customers, project managers, and staff in the evaluations. Where necessary, create incentives for participation, for instance, through contract clauses, “free” vacation days, or using POE results as part of a review of qualifications when selecting consultants and contractors.

• Provide access to the information for different audiences. Upper-level management, architects, engineers, project managers, clients, and building users have different levels of responsibility in the building process and require different information from POEs. Lessons learned through POEs should be presented in a variety of formats to meet the needs of various stakeholders; these formats can include databases, design guides, case study reports, and policy and planning documents.

• Create simple databases that can be accessed by key words and that use simple methods to display overall results to aid interpretation. Ideally, a database should include the design hypotheses and assumptions for each project, the specific measures used to test the hypothesis, before and after photos of the space, brief summaries of the data, some analyses of cost, size and materials, key lessons learned, connections to other studies, connections to the full research findings before and after, and recommendations for future designs.

• Identify the critical points in the building process at which information or a POE can help resolve a problem or issue of considerable importance to participants. Use the information gathered to develop or modify policy documents.

• Build on facility evaluations that are the subject of complaints or controversy. Focus on understanding the reasons for problems or failures, and use this information to modify programming or planning processes to avoid repeating the experience.

• Use POEs of innovative buildings or components to help decide whether such innovations should be considered for future buildings.

• Create protected opportunities for innovation and evaluation. Methods for doing this include sanctioning research with the clear understanding that not all innovations will be successful, starting small with projects that have an experimental component, and evaluating the results before applying them on a broader basis.

The use of performance-based approaches to facility acquisition and evaluation is a worldwide trend. Performance-based approaches require greater attention to the definition and description of purposes (demand and results) of a project or program, both in the short and long term, and to ways of measuring whether the desired results have been achieved.

Performance criteria for POEs for individual buildings are based typically on the stated design intent and criteria contained in or inferred from a functional program. Measures include indicators related to organizational and occupant performance, such as worker satisfaction and productivity, and safety and security, but may also include measures of building performance as perceived by users such as air quality, thermal comfort, spatial comfort, ergonomics, privacy, lighting comfort, noise (from the building and offices), and aesthetics.

A performance-based approach that could be used to measure the quality of services delivered by the facility in support of individuals or groups involves the use of scales created by the International Centre for Facilities. The scales have been approved and published by the American Society for Testing and Materials (ASTM) as ASTM Standards on Whole Building Functionality and Serviceability (ASTM, 2000). The ASTM standard scales include two matched, multiple-choice questionnaires. One questionnaire, used for setting workplace requirements for functionality and quality, describes customer needs (demand) as the core of front-end planning. The other, matching questionnaire is used for assessing the capability of a building or design to meet these levels of need, which represents its serviceability. It rates facilities (supply) in terms of performance as a first step toward an outline performance specification. In the pre-project planning phase, the scales can be used to determine if the proposed facility will meet the needs of the customer and, if not, the changes that can be made to improve the fit. Once the facility is built a second evaluation can be made to determine how closely the facility fits the original expectations.

Evaluating the performance of buildings as a financial asset within a portfolio or inventory of buildings is more difficult. One potential approach is the Balanced Scorecard system of measurement adapted to facilities. The Balanced Scorecard is a business tool that assesses four categories of performance: financial, business process, customer relations, and learning and growth (human resource development). The four categories are used to avoid overemphasis on financial incomes, to capture the full value of the product or process, and to balance levels of analysis from individual and group outcomes to higher-level organizational outcomes. Performance measures for facilities often overemphasize costs because there are few data to show linkages between facility design and business goals. Cost-focused strategies include reducing the size of work stations, using a universal plan with only a few work station options, eliminating private offices or personally assigned spaces, and telecommuting. A Balanced Scorecard approach, adapted to facilities, can answer questions such as the following:

• How can workplace design positively influence outcomes that organizations value?

• How can workplace design reduce costs or increase revenue?

• How can workplace design enhance human resource development?

• How can the physical workplace enhance work processes and reduce time to market?

• How can the physical work environment enhance customer relationships and present a more positive face to the public?

By asking these questions at the beginning of a design project the Balanced Scorecard approach can provide an analytical structure to the entire process, from conceptualization through evaluation and finally to lessons learned. POE results can be used to develop measures for all categories of performance and to evaluate the organization’s success in meeting its performance goals. A core set of measures can be used across facilities to gain a better understanding of the entire building stock, while other measures would be unique to the goals and objectives of the particular organization, department, or division.

Federal agencies that own, provide, use or manage large inventories of facilities face a number of challenges. They are responsible for delivering buildings that are safe, secure, sustainable, accessible, cost effective to operate and maintain, responsive to customer needs, and supportive of worker productivity. Agencies must also become more business-like in their practices, more accountable to the public, and to manage their processes such that the results are measurable (i.e., on time, within budget, and of a quality to last 50 years or longer). In most agencies, the staff resources to meet these challenges were reduced during the 1990s and agencies face further loss of technical expertise through retirements and attrition.

In this environment, POE and lessons-learned programs, appropriately designed and managed, can add significant value to federal facility acquisition processes. A constructed facility is a culmination of policies, actions, and expenditures that call for evaluation. POE and lessons-learned programs can provide a systematic method for assessing the impact of past decisions and using these assessments in future decision making. They can partially mitigate the loss of in-house staff by creating an institutional data base that remains with the agency through changes in management and attrition in personnel. They also provide an opportunity to improve user satisfaction and reduce the cost of design development by sharing information throughout an agency and with outside contractors.

Although POE and lessons-learned programs have been instituted in relatively few federal agencies, those agencies have reported significant benefits. The General Services Administration, the Administrative Office of the U.S. Courts, and the U.S. State Department, among others, have used lessons from POE surveys to improve the design of federal buildings, to lower operating and maintenance costs, and to provide quality assurance. The Naval Facilities Engineering Command and the Army Corps of Engineers, among others, have used POE data to design buildings that better meet user needs and help to support the retention of military and civilian staff. The U.S. Postal Service has used POE data to better meet its customer needs and, in so doing, the USPS has been better able to compete with private sector mail delivery services. The U.S. Air Force and others have used POE data for quality assurance in identifying building defects and repairing them early in the life of the building when it is most cost-effective to make such repairs. The GSA and NAVFAC are restructuring their programs to better link POE to a wide range of facility management processes as federal agencies shift their emphasis from facility engineering to capital asset management.

As Web-based surveys, building controls, and geographic information systems continue to evolve, conducting POE programs and capturing lessons should become easier and yield more useful data. Instituting these programs in additional federal agencies in accord with the guidelines highlighted above should result in benefits that outweigh the costs. However, establishment of these programs will require leadership from both senior executives and program managers and a willingness to learn from both successes and failures.

ASTM (American Society for Testing and Materials). 2000. ASTM Standards on Whole Building Functionality and Serviceability. West Conshohocken, Pa.: ASTM.

Brill, M., S.M., Margulis, and E., Konar, 1985. Using Office Design to Increase Productivity (2 vols.). Buffalo, N.Y.: BOSTI and Westinghouse Furniture Systems.