The federal government is the largest owner of facilities in the United States. More than 30 individual federal agencies own, use, and acquire facilities to support agency missions and programs. Some federal agencies conduct post-occupancy evaluation (POE) and lessons-learned programs as ways to improve customer satisfaction, to increase building quality and performance, and to facilitate organizational learning. This chapter provides information about POE processes in six federal agencies: the U.S. Air Force, Office of the Civil Engineer; the General Services Administration, Public Buildings Service (PBS); the Department of the Interior, National Park Service (NPS); the U.S. Navy, Naval Facilities Engineering Command (NAVFAC); the U.S. Department of State, Office of Overseas Buildings Operations (OBO); and the U.S. Postal Service (USPS).

The six agencies are sponsors of the Federal Facilities Council and volunteered to participate in the study. The information was gathered through a questionnaire and telephone interviews conducted by Krista Waitz of Kwaitz Consulting. National Research Council staff wrote the summary of findings and descriptions of POE programs.

The study design was not a scientific one, nor was it based on random sampling. Thus, the information provided should not be generalized. The remainder of this chapter contains information about the survey questions, a summary of findings, and descriptions of the POE programs in the six agencies.

In January 2001, a questionnaire was designed and issued to six sponsor agencies of the Federal Facilities Council who volunteered to provide information about their post-occupancy evaluation processes. In some cases, more than one person in the agency responded and the responses were combined and reconciled. The agency representatives were asked to respond to the following questions:

1. Approximately what year did your agency establish a post-occupancy evaluation program? On average, approximately how many POEs have been conducted by your agency in each of the last five fiscal years?

2. What were the driving factors for establishing a POE program?

3. What is the focus of the POE information-gathering process (e.g., user satisfaction, achievement of design objectives, building performance, other)?

4. What are the expectations, goals, and objectives for the program? Have they been achieved? Have there been unanticipated results?

5. How is the information gathered through POEs tied into the feedback loop (lessons learned) for planning, programming, and capital asset management?

6. Is the information gathered through POEs used in real estate decision-making and capital asset management? If yes, please note what information is used, how, and when it is used. If no, why not?

7. Is it your agency’s policy to conduct POEs for all buildings or for selected facilities? What are the criteria for determining whether a POE will be conducted?

8. What do you consider to be barriers to conducting successful POEs?

9. Who is typically involved in conducting a POE in terms of in-house personnel and consultants? (Please list positions or types of skills involved not individuals.) What types of technologies are used?

10. What is the estimated cost in time and dollars for conducting a typical POE?

11. What data collection methods, technologies, and survey forms have been used over the life of the program? Please provide copies of survey forms that have been used.

12. To what extent does your agency make use of data management systems, Internet tools, or other information technology applications to share information and disseminate results of POEs?

Each of the six agencies studied had had a POE program in place at least since the 1980s. The POE programs of the PBS, NPS, and NAVFAC were being restructured to meet new objectives, and the results of the reorganized programs were not yet available.

The number of POEs conducted annually, on average, ranged from less than 1 to 30. Post-occupancy evaluations are typically performed within 4 to 24 months following occupancy of a new or renovated facility and are performed only once for an individual building.

Each of the six agencies used POEs to determine client or user satisfaction at some level, but it also used them to fulfill other objectives. These objectives included determining building defects within the construction warranty period, supporting design and construction criteria, supporting performance measures for asset management, evaluating construction inspectors, lowering facility life-cycle costs by identifying design errors that could lead to increased maintenance and operation costs, clarifying design objectives, improving building performance, and supporting corporate sales and image objectives. The restructured programs in NAVFAC and PBS are focused on developing metrics for client satisfaction and for management-related issues.

No two agencies use the same process or tools for conducting POEs and capturing lessons, although some share common elements. The National Park Service is in the process of developing new procedures and tools for conducting POEs and sharing lessons learned. Currently, at the NAVFAC, an independent agent remote from the designer of record conducts the POE using a statistically based questionnaire; a focus group discussion is then conducted to summarize the results of the survey. The NAVFAC criteria office administers survey results. The questionnaire is being modified so that it can be administered from field agencies and can be completed on-line or downloaded from the Web. The survey documents and results can be downloaded from a database on the Web.

The Air Force, in contrast, uses a questionnaire administered by a staff team; feedback is given primarily to the construction agent, although the Air Force plans to also share the results with users, the base civil engineer, and the major command.

The Office of Overseas Buildings Operations uses pre-trip user questionnaires, on-site interviews, and facilitated town meetings to gather the information, which is then summarized in a formal report. OBO’s POE is conducted by a multidisciplinary in-house team that is customized to address known deficiencies.

The PBS performance measures-oriented approach uses a set of questionnaires developed in cooperation with the Center for the Built Environment at the University of California at Berkeley. The questionnaires are being designed to be administered over the Web. The lessons learned will provide input to the ongoing performance measures program of the Office of Business Performance.

The U.S. Postal Service uses two levels of surveys. The first, a basic questionnaire that can be completed in about an hour by the administrative service officer and the postmaster of a new facility, is required for all new construction. For larger, more complex projects, POEs are conducted over two to four days using a multidisciplinary team. Both types of surveys use electronic questionnaires in Microsoft Excel. The information gathered is sent directly to the staff maintaining the agency’s design standards.

Information obtained from POE programs has been used by the OBO, NAVFAC, PBS, and USPS in support of their design criteria and guidelines. The NPS and the Air Force programs anticipate using POE results for the support of design criteria, among other objectives. To date, none of the six agencies reported that POE information was used directly in future real estate decision-making and capital asset management, although PBS’s POE program was being restructured with those objectives in mind.

A number of barriers to more effective use of POEs and lessons-learned programs were identified. These could be categorized generally as resources, feedback, and participation and commitment.

Several agencies noted it was difficult to obtain or earmark the funding needed to conduct POEs regardless of whether the POEs were to be conducted using consultants or in-house staff. In some cases, in-house staff may not be available to conduct the POE or may not have the technical skills needed for quality results.

Because POEs often focus on identifying deficiencies, they risk becoming instruments to focus or deflect blame for unsatisfactory results. One agency cited the concern by senior executives that lessons learned may be considered a weakness by Congress or the Inspector General. Other agencies noted that conducting a focus group to achieve consensus about the cause of failures without judgmental discussion can be difficult, and as a consequence, they may be reluctant to do so.

One agency noted that because of construction schedule constraints, staffs may be focused on future projects or those under visible construction. Thus, once a project has been completed and occupied, items such as financial closeout, construction as-builts, and POEs may not be a high-priority item and may not receive adequate oversight or attention. For programs administered through a headquarters’ office, there may be a lack of field-level attentiveness to the process. Obtaining the clients’ attention to ensure adequate participation in the survey or getting people not originally involved with the project to participate in a survey was also identified as a barrier. Organizational structures can also create barriers when responsibilities are assigned such that POE administration and database development require interoffice collaboration.

The costs reported to conduct POEs ranged from $1,800 for a simple standard questionnaire that could be completed in one hour to $90,000 for an in-depth analysis including several days of interviews, multidisciplinary teams, site visits, and writing up reports. The costs did not include implementation of any changes resulting from a POE study. Costs per square foot of space evaluated were not available. Other variables accounting for the range of costs included whether the facilities were located in the United States or abroad, whether in-house staff or consultants were used, and how the resulting information was packaged and distributed.

In the mid-1980s the National Park Service completed formulation and development of an extensive POE program; however, due to changes in staff and downsizing of the NPS central design office, the program was not fully executed. In 1998, the POE program was reinstituted as part of the business practices for the NPS central design office. At that time, it was decided to reestablish the POE program for three reasons: (1) to create a feedback loop that would allow designers to interact with facility users and learn if facilities were meeting the needs of the users; it also served as an opportunity for users to assess their original requirements and determine if they had adequately identified their needs; (2) to evaluate the effectiveness of consultant construction inspectors who had recently replaced all NPS construction inspectors; and (3) to improve long-term facility life-cycle ratio costs by identifying any design errors that could lead to increased maintenance or operational costs.

The focus of the NPS POE program is to improve user satisfaction, building performance, and designer efficiency. The central design offices for the NPS seek to retain highly talented designers over a long-term career. By building relationships with facility users, designers better understand user needs and anticipate requests, allowing for more efficient use of design funds. The POE program seeks to build on this type of relationship through open communications and on-site review of completed construction projects. The NPS expects the value-added component of the POE will be in lessons learned and improved design efficiency. The NPS noted that maintenance and operations costs for facilities are escalating annually, and if the POE provides information for future designs that lead to improved maintenance or operations, the payback will be dramatic. In addition, if the POE provides feedback on products or techniques that improve user satisfaction, reduce maintenance, or improve operations, this information can be shared with designers to reduce the cost of design development.

Data collection tools and the forum for sharing information are still under development. However, the POE program is envisioned to include an evaluation form and follow-up meetings approximately six months to one year after completion of construction. The POE meetings will include the users, project manager, and design team captain. After completing the evaluation, the materials will be shared within the project management and design divisions. The results of the POE will also be placed in the central technical information files and may be accessed by various levels of project managers, designers, and other technical staff.

The NPS anticipates its POE teams will consist of (1) users, including park superintendent, chief of maintenance, park rangers, park interpreters, and administrative staff; (2) project managers, a multidisciplinary group consisting of park planners, architects, landscape architects, and various disciplines of engineers—the technical expertise will vary by project; and (3) design team members. Generally a team captain for a building project would be a senior architect; for a road construction project, a senior civil engineer or senior landscape architect would attend the POE; and for a utility project, a senior civil engineer would attend. Depending on the complexity of the project, other members attending a POE meeting could include a mechanical engineer or an electrical engineer.

The Naval Facilities Engineering Command has conducted POEs since the 1960s. In its original form, NAVFAC’s POE program had an instructional base and was conducted by a project team after construction to identify lessons learned. In 1997, a new statistically based concept was initiated. The goal is to establish a statistical basis from which NAVFAC can measure agency improvement and work toward continuous improvement by implementing process changes and modifying design criteria. The POE program is intended to apply to each completed facility within its warranty period. On average, 20 POEs have been conducted in each of the last five fiscal years.

There were two driving factors for restructuring the POE program at NAVFAC. The first was a published Department of Defense-level survey of occupant satisfaction with their facilities. The second was a NAVFAC headquarters’ management initiative to create metrics measuring how, or if, the agency exceeded client expectations. The agency focuses on improving client satisfaction, determining where its product lines or processes give rise to client dissatisfaction and improving the individual facility on which the survey is conducted. NAVFAC is issuing policy that will require a survey of all facilities within 6-10 months of building occupancy. The focus of the POE information-gathering process is to measure user satisfaction from the perspectives of the building owner, the customers (student, family occupant, day care family member, etc.), the building occupant at the working level, and the staff maintaining the facility. Client satisfaction at the user level is measured whether or not the client was involved in the planning, design, or construction phases. The measuring process includes a survey of participants after which a focus group discussion is conducted to summarize the positive and negative aspects of the facility.

Currently, all POE surveys are conducted by an “independent agent” remote from the designer of record, typically a consultant. The NAVFAC criteria office administers the surveys. The criteria manager for the facility type identifies criteria issues and action(s) to be initiated and implements interim guidance to the organization when required. The survey will be used in developing and modifying planning and design criteria. The data will be reviewed annually to determine process improvement needs. NAVFAC is

working to determine how the survey will interface with its knowledge management system, combining to yield a single lessons-learned concept for the organization.

When the POE program process began, NAVFAC used a checklist format focusing only on design and construction. The survey was changed to accommodate all of the agency’s processes (i.e., planning, design, construction, and maintenance turnover). Additional minor changes are anticipated to better assess safety and procurement issues. A copy of the survey is contained in Appendix D.

The database is being modified to Web-enable the survey content and to create a field-managed site as opposed to a headquarters central database. In addition, the database is being made integral to corporate ORACLE-based management systems. It will draw information from the management system and alert assigned individuals when it is time for the survey.

The focus of the Air Force POE information-gathering process is user satisfaction, achievement of design objectives, and improved building performance. The purpose of the POE is to note all defective work, report construction deficiencies to the construction agent for correction by the contractor, and document problems or mistakes made during design for use as lessons learned on similar projects.

The POE is conducted by a staff team using a questionnaire. The Air Force plans to share the results of the POE with the construction agent, the user, the base civil engineer, and the major command for use in any future designs and for incorporation into Air Force design standards. It is Air Force policy for a POE to be accomplished sometime during the ninth to eleventh month following beneficial occupancy (acceptance of the facility by the user agency).

At the Office of Overseas Buildings Operations (formerly Foreign Buildings Operations) of the U.S. Department of State, lessons learned and design or construction alerts have been issued since 1985. The driving factors for establishing a POE program were the concern for user satisfaction, comfort, and safety and a general desire to capture best practices. The focus of the POE information-gathering process is on user satisfaction, achievement of design objectives, and building performance, including interior flexibility and functionality.

The POE methodology followed was adapted from the U.S. Postal Service. Over the life of the POE program, occupant surveys, on-site interviews, and facilitated town meetings have been used for data collection. The first step in the POE process is to send an occupant survey to an overseas post. The preliminary results from the occupant survey are used to determine what disciplines should be represented on the multidisciplinary team that will be conducting the POE; thus, the team is customized to address known deficiencies. Typically, architects; electrical, mechanical, and structural engineers; facility maintenance; and security specialists are involved in conducting POEs. Once at the site, the team conducts a walk-through followed up by interviews with occupants. At the conclusion of the site visit, members of the team reconvene to discuss their observations and to generate recommendations for a report. Due to constrained resources, a POE may consist solely of the occupant survey.

One result of the POE program has been the revision of design guidance on such topics as roofs, elevators, and Ambassadors’ residences. POE results were also used for developing a serviceability demand profile for generic embassy office buildings slated for design and in design guidelines for future embassies being acquired under a specialized procurement process.

The Public Buildings Service (PBS) of the General Services Administration (GSA) first instituted a POE program oriented toward design and construction criteria development in 1977. In-house technical experts, including an environmental psychologist, architects, and engineers, conducted the surveys. The program was curtailed in 1982 due to a reduction in staff. PBS’ POE program was reinstituted in 1986, using contractor support. Between 1986 and 2000, the PBS completed approximately 30 POEs for a variety of projects, including courthouses, office buildings, U.S. border stations, major renovations, and historic restorations. In 2000, the PBS restructured its POE program to focus on performance measures for asset management. Both the design criteria-oriented POE program and the performance measure-oriented POE program were

intended to provide an information stream that would inform program managers, criteria managers, and project managers about design- and delivery-related problems and associated best practices.

Criteria-based POEs were developed to provide technology- and procedure-based feedback to those in PBS’s central office responsible for national program and design criteria direction. Equally important were the perceived benefits of offering those same lessons to the delivery teams responsible for new projects. The primary focus was on building systems evaluation, client satisfaction surveys, and interviews with major client agencies. The building systems evaluations included functionality issues as well as an overview of how well the building complied with design criteria. The surveys were customized for different types of buildings and project delivery systems, including office buildings, courthouses, border stations, and lease-build or design-build projects. (An example is included in Appendix D.) Major lessons from the POE process related to long-term building maintainability, building functionality, client needs, and property manager and asset management needs. Specific issues concerned energy efficiency; indoor air quality; heating, ventilation, air conditioning, and electrical systems; thermal comfort; design of loading docks; access to equipment; window washing; and accessibility for the physically disabled. The criteria were constantly revised to incorporate lessons from POEs.

From 1986-2000, criteria-based POEs typically involved a team of five to six design-related disciplines, using outside architect-engineer professional services. PBS central office coordination and involvement were provided through site visits, access coordination, and report critiques. A report and a 30-minute video were created and distributed to each of GSA’s field offices (11 regions) and to senior-level management at headquarters.

The tieback to lessons learned was direct: the people who conducted the POEs also developed the criteria. However, because the POE reports were voluminous and oriented toward detailed evaluations of technology applications, getting project managers and designers to read and adhere to findings was a challenge. Thus, various forms of information exchange media were applied to mitigate this problem, including condensed videotapes and interactive “lessons-learned” compact disks (CDs). A compendium of lessons was distributed every three years. In 1998, PBS prepared a Compendium of Lessons Learned CD-ROM that was widely distributed, and a DVD (Lessons Learned, Volume 2) is being prepared for mass distribution. The POEs are also in the construction criteria database of the National Institute for Building Science.

A second program that resulted from the POE lessons is HVAC (heating, ventilation, air conditioning) Excellence in Federal Buildings. PBS held numerous workshops that included staff, architects, engineers, and representatives from professional societies and technical organizations as part of an awareness program to highlight HVAC issues.

The driving factor behind the shift to a performance measure-oriented POE program was PBS senior management’s desire to evaluate how well PBS’s assets are achieving their objectives on a project and program basis. Performance measure-oriented POEs are being pursued to help indicate whether delivery practices and criteria are effective and to identify systemic problems, whereby specialty studies could be pursued.

The focus of this POE information-gathering process is financial asset assessment. A set of extensive questionnaires is used in an attempt to identify customer satisfaction with various building components or features. Different questionnaires are directed to different key personnel, including operating staff and design-delivery team members. The measures are intended to help determine if GSA is meeting a number of key management indicators including comparison of construction “pro forma” with final pro forma, maintenance and cleaning costs benchmarked against national standards, utility costs, sustainability, energy usage against FY 2010 goals, accessibility for the physically disabled, and client satisfaction.

Support for questionnaire development and database management is currently provided by the Center for the Built Environment, within the University of California at Berkeley. Questionnaire delivery and assessments are being coordinated by senior architects and engineers within the PBS Office of Business Performance. PBS is currently designing tenant, operations and maintenance, and design and construction survey tools that can be administered over the Web.

The goal is to perform a POE for every Congressionally approved new building one year after full

occupancy. Once the performance measure-oriented POE program is fully implemented, 10-20 POEs will likely be completed each year. The tieback from the ongoing performance measure POE program will go from PBS’s Office of Business Performance to other appropriate offices within the agency.

The U.S. Postal Service established its POE program in 1986. The driving force behind the establishment of the POE program was the desire to improve the planning, design, and construction of future facilities. The focus of the POE information-gathering process is user satisfaction (customers and employees), clarification of design objectives, achievement of design objectives, building performance relative to technical systems such as cooling and lighting, and supporting corporate sales and image objectives and economics. On average, the agency has conducted approximately 30 POEs in each of last five fiscal years.

The first POE application was an effort to standardize the hundreds of Postal Service retail spaces produced each year. Appropriate design was found to be a much more powerful factor in customer satisfaction than had been anticipated. Also, building image was a much more powerful support for overall corporate identity than previously realized.

Two levels of POEs are used currently. A basic POE (completing the questionnaire) is required for all new construction and for owned facilities greater than 9,000 square feet within four and six months of occupancy. The administrative service office manager and the postmaster complete the basic POE questionnaire, which typically takes 30 minutes to an hour. An example is included in Appendix D.

More extensive POEs are conducted on larger projects (more than 30,000 square feet) or other special projects. These POEs involve architectural or engineering firms (including environmental psychologists as consultants) and are conducted over a period of two to four days. Customers and employees are interviewed, and extensive lighting and HVAC data are gathered.

The results of POEs go directly to the staff maintaining the Postal Service Building Design Standards. The information gathered through the POE process is not used in real estate decision-making or capital asset management; however it is used in planning, design, and construction decisions. Real estate decisions are affected only as site-planning criteria are modified.