CHAPTER 2

Quality Management Concepts

Quality management is a mature practice at state agencies. However, transitioning to paperless and digital delivery of design information exposes a vulnerability in how quality management is implemented. For decades, the quality management process has been focused on and formalized around reviewing plans, with less attention paid to performing and documenting the quality management process on the digital CADD files used to create plans. This chapter provides a refresher on core quality management principles for two reasons: to provide a resource for individuals who are new to implementing quality management on agency data, and to offer justification for guidance in later chapters.

2.1 Quality Management Principles

Quality management principles guide organizations in their pursuit of continuous improvement and excellence. Comprehensive quality management touches everyone and is not the responsibility of a single organizational unit. Incorporating quality management principles into the development and review of 3D models within state agencies enhances project outcomes and increases a project’s long-term value. These principles are the foundation upon which a robust and reliable quality management program is built.

2.1.1 International Standards for Quality Management Systems

The ISO 9000 series is a set of international standards for quality management systems, and it provides a generic framework that is applicable to many industries. The ISO 9000 series explains the fundamentals of quality management systems and the seven quality management principles that promote organizational improvement. The principles cover customer focus, leadership, engagement, process approach, improvement, decision-making based on evidence, and relationship management.

The ISO 9001 standard establishes requirements for organizations to be certified as compliant with the standards. ISO 9001 also introduces the Plan-Do-Check-Act (PDCA) cycle as a process for implementing and continuously improving a quality management system (Figure 4).

In the plan phase, an organization describes the program context, including identifying how stakeholders define quality. Leadership needs to commit to the quality management system and provide the resources necessary to develop, implement, and maintain it. The organization also prepares to implement the program and designs the quality management system to address risks and achieve targeted quality outcomes.

In the do phase, the organization implements the quality management system. Staff receive the resources, training, and documented information needed to operate the quality management program. Staff implement these resources on products that the organization produces.

In the check phase, the organization measures outcomes of the quality management program and evaluates its performance.

In the act phase, the organization commits to program improvement by preparing to resume the plan phase, at which time it will implement feedback received in the check phase.

2.1.2 Quality Management for Software Products

The International Software Testing Qualifications Board (ISTQB) has seven principles of quality management for software development that can be used as a methodological basis for 3D model reviews. While the processes of software development and testing differ from those of design, the ISTQB principles are broad, and they complement the ISO 9000 standard. These principles should be considered when assessing review tasks performed by humans as well as automated tasks within review software tools. The seven ISTQB principles to consider when developing a quality management system for 3D model review are

1. Testing shows the presence of defects, not their absence.
2. Exhaustive testing is impossible.
3. Early testing saves time and money.
4. Defects typically cluster together.
5. Routinely review the testing regimen as the context changes.

6. Testing is context dependent.
7. Absence of errors is a fallacy.

The ISTQB principles reinforce the need to utilize the PDCA cycle in quality management systems as follows:

• In the plan stage, capture the context, including the hallmarks of quality, the risks and consequences of quality failures, and the types of defects that can arise. Establish a framework that includes intervening early and establishing resources by combining process and product controls to detect known defects.
• In the do stage, note that defects typically cluster around newer areas of the design that have been developed or modified since the last milestone, and focus efforts in those areas.
• In the check stage, track the types of defects identified and those that slip through the cracks, along with the consequences of those issues. Evaluate how much time is being spent detecting different types of deficiencies. Be skeptical of claims that defects were not found, and probe deeper.
• In the act stage, revisit the context of the organization and the design process. Evaluate the quality management system’s performance. Prepare to update the system to better address the new context as well as allocate resources where the risks are greatest and where new types of defects have been identified.

While agencies are not responsible for establishing a robust system for software testing, agencies could develop a process for evaluating and validating software that has the functionality needed to produce quality designs or conduct quality management reviews. Section 6.4 provides suggestions for existing and future software functionalities.

2.1.3 International Standards for Information Management

The ISO 19650 standards define information management using BIM, including principles and concepts (ISO 19650-1), requirements for delivery (ISO 19650-2) and operational phases of assets (ISO 19650-3), information exchanges (ISO 19650-4), and security (ISO 19650-5). This family of standards identifies key fundamental attributes, governance, and strategies that are significant for the quality management of 3D models and their application in design development. These standards define the processes for sharing and delivering information, including validation and authorization activities during the delivery phases of a project. The standards also define the CDE requirements for security and information quality.

The ISO 19650 series exemplifies principles outlined in ISO 55000 (asset management overview, principles, and terminology) and ISO 9001 (quality management systems requirements). Asset information management principles, defined in ISO 55000, focus on the owner’s responsibilities, leadership, and governance in relation to assets. Additional key principles, defined in ISO 9001, focus on implementing the PDCA cycle of change management. These concepts and principles can be used to establish and maintain a quality management system for information exchanges.

Throughout this guide, specific sections highlight and expand upon the relevance and implementation of ISO 19650. ISO 19650-2 (Part 2), in particular, provides guidance for project development and delivery activities, including project planning, production methods and procedures, CDE solutions and workflows, and assessment of the project team’s capabilities and capacity. Creating information models based on the requirements defined by these standards requires specific QA and QC checks. Projects need to follow the information management process based on principles of information requirements, using CDE solutions and workflows. These workflows include storing files in the CDE, which requires a managed process for version

• A description of the overarching process (i.e., who does what, when, and how); and
• A description of how to use the resources that form the quality management framework, such as templates for project quality plans, checklists, software standards, quality artifact–documentation standards, and templates for certification.

2.2.2 Standards for Software

Software standards are one way that agencies implement process control. If an agency standardizes the use of a software product, it is usually because the agency has thoroughly tested the software and any customizations. Consistent and predictable project outcomes are more likely when the production and use of digital data follow a standardized process and employ standardized software configurations as well as analytical modeling software with well-vetted configurations. When establishing an agency’s standards and adoption of new software, it is important for agencies to take advantage of process control during review and implementation.

These configurations can include libraries and resources that, when used correctly, result in a design that complies with an agency’s CADD standards and the design standards that the CADD environment was configured to implement. For 3D modeling software, these customizations and collections of libraries are called a workspace or software configuration. Typically, an agency’s CADD manual or model development standards manual (MDSM) specifies the use of a specific software configuration.

2.2.3 Project Quality Plans

The project quality plan is a detailed plan for executing the quality management framework on a project, including a defined review schedule with durations and budgeted hours. It identifies the individuals responsible for every role in the process for each milestone review, as well as checklists and other resources to support each of these reviews. A project quality manager should take into consideration the 3D modeling skill set needed for executing the reviews when assigning responsibilities, and they should plan for any training that may be needed. For paperless review documentation, there is an opportunity to link directly to the reviewed models, using the project quality plan as a launchpad for quality audits.

Projects that adopt 3D model–based design often use a BEP or model development plan to create and share the 3D models that underpin the design. Some BEPs include a section on model reviews, whereas others focus specifically on the collaborative use of 3D models and precisely define the information exchanges at each milestone. The project quality manager needs to understand these information exchanges to determine what skill sets are needed to perform reviews. The project quality plan should contain references to the BEP, just as the BEP should contain references to the project quality plan. For example, the BEP routinely defines the modeling software that will be used to develop and review models.

2.2.4 Standards for Quality Artifacts

Quality artifacts provide an auditable record of how the quality management process was implemented on a project and document the scope and extent of the checks. They record the results of the checks and the act of checking itself. In the paper-based review process, a common practice is highlighting elements on a plan set in one color as they are checked and validated. A specific review scope defines what should be checked and highlighted, with different types of reviews producing multiple quality artifacts tracking different elements on the same plan set. A comment log that only tracks defects and their resolutions is insufficient; quality artifacts that record digital checks also need to provide evidence of checks where no defects were found.

Using standards for quality artifact documentation improves the consistency, repeatability, reproducibility, and traceability of the quality process. When the form, format, and media of quality artifacts follow a standard, the skills and software proficiency needed to implement and audit the quality management process on a project are predictable. Acquiring these core competencies is the foundation of training programs and enables resource sharing across projects.

Some considerations for establishing standards for quality artifacts include

• Software required to access the quality artifacts. Is it reasonable to expect that a potential auditor has access to the software? Is it possible for a data file to be corrupted if it is opened in a different version of the software?
• Software core competencies needed to access the documentation. Is it reasonable to expect that a potential auditor has these skills?
• The document retention period. If a quality artifact can only be accessed using proprietary software, is it reasonable to expect that the documentation will be accessible over its full retention period?
• The model versions associated with quality artifacts. Is it possible to retain a version of the 3D model associated with each quality artifact?

2.2.5 Audits and Certifications

Audits of quality artifacts verify that the quality management process was properly executed on the project. Some states require an audit and a certification attesting that the quality management process was properly executed to approve a design for letting. The audit could be the responsibility of the design manager or project quality manager, or it could be a responsibility reserved by the agency as part of the preparation for letting.

2.2.6 Monitoring and Feedback

Digital technologies are evolving rapidly as the industry transitions to 3D design and digital media for contract documents. As soon as this guide is published, any resources, checklists, or software-specific suggestions herein may need to be updated or replaced with new information. Because the technologies used to develop and review 3D models are still evolving, agencies need to continually monitor and update their quality management practices, including performance monitoring.

Processes for quality management such as performance monitoring, capturing feedback, and making routine updates to the quality management framework, including updates to review tools and job aids, allow for incremental changes in the quality management framework. Major milestones in digital transformation that should precipitate a larger update to quality management resources include implementing new design authoring or design review software and changes to the media through which information is delivered for construction.

Agencies that incorporate their quality management process into project management systems can introduce performance measures that are tracked automatically. These include

• Tracking each quality management milestone and monitoring across the program,
• Tracking certifications and monitoring compliance with policies related to letting,
• Providing surveys for auditors to assess whether the core competencies needed to properly audit the project fall within their regular skill sets, and
• Establishing an expiration date for quality management resources to push routine updates of the quality management framework.

While automation can help with data collection and performance monitoring, outcomes improve when responsibility for meeting the performance standards is assigned and properly resourced.

2.3 Project Execution

Applying quality management principles and practices on projects requires a clear definition of how to achieve quality in 3D models. Then a quality management framework can be implemented and incorporate 3D model–deliverable workflows.

2.3.1 Definition of Quality for 3D Models

ISO 9000 standards define quality as the degree to which a set of inherent characteristics of an object fulfills requirements. In terms of 3D model quality, the inherent characteristics are the degree to which modeled elements meet specific object attributes, such as defined structure, accuracy, integrity, constructability, and accessibility:

• The structure requirements of an object refer to defined development standards, such as LOIN, which is standardized as ISO 7817.
• The accuracy requirements of an object refer to the design criteria, such as standards and specifications, and the overall design intent.
• The integrity requirement of an object refers to the spatial integrity of an object, such as enclosed regions, contiguous line strings, and no clashes with other objects in space.
• The constructability requirement of an object refers to the ability to construct the object without issues.
• A model’s accessibility requirement refers to a third party being able to open a model and understand its contents, interpreting them as the designer intended.

2.3.2 Quality Process Framework

The goal of a project is to develop a design that complies with the defined standards and fulfills the project purpose and need while also meeting agency objectives for practical design, whole life-cycle cost, constructability, and project cost. The project manager is responsible for the design quality, but they may delegate this role to a design manager or project quality manager.

The QC process includes five steps: initiate, check, back check, update design, and verify changes.

QC Process

Initiate: After thoroughly checking and correcting their own work, the designer prepares the design deliverables for review and prepares the quality artifacts needed to execute and document the review. Transmit the packet to the reviewer.

	Check: The reviewer receives the review packet and executes the review, documenting the checks and any markups or comments. Upon completion of the review, the reviewer transmits the packet to the back checker.
	Back check: The back checker receives the packet and reviews the reviewer’s documentation. The back checker indicates their concurrence or input regarding the reviewer’s markups as well as any comments providing final instructions for the designer, then they transmit the packet to the designer.
	Update design: The designer makes the indicated updates and notes the updates on the quality artifacts. The designer then resubmits the packet to the person who will verify that the updates were made as indicated.
	Verify changes: The individual receives the packet and verifies that the updates were made as indicated, updates the quality artifacts, and returns the packet to the design quality manager.

The QC process is executed during the design phase of the quality management framework:

1. Start-up phase: Assign quality roles and develop a project quality plan.
2. Design phase: At each milestone, use resources identified in the plan to review the design and document evidence of the process. During the design phase, the resources are typically used in a five-step process (Figure 5).

3. Pre-letting phase: Review quality documentation to verify that the plan has been executed. Certify the design to attest that the quality management framework was properly implemented. Audit the quality artifacts to verify that they are complete and signed off.

The quality management framework identifies roles and responsibilities for the five-step process. The practice of assigning responsibilities for each step is not new—each agency may determine its own conventions. For example, the back checker mentioned in Step 3 of the QC process can be the original designer or independent of the designer. Appendix C provides additional context for who can fulfill specific roles within agencies.