APPENDIX B

Stakeholder Engagement Activities

Overview

Stakeholder engagement activities were conducted to validate and refine the list of RNs and gaps identified through the literature review. These activities consisted of an initial survey, two listening sessions with broad participation, three focus groups, and a series of individual interviews.

Survey

In preparation for the stakeholder engagement activities, a survey was sent to the AASHTO CKM member list, asking them to characterize their KM implementation status and to provide names and contact information for stakeholders who might be interested in participating in project outreach activities. Twenty-eight responses were received.

The following map (Figure B-1) was constructed based on the survey responses and knowledge of the research team members. (Those responding “it is complicated” were assigned to the discoverer, explorer or practitioner categories.)

Listening Sessions

Two listening sessions were held via web conference on December 15 and 20, 2022, to review potential KM research topics, obtain feedback and hear stakeholder ideas for new topics. To prepare for the listening sessions, a slide deck was created with background information on seven categories of research topics – both prior research efforts and potential future areas for further investigation.

Preparation

An email invitation (including the slide deck as an attachment) to the listening sessions was sent to a mailing list of 377 individuals, consisting of:

• Members of the NCHRP 23-14 panel
• Members of the AASHTO CKM
• Members and Friends of the TRB Committee on IKM
• Stakeholders from the AASHTO CKM survey

In addition, the invitation was sent to chairs/managers of several other groups with a request to forward along to members of the:

• AASHTO Agency Administration Managing Committee
• AASHTO Committee on Human Resources

• TRB Statewide/National Data and Information Systems Committee (AED10)
• TRB Artificial Intelligence and Advanced Computing Applications Committee (AED40)
• TRB Workforce Development and Organizational Excellence Committee (AJE15)
• TRB Research Innovation Implementation Management Committee (AJE35)

The email invitation asked recipients to forward it on to colleagues who might be interested, so the reach was potentially broader than the above groups.

Participation

The two listening sessions had between 30 and 35 attendees each. The invitation provided a link to an online survey that could be used to provide input on RNs instead of or as a supplement to attending the listening session. Ten responses were received.

Participant Input

Following an overview of the project and introductory material about knowledge management, the facilitator introduced seven categories of potential KM research and asked participants to offer their thoughts on key challenges and specific RNs. After all of the topics were discussed, participants were asked if there were any other topics of interest not covered. At the end of each session, participants were asked to rate the importance of each of the seven categories. The same poll was used for both sessions to obtain a cumulative score.

Participant input related to the seven topics is shown below, followed by the poll results.

1. KM Guidance for Transportation Agencies – What is KM, Program Implementation, KM Techniques (what, how, why and when to use)?

Challenges

• This is new territory for many organizations - there are few existing internal templates on which to build a program.

• There are lots of moving parts in a DOT and it is difficult to keep up with who to go to for a particular question. There is a lot of undocumented knowledge – in peoples’ heads.
• Duplicative efforts due to difficulty keeping up with who is doing what across a large organization.
• Remote work (partial or full) is making it more difficult to share information.
• Getting leadership support for KM – and sustaining it through leadership changes each election cycle).
• Conflation of positional authority and subject matter expertise; need to clarify distinctions between subject matter expertise, practitioner knowledge, and general knowledge.
• Support for emergent practices/startup investigations of new issues – need to raise awareness of how siloed resources are a barrier and how data and information is used across technical disciplines.
• There is no federal mandate for KM.

Needs/Ideas

• Guidance on how to map current knowledge and identify gaps
• Guidance on how to curate, maintain and provide access to knowledge bases
• Guidance on pursuing a risk-based approach to KM
• Improved tools for knowledge capture and sharing that could be used by maintenance and construction personnel
• Guidance on conducting knowledge interviews (capturing operational knowledge) – and managing/providing access to completed interviews
• Guidance on creating expertise directories
• Guidance on creating and maintaining functional org charts
• Connection between KM and data/information governance (DG) – how governance might be used to influence what is captured and how
• Support for organizing a leadership forum on KM – provide opportunities for leaders to discuss challenges and practices at their agencies (Could be proposed as a 20-24 project)
• Support for case studies, peer exchanges, and other discussion pipelines to share current practice
• Research on the role and value of systems thinking given that knowledge, data and information sharing, and organizational development are all cross-organizational practices. (See Stephen Haines - Strategic and Systems Thinking: The Winning Formula)
• Exploratory research on creating a federal-level KM practice guideline or regulation

2. Knowledge Sharing Across Transportation Agencies – Facilitating Sharing of Information and Lessons Learned Across Agencies

Challenges

• Codified knowledge sharing is hampered by lack of standard metadata, lack of a good thesaurus to manage term changes and term relationships.
• Need for common terminology.
• No easy way for employees to network across agencies – especially at lower levels. Difficult to keep track of who your peers are given turnover.

Needs/Ideas

• Scoping study for a cross-agency knowledge directory – to include looking at how to describe/categorize various functions and roles; stewardship models and upkeep mechanisms
• Common DOT knowledge architecture to facilitate knowledge transfers
• See Topic 6 for ideas related to terminology management

3. KM Value/Metrics – How to Quantify, Describe, and Communicate the Value of KM to a Transportation Agency

Challenges

• Metrics are challenging – knowledge is not particularly subject to measurement.
• It is easy to track volume (# postings, # clicks) but this is not necessarily meaningful – would like to understand effectiveness and impact.
• A measure of information relevance would be useful, but it is not clear how to implement this.
• Implementing KM (and other emergent practices) are judged against metrics for operational activities. It would be helpful to examine metrics that support testing of emergent practices and provide accountability.

Needs/Ideas

• Model surveys for tracking trends in ease of finding information, use of centralized information repositories
• Synthesis of KM metrics used by other organizations that are adaptable for transportation agencies
• Guidance or resources for using capabilities of existing search and document/content management tools to track utilization and search success (partner with private sector on this effort)

4. Domain-Specific KM Applications – Specific Applications of KM (e.g., capturing knowledge from an experienced bridge engineer)

Challenges

• Each group in a DOT has different issues and requirements for knowledge capture and sharing.
• Lack of feedback loops to support continual improvement both within and between processes.
• Translating legal knowledge (e.g., related to contract negotiation) to provide decision makers with an independent ability to understand what can and cannot be done.

Needs/Ideas

• Domain-specific guidance and models for doing knowledge interviews (and using them)
• Guidance for fostering learning and knowledge transfer via improving handoffs across typical DOT work processes

5. KM/Learning Culture in Transportation Agencies – Overcoming Cultural Barriers to Knowledge Sharing and Learning in Transportation Agencies

Challenges

• People are not willing or motivated to share information outside of their silos.
• Cultivating an environment in which mistakes are OK, focus on the end goal.
• People are spread thin – there needs to be a culture of KM developed through everyone’s job role.
• There needs to be more clarity on what information and knowledge is required and generated at different levels of an organization (strategic, tactical, operational).
• Need to improve handoffs between domains and add feedback loops within and between processes to support continual improvement.

Needs/Ideas

• Guidance on how to do ethnographic studies of transportation agencies that can identify cognitive biases (and perhaps a pilot demonstration of one)

• Research on cultural barriers/issues related to navigating the multi-generational workforce
• Sample curricula that agencies could use/adapt for agency-wide KM training programs
• Study on motivating or incentivizing collaboration (including factors related to information technology support)

6. Data & Information Organization and Management – Improving How We Curate, Organize, and Share Our Information Resources to Maximize Their Use

Challenges

• Use of multiple storage options for critical information in an organization; difficult to access information across different silos; lack of central repositories.
• Lack of standard metadata – within and across transportation agencies.
• Legal issues associated with data sharing outside the agency (general public and other agencies); need to balance open access with privacy/security considerations.
• Handling information/data/open records requests – limited bandwidth to vet and respond; difficult to find and access the records that are stored in various locations and various formats.
• Difficult to maintain records of lessons learned given that anything we write down is subject to the open records laws.
• Gaps in knowledge about IP issues related to technology development at DOTs.
• Opportunity to use knowledge maps.
• Understanding the fundamentals needed to operate in the new digital age.
• Need to modernize records management to reflect knowledge capture, knowledge transfer, and institutional forgetting; and to use “big bucket” records management to support open government and collaboration.

Needs/Ideas

• Practical guidance on how to best share information – using available tools (e.g., websites, portals, SharePoint, TEAMS, OneDrive, Dropbox, etc.) to organize and share information
• Continued work on findability, including shared transportation business glossaries, improving metadata creation and management, evolution of transportation thesauri for uses beyond research tagging
• Research on emerging data sharing and interoperability standards and models (e.g., OIEE interoperability)
• Study focused on KM tools/techniques that can be used to equip agencies for responding to open records requirements

7. Workforce Development/Succession Management – Reducing Impacts of Employee Departures Through Knowledge Transfer and Collaboration + Being More Strategic About Transforming the Workforce to Meet Changing Agency Needs

Challenges

• Rapid turnover – “a revolving door.”
• Aligning KM strategies with other HR and Talent Development strategies.
• In organizations with unions, how can we implement succession shadowing scenarios while not appearing biased while remaining fair? Because in order to gain the implicit knowledge, the successor needs to spend time with their predecessor before they are out the door.
• How to build resiliency with respect to workforce skills – to adapt based on changing needs (agency priorities, technology advancement, new programmatic requirements).
• Thinning of the workforce – the people that we are able to retain are spread so thin doing their primary job duties they find it hard to make time for succession planning.

• How do we incorporate KM perspectives into the onboarding process – need a stronger understanding of what the position entails.
• Achieving equity within organizations (e.g., addressing unequal training resources).
• Marginalized/unsupported programs.

Needs/Ideas

• Guidance on taking a proactive, risk-based approach to preventing knowledge loss due to turnover, retirements, role change, etc.
• Study of how KM can support creating a more resilient organization
• Study of succession management effectiveness or value
• Model guidance for maintaining job books or other resources to help a successor understand their job – what the last holder of that position did and how, who they worked with, key resources they used
• Guidance on how to integrate KM with Talent Development and Onboarding strategies

Focus Groups and Interviews

Focus groups and interviews were held in March to April 2023 to supplement the earlier large group listening sessions and gather more in-depth information about stakeholder perspectives on KM challenges and RNs.

Three focus groups were held, targeting three subgroups:

• Representatives of DOTs who have implemented one or more KM initiatives
• Representatives of DOTs in early stages of KM implementation
• Representatives of DOTs who have not yet begun KM implementation

Key Findings

The focus groups were followed by a series of interviews with KM practitioners and experts.

Findings of the interviews and focus groups are summarized below, organized by the following discussion questions:

• What challenges or initiatives are driving your interest in KM?
• What KM research products have been helpful to you?
• What are the gaps – what new KM research products are needed?

KM Challenges and Drivers

Key challenges motivating interest in KM include employee turnover, improving efficiency, facilitating information, managing data and information, and building new knowledge through data analytics.

• Employee Turnover. The primary driver of interest in KM is related to the loss of institutional knowledge and continuity associated with the exodus of senior employees, the thinning of the workforce and increased employee turnover in general. This has eroded the experience base in DOTs and impacts employee productivity and agency performance. One participant noted that long-time employees take on various important tasks that are not necessarily in their official job descriptions, which fall through the cracks when they leave. There is interest

• in techniques for smoothing workforce transitions by building CoPs, capturing and transferring critical knowledge, and enhancing available resources that help employees make effective decisions in different situations. KM practitioners noted that producing job documentation is challenging because it is difficult to motivate busy employees to make time for this. There is a need to explore ways of making this part of the culture and creating accountability.
• Improving Efficiency. The siloed nature of DOTs makes it difficult to share knowledge and information across divisions and field offices (districts or regions). Agencies are challenged to make sure people are aware of what others in the agency have done/are doing to take advantage of lessons learned and avoid duplication of effort. There is interest in facilitating cross-functional collaboration, enhancing consistency and improving handoffs or touch points between different business processes.
• Facilitating Innovation. DOTs are interested in facilitating improvement to processes and methods. This requires incentivizing employees to identify, vet and implement promising improvements. Interest in innovation comes both from DOT research offices, who want to encourage and facilitate adoption of research results as well as innovation offices and other functions related to organizational excellence. Some DOTs are pursuing initiatives to promote a learning culture that encourages experimentation to help put innovative ideas into practice. One interviewee noted that implementation of emergent practices within a DOT needs to be handled differently than mature operational practices with respect to metrics, resourcing needs, expectations and cross-organizational engagement.
• Data and Information Management. DOTs face challenges related to managing the rapidly growing volume of data and documents to ensure that they are accessible, findable and usable; to meet records management and public information requirements; and to reduce redundant, obsolete, and trivial (ROT) content. There are also challenges related to data quality and integrity, lack of ability to integrate data across disparate systems, and insufficient documentation of data. While managing explicit (i.e., documented) knowledge resources is just one component of KM, this is a significant area of concern. There is strong and growing interest in data governance at DOTs, which, while primarily focused on structured data, provide the organizational mechanisms for improving overall information management practices. This is also an area where current and emerging applications of AI offer promising solutions.
• Business Intelligence (BI) and Analytics. DOTs are pursuing BI initiatives to enable discovery of new insights (knowledge) from data. There is a need to ensure that these efforts are aligned with agency strategy and business needs, and that there are mechanisms in place to facilitate the diffusion and application of insights gained.

Prior KM Research of Value

Stakeholders had used the following research products:

• NCHRP Report 813: A Guide to Agency-Wide Knowledge Management for State Departments of Transportation – the KM litmus test in this document was particularly useful
• Domestic Scan Report – NCHRP Project 20-68A, Scan 12-04, Advances in Transportation Agency Knowledge Management
• NCHRP Report 867: Keeping What You Paid For—Retaining Essential Consultant-Developed Knowledge
• 2016 TR News issue on KM
• Documents from other agencies (Caltrans, West Virginia, New Hampshire, FHWA)
• University research conducted specifically for the DOT
• Materials from American Productivity and Quality Center (APQC) materials – especially their 1-page roadmap

Gaps to be Addressed in Future KM Research

The following gaps were identified based on the interviews and focus groups:

1. Understanding of the Context for KM at DOTs. There is a need to improve our understanding of what “KM” actually means in a DOT and how KM techniques can help a DOT to improve performance and respond to emerging requirements and needs. Research is needed to obtain an “on the ground” understanding of processes for knowledge creation/capture, sharing, discovery and application as part of different DOT business areas (e.g., safety, asset management, resilience planning, system operations) and situations (e.g., response to new legislative mandates; new technology adoption). This research could improve the ability to understand and communicate the value of KM to DOTs. It could also lead to tailored guidance and tools for advancing KM in support of DOT needs and priorities.
2. Current KM Practice Examples. There is a continuing need for current information about what different states are doing (or industry programs that are transferable to transportation agencies). Information of particular interest includes a description of the practice, the cultural impact on the organization, and discussion of any hurdles faced in implementation and how they were overcome.
3. Targeted Guidance and Tools for Operationalizing and Integrating KM. There is a need to go beyond existing introductory, conceptual guidance on KM for DOTs and provide more practical implementation guidance and tools geared to different audiences and situations. Of particular interest are:
   • Tools and examples for communicating the value of KM to different audiences.
   • Guidance on who in a DOT to engage in KM initiatives, why and how.
   • Steps to start building KM in the organization; pros and cons of different placement of the KM lead in the organization; qualities to look for in a KM lead/champion.
   • Relationships between and alignment across KM, records management, information management, data/information governance, HR, training/employee development, innovation, Lean, change management, other DOT functions.
   • How to find out what KM activities are already taking place, how knowledge is being captured and shared, how to track progress.
   • Tools to check the “KM Health” for a business unit to prioritize and target application of different KM techniques.
   • Practical guidance for managing and governing knowledge assets (content) – platforms, model for “what good looks like.”
   • Practical guidance (with examples) for producing job aids.
   • Practical techniques to improve the employee onboarding process.
   • Practical techniques for capturing essential employee knowledge as part of the off-boarding process.
   • Practical and useful techniques for applying knowledge mapping and process mapping methods, with examples.
   • How to assess what existing knowledge is no longer serving the agency well – how to identify and do “institutional forgetting.”
   • Communicating the value of KM to engineers.
   • Advice on how to roll out different KM initiatives – CoPs, employee expertise directory, lessons learned repository, social network analysis, etc.
   • Guidance on how to integrate results of lessons learned and AARs into training materials, SOPs, and manuals.
   • Guidance on how employees should be checking the knowledge they receive (via other people or documents) to ensure that it is accurate and trustworthy.
   • Guidance on expectations for employees regarding when and how to share data/information.
   • Strategies for overcoming common barriers to knowledge capture and transfer.

• Models for how to organize and package content/knowledge assets so that they are easily consumable and suited to different learning styles (e.g., links from a process map, search-able video library, integration into guidance documents, etc.).
• Model change management plan and implementation guidance for KM covering how to identify and address different types of resistance; desirable qualities for a champion and change agent.
• How to sustain a KM program, pitfalls to watch out for (e.g., single KM lead, lack of dedicated/consistent resource streams).
• Tools/guidance on working across different organizational cultures.
• Tools/guidance for getting “air time” for newer practices like KM.
4. Applications of Emerging AI Tools and Techniques. There is a need to better understand applications, benefits and limitations of available and emerging AI applications (including large language models and generative AI) for knowledge discovery and delivery.
5. KM and Innovation/New Practice Adoption. There is a need for research to improve understanding and awareness of how DOTs can best support the advancement of emergent practices that require cross-functional coordination and collaboration, as well as time for experimentation, learning and evolution. This research should consider both internal KM processes as well as peer to peer knowledge sharing across agencies.
6. Applications of Knowledge Representation Techniques for Search and Data Integration. There is a need for research to explore and advance use of different knowledge representation techniques including thesauri, ontologies and knowledge graphs within the transportation domain, and to understand and document the benefits of these applications (e.g., facilitating data integration, enabling knowledge discovery).
7. Culture and Change Management. There is a need for research on successful approaches to implementing culture change at DOTs, exploring how to foster an innovation mindset, how to deal with resistance to change (“this is how we’ve always done it”), and how to apply existing models for understanding current culture (e.g., four lenses, seven elements of culture) within a DOT setting.

Advancing Enterprise Architecture (EA) Practices at DOTs. There is a need for research to provide a better understanding of how EA practices can work in tandem with KM at DOTs to improve alignment across business processes, information systems, technology, and workforce capabilities.