2. Research Approach

This section describes Iknow’s research approach. Iknow’s approach consisted of three primary phases and two project initiation activities. The three phases were:

1. Phase 1 – Conduct Secondary Research
2. Phase 2 – Conduct Primary Research
3. Phase 3 – Prepare Final Research Products.

The three phases covered nine workstreams.

2.1 Summary of Research Deliverables and Due Dates

The duration of the NCHRP Project 23-17 was two years, beginning in July 2023 and ending in June 2025.

A schedule of the project phases, workstreams, deliverables, and due dates is presented in Table 1. The Iknow project team created 14 primary deliverables and more than 27 other project outputs.

Table 1
Primary Research Phases, Workstreams, Deliverables, and Due Dates

During the project, Iknow completed and submitted 14 monthly status reports and six quarterly status reports, conducted four Bimonthly Meetings, conducted the project kickoff meeting with the Program Officer and panel, and led one in-person Interim Project Meeting with the Program Officer and panel at the National Academy of Science’s Keck Center in Washington, DC. Please see Table 2 for a complete list of secondary research deliverables and their respective due dates.

Table 2
Secondary Deliverables and Due Dates

2.2 Detailed Description of the Project Phases

The project initiation activities and the three research phases are described in detail below.

2.2.1 Project Initiation

Project initiation launched the research effort. It consisted of two activities:

1. The Project Kickoff Meeting was held on July 13, 2023. Dr. Palowitch and Mr. Leeb met with Mr. Jared in Washington, DC, and the panel members attended by teleconference call.
2. The Iknow project team completed and submitted the Amplified Research Plan on June 26, 2023. A revised plan was submitted on July 9, 2023. The Amplified Research Plan was a refinement of Iknow’s original work plan, which was submitted in its project proposal. The revised Amplified Research Plan incorporated valuable feedback from the panel, captured additional background on the four research questions, and included the panel’s guidance in other areas. Iknow also received additional sources for the literature review.

2.2.2 Phase 1 – Conduct Secondary Research

2.2.2.1 Objective

Secondary research, also known as desk research, involves the analysis and synthesis of data already collected and published by others. Common data sources for secondary research include academic journals, government reports, industry publications, online databases, and other types of published articles from various media channels.

Iknow’s goal of Phase 1 was to efficiently scan, review, and ingest published information that could be relevant to the Guide. During this phase, the Iknow project team became familiar with the prior work done in knowledge management at the state departments of transportation through published articles and the NCHRP’s research reports. This secondary research provided a reference point for identifying the differences and potential gaps between the state DOTs’ KM programs and other state-level departments and agencies, as well as KM initiatives at commercial and nonprofit entities.

2.2.2.2 Methodology

Iknow’s methodology for Phase 1 consisted of three steps.

Step 1. Develop the Literature Review Plan

The Iknow project team identified three categories of relevant published literature:

1. Transportation-focused sources. Several examples of relevant sources include the NCHRP reports and publications, DOT resources (National Transportation Library, Volpe Center), and TRID, an integrated database that combines the records from the TRB’s Transportation Research Information Services (TRIS) Database and the OECD’s Joint Transport Research Centre’s International Transport Research Documentation (ITRD) Database.

2. KM-focused sources (industry agnostic). Several examples of relevant sources include the APQC’s resource library, KMWorld, Association for Intelligence Information Management (AIIM), and the ACM Digital Library.
3. General academic/research-focused sources. Two examples of relevant sources include the Journal of Knowledge Management and the Journal of Intellectual Capital. The project team conducted literature searches on:
   • Directory of Open Access Journals (DOAJ), a comprehensive online database that indexes and promotes high-quality, peer-reviewed open-access journals from around the world
   • WorldCat, a catalog that aggregates the collections of tens of thousands of libraries worldwide, operated by the Online Computer Library Center (OCLC)
   • Scopus, a comprehensive abstract and citation database launched by Elsevier in 2004
   • Education Resources Information Center (ERIC), a comprehensive online digital library of education research and information
   • Google Scholar.

   Some of the search parameters included: knowledge management, tacit knowledge, performance measurement, KPIs, assessment, evaluation, and knowledge transfer.

The Iknow project team collected 8 to 15 representative articles from each category to test various online search approaches. The Iknow project team summarized these efforts in the Literature Review Plan.

* * *

The Iknow project team completed and submitted the Literature Review Plan on July 13, 2023. This deliverable defined the approach Iknow would take to examine relevant published literature.

Iknow received the panel’s comments on the Literature Review Plan and submitted a revised plan on September 13, 2023, incorporating the panel’s feedback.

Step 2. Conduct the Literature Review

The Iknow project team performed an extensive search for published articles from July 13, 2023, to September 8, 2023. Two questions guided our literature review:

1. What reports and articles have been published on KM-related programs and activities in state DOTs and other transportation departments? The answer to this question gave the project team a baseline for the guidebook. We didn’t want to repeat material covered in previous reports, but we did want to update the guidebook’s readers on KM topics that were new or have recently advanced.
2. What articles have been published that address the four objectives of the guidebook, specifically KM assessment and measurement methods, KM connections to other organizational practices, options for KM program placement within organizational structures, and internal partnerships?

1. The answer to this question helped frame the body of knowledge needed to fully address the guidebook’s scope.

Additional criteria used for conducting the searches were guided by the four research objectives.

Iknow found, downloaded, and read more than 180 articles that met its various search criteria. However, many articles turned out to be inapplicable despite relevant titles and abstracts.

Step 3. Prepare the Literature Review Summary

The Iknow project team found 49 articles that were relevant to the four research topics and prepared written summaries of each. These summaries can be found in Appendix A1.

The team consolidated the article summaries into the Literature Review Summary.

* * *

The Iknow project team completed and submitted the Literature Review Summary on September 11, 2023, and a revised Literature Review Summary on September 13, 2023.

2.2.3 Phase 2 – Conduct Primary Research

2.2.3.1 Objective

Primary research refers to the process of collecting new data through direct methods such as surveys, interviews, observations, and experiments. The aim of conducting primary research is to address specific questions or problems identified by researchers or stakeholders. It allows for a deeper understanding of a topic by generating unique insights that are directly relevant to the research objectives.

Iknow’s goal of Phase 2 was to conduct primary research by interviewing the KM leads at state DOTs. The importance of interviewing end users of the KM Guide cannot be overstated. In the realm of technical projects, particularly those involving the design, development, and implementation of new products and services (like knowledge management), understanding the needs and perspectives of end users is paramount. Six reasons why conducting interviews with end users is essential are listed below.

1. Insight into User Requirements. End users provide firsthand insight into their requirements, preferences, and pain points. By directly engaging with the KM leads, Iknow gained a deeper understanding of the topics and features essential for effectively meeting user needs.
2. Validation of Assumptions. Often, on our client engagements, Iknow makes assumptions about user behavior or preferences. Conducting interviews helps validate or refute these assumptions, ensuring that development efforts are aligned with real user expectations. This validation mitigates the risk of investing resources in developing recommendations and solutions that may not resonate with the target audience.
3. Identification of Unmet Needs. End-user interviews uncover unmet needs or latent desires that may not have been apparent through secondary research and other traditional research methods. Identifying unmet needs presents opportunities for innovation and differentiation,

3. allowing Iknow to deliver recommendations and solutions in the KM Guide that truly address user pain points and enhance their overall experience.
4. Enhanced User Experience. Iknow can prioritize the Guide’s content by incorporating user feedback obtained through interviews to improve the end-user experience. This customer-centric approach fosters higher levels of end-user satisfaction, ultimately improving the DOT’s performance and success.
5. Reduced Rework and Development Costs. Understanding user needs early in the KM Guide Project minimizes the likelihood of costly redesign and rework later in the writing life cycle. By addressing user requirements early on, Iknow can streamline the development processes, reduce iteration cycles, and optimize our resource allocation, leading to shorter review and approval cycle times and significant cost savings.
6. Increased Adoption and Acceptance. Engaging end users in the development process fosters a sense of ownership and involvement. As a result, end users are more likely to embrace and advocate for the final product, leading to increased adoption rates of KM within the DOTs.

2.2.3.2 Methodology

Iknow’s methodology for Phase 2 consisted of five steps.

Step 1. Solicit Input from the Panel

Iknow developed a data request for the panel members. The objective was to get the panel’s input on 1) state DOTs with leading KM programs or notable (individual) KM projects, and 2) contacts in the DOT community that should be interviewed as part of the Stakeholder Engagement workstream. Mr. Jared distributed the data request to the panel members on September 15, 2023, and he sent the completed questionnaires to Iknow on September 26, 2023. Iknow analyzed the data from the questionnaires.

Iknow conducted its first bimonthly meeting with the panel on October 19, 2023. Most of the meeting was spent reviewing the panel’s input from the data request. During the meeting, the panel members added more names and contact information to the list of potential interviewees. The panel also contributed additional information on other questions.

Step 2. Identify KM Leads at State DOTs

Iknow continued developing the master list of KM leads in November and December 2023. The desired interviewees were the heads of knowledge management in state departments of transportation, DOT executives who support KM programs, and any state DOT employee interested in implementing knowledge management.

The points of contact were developed from a combination of panel member recommendations, our literature search, and the AASHTO CKM membership list. The interviewee list grew to roughly 120 people from 47 U.S. states. Iknow reviewed the list of potential interviewees with Mr. Jared.

Iknow set a target to speak with the heads of KM in 25 state DOTs.

Step 3. Develop the Interview Guide and Conduct the Initial Outreach

The Iknow project team defined the interview approach and developed a comprehensive interview guide. The interview guide is presented in Appendix A2.

The Iknow project team continued to expand the list of potential stakeholder interviewees and began the initial outreach to the KM leads in all fifty states and the District of Columbia in November 2024. The objective of the initial outreach was to invite them to participate in the interviews for this project. The initial outreach continued until January 2024.

* * *

The Iknow project team completed and submitted the Stakeholder Engagement Plan (Tech Memo 1) on November 9, 2023. This deliverable described the stakeholder engagement approach and listed the names of potential interviewees at the state DOTs. Tech Memo 1 also contained the interview guide.

Iknow received the panel’s comments on Tech Memo 1 on December 6, 2023. Iknow prepared responses to the panel comments and submitted them on December 10, 2023.

Step 4. Conduct Stakeholder Interviews

The Iknow project team conducted interviews with the KM leads from twenty-seven state DOTs and the District of Columbia. Iknow spoke with 42 people in total (some states had two or three people attending the interviews). The interviews were conducted between December 8, 2023, and February 23, 2024.

All of the interviews were one hour long and followed the interview guide presented in Appendix A2. Table 3 lists the completed interviews sorted alphabetically by state.

Table 3
Interviews Completed with State DOTKM Leads

A written interview summary was prepared for every interview. The twenty-seven interview summaries are presented in Appendix A3. One-page data sheets were collected from about 18 states. The data sheets are included with the interview summaries.

Step 5. Synthesize Findings

The Iknow project team analyzed the data collected from the interviews and data sheets to uncover insights and findings. The stakeholder interviews yielded 21 important findings. The findings are presented in Appendix A4.

The stakeholder interviews were valuable because they ensured the Guide would truly resonate with the target audience, ultimately increasing the adoption rate of KM within the DOTs and contributing to the DOTs’ business success.

* * *

The Iknow project team completed and submitted Technical Memorandum 2 on March 8, 2024. This deliverable contained the findings from the stakeholder interviews and the written interview summaries.

The project team received comments and feedback from the panel on March 29, 2024. Iknow updated Tech Memo 2 and submitted the revised report on April 2, 2024.

2.2.4 Phase 3 – Prepare Final Research Products

2.2.4.1 Objective

Iknow’s goal of Phase 3 was to prepare the final research products listed in the RFP. The research products incorporated the findings from the primary research (i.e., interviews with the KM leads from state DOTs), secondary research, feedback from the project’s panel members, and Iknow’s own knowledge from almost 25 years of experience working in the knowledge management field.

2.2.4.2 Methodology

Iknow’s methodology for Phase 3 consisted of four steps.

Step 1. Develop the Guide’s Outline

The stakeholder interviews helped identify five categories of DOT staff who may be interested in reading the Guide. The target audiences are:

1. KM Leaders and Practitioners
2. Senior Executives and Decision-Makers
3. Financial and Resource Allocation Staff
4. Technology and IT Personnel
5. Operational and Project Managers.

The primary audience was KM Leaders and Practitioners. More specifically, the primary audience was KM leads, teams, and functions that are relatively new to knowledge management.

The stakeholder interviews also revealed that KM leads wanted the Guide to be a user-friendly resource, written in an engaging and conversational tone.

Based on the insights gleaned from the stakeholder interviews, the Iknow project team prepared the initial outline for the Guide. The Iknow project team completed and submitted the Guide Outline deliverable on June 7, 2024.

The Guide’s outline was revised several times between June and September 2024. The Guide’s final outline is presented in Table 4. The table includes the rationale for each section’s inclusion.

Table 4
Outline for NCHRP Research Report 1164: The Business Case for Knowledge Management: A Guide

As part of the Draft Guide Outline (Tech Memorandum 3) deliverable, the Iknow project team also developed a template for writing the content in Section 5, which addressed the four primary research objectives. The team wrote one of the primary research sections using the template to demonstrate how the content will follow it and to present an early sample of the writing style, tone, voice, language, grammar, and formatting.

* * *

The Iknow project team completed and submitted the Draft Guide Outline (Tech Memorandum 3) and the Interim Report on June 7, 2024.

Iknow conducted the project mid-point meeting with the panel on August 1, 2024. The interim meeting was a one-day, in-person project mid-point meeting held in Washington, DC.

During the interim meeting, the panel decided to rename the primary project deliverable from “KM Guidebook” to “KM Business Case Guide.” This change meant the project team needed to add several new sections to the outline to explicitly address developing a KM business case (i.e., Sections 3, 4, and 6 in Table 4).

The panel also approved moving the Stakeholder Focus Group Notes deliverable from Workstream ID 6 to Workstream ID 7. Mr. Anyacho, the panel’s chair, recommended that the focus group participants be people the project team already spoke with. “They already understand the goal of the Guide.”

The panel’s comments from the interim meeting were captured in Iknow’s Responses to Panel Comments on the Interim Report, which was published on August 10, 2024, and incorporated into the outline for the Guide, which was revised on August 12, 2024.

Iknow prepared and submitted notes from the interim meeting on August 30, 2024.

Step 2. Develop the Writing Plan

The Iknow project team developed its internal writing plan, and individual writing tasks were assigned in mid-August 2024.

Step 3. Create the Research Outputs

The Iknow project team wrote the sections of the approved outline. The draft Guide was prepared over a period of roughly seven months, from early August 2024 to the end of February 2025.

Iknow published an initial draft of the Guide on September 18, 2024. The initial draft was sent to Mr. Jared for distribution to the panel before the bimonthly meeting scheduled for October 1, 2024. The purpose of distributing this early draft is to seek the panel’s feedback since the focus of the Guide expanded at the Interim Meeting.

Iknow submitted the Guide draft (Draft 1) on November 4, 2024. Iknow prepared and submitted written responses to the panel’s comments on the Guide draft (Draft 1) on December 9, 2024.

The Iknow project team defined six outreach pieces and developed draft text for each piece. Iknow contracted with an external graphic designer to professionally produce each piece.

* * *

Iknow discussed an initial version of the Guide with the panel during the bimonthly meeting held on Tuesday, October 1, 2024. Iknow discussed the full version of the Guide (Draft 1) with the panel during the bimonthly meeting held on Thursday, December 12, 2024.

Iknow submitted three deliverables to the Workfront portal on January 3, 2025: the Guide (Draft 2 incorporating panel comments), the stakeholder outreach plan, and draft outreach materials. Iknow received comments from the panel, and prepared and submitted written responses to the panel’s comments on February 18, 2025.

Iknow conducted a bimonthly meeting with the panel on Friday, February 7, 2025. Iknow prepared a presentation on the project’s status and discussed the panel’s written feedback.

Step 4. Review and Revise

The Iknow project team made significant improvements to the Guide (Draft 2) based on the panel’s feedback throughout Phase 3. The suggestions included several business case examples, navigation icons, professionally created illustrations, editing to improve readability, and other items suggested by the panel.

The project team followed the Agile methodology for writing, reviewing, and revising the text of the Guide in this Phase.

The Iknow project team used several electronic text editing tools to find and fix grammar, punctuation, and other writing errors.

The Iknow project team solicited feedback from state DOT KM leads as part of the Review and Revise Step. The project team invited the DOT KM leads from eight states to participate in stakeholder feedback sessions. The sessions were held in late January 2025. The Guide (Draft 2) was sent to the

participants several weeks prior to the feedback sessions to give them the opportunity to thoroughly review it.

Iknow developed an interview guide to use during the stakeholder feedback sessions. The interview guide is presented in Appendix A5.

The Iknow project team conducted six stakeholder feedback sessions in late January 2025. The interviewees are listed in Table 5.

Table 5
Participants in the Stakeholder Feedback Sessions

The project team consolidated the notes from the six sessions, summarized the comments, and incorporated the feedback into the draft Guide.

Iknow continued updating the final version of the Guide through the end of February 2025.

Iknow submitted the following five deliverables on March 3, 2025:

• NCHRP Research Report 1164: The Business Case for Knowledge Management: A Guide (final draft)
• Outreach materials (draft)
• NCHRP Web-Only Document 437: Assessing and Measuring the Business Value of Knowledge Management (draft)
• Implementation plan (draft)
• Slide summary (draft).

Iknow received comments from the panel on the five draft final deliverables on May 19, 2025. The project team incorporated the comments into the five deliverables and submitted the final deliverables on May 30, 2025.