APPENDIX A: Literature Review

A barrier to RVAM, in addition to reduced funding for vegetative assets, is the fact that tools and technology used for structural asset management often cannot be used for vegetative assets without being retrofitted. While structural assets are visibly static, except for deterioration and direct damage, vegetation is ever-changing. Due to plant life cycles, vegetation changes in height, girth, coloration, and shape throughout the year and across years (i.e., tree leaves turn colors in the fall and then fall off the tree; a shrub increases in size each year). These constant changes make it more difficult to track these assets with Light Detection and Ranging (LiDAR) and photogrammetry. LiDAR is a remote sensing method that uses 3-D lasers to create a map or image of the environment (Ussyshkin and Theriault 2011). Photogrammetry is a technology that uses photographic images and patterns of electromagnetic radiant energy to create a 2-D or 3-D model of the environment (Tran et al. 2022) to determine reliable information about the environment and physical objects. Photogrammetry can be used in conjunction with LiDAR to capture the entirety of the ROW. These remote sensing technologies can be used with images gathered from cellular devices, cameras, tablets, or other hardware. To ensure accurate results, these tools must be employed at certain times of the year, especially when identifying invasive plant species. Companies are testing different methods of using variations in pigmentation to identify plant species (Lawrence et al. 2006; Safaei et al. 2022). At the time of this report, LiDAR is one of the more commonly used remote sensing technologies adopted by DOTs domestically and internationally.

The constantly changing nature of vegetation also makes management more difficult, as it requires repeated management actions every year. This means some tools and technology are not always suitable for vegetation asset management. In contrast, structural assets retain their shape throughout their life cycle. A change in a structural asset’s shape or coloration, such as visible cracks or breaks in the surface, can indicate that it needs maintenance. As such, asset tracking and monitoring software and hardware differ for structural assets (like bridges, pavement, and guardrails) and vegetative assets (like turfgrass, trees, shrubs, formally landscaped areas, etc.). Tools and technology would need to effectively monitor seasonal vegetation changes in addition to monitoring efforts for invasive species identification to accurately inform maintenance actions, such as seeding, mowing, trimming, or removal.

Many departments of transportation (DOTs) use commercial off-the-shelf (COTS) tools and technology that were initially developed for a different sector and then refurbished for DOT use for vegetation management. This allows DOTs to use newer technologies that are developed for related types of work (i.e., DOTs utilizing farming technology for herbicide application and tracking).

Tools and technology such as geospatial technology, remote sensing and monitoring technologies, mobile devices/software applications, and nondestructive evaluation methods are applicable for inspecting highway infrastructure and managing assets during and after construction. DOTs are currently using these tools and technologies and conducting research to determine whether they can be used in a greater capacity (Olsen et al. 2013; Tran et al. 2022). Though many of these technologies can be used for multiple purposes, they have been nearly exclusively used for structural assets and are only starting to be used more often to manage vegetative assets. However, developers have found issues with the ability of the technologies and systems to accurately manage, monitor, and document vegetation due to changes in growth patterns and the coloration of vegetation throughout the year.

Data Collection

Geographic Information Systems (GIS) can store and analyze geographic data. These systems can be used either by shapefiles created manually in the office or through the use of a Global Positioning System (GPS) where points, lines, or polygons are collected and added to the geodatabase. The information collection can be done through either field collection or crowdsourcing (Hall 2015). Field collection is the most commonly used method to collect data on the DOT right-of-way (ROW). Personnel can use LiDAR, handheld GPS devices, or vehicle-mounted cameras to collect data. The information collected is either still images or videos, depending on the software used to analyze the data.

A tablet, cellular device, or handheld device designed exclusively for GPS data collection can be used. Tablets and smartphones are the most common devices for tracking field data because they have additional uses, such as capturing and sharing images, communication, and other applications within the system. In addition, due to their frequent and widespread use in professional and personal settings, staff are familiar with these devices (Lwin et al. 2014). These electronic devices can be carried by staff or installed in trucks to control other systems within the vehicle, such as herbicide applicator equipment. As tablets and cellular devices can be used for multiple applications within DOT operations, DOTs can purchase fewer devices than they would otherwise need if the units only had one function. LiDAR can be used to create 2-D and 3-D maps of any surface in sight of the sensing unit. Both aerial and mobile LiDAR are available and accurately map large areas of land at high rates of travel while static LiDAR can collect highly accurate data over a small area (Ussyshkin and Theriault 2011). The ability to map geometric data and accurately assess conditions makes LiDAR a useful tool for landscape monitoring purposes as well as construction quality control. LiDAR is an essential tool for developing a 3-D model of the roadside ROW. This system can be used for both structural and vegetative features. LiDAR can effectively evaluate roadside ditches for slope and drainage (Habib et al. 2021). More commonly, these systems are used for structural assets to identify system deterioration.

LiDAR has been tested over the years and has historically been limited by satellite orbits and wavelengths, which impact available satellite and spatial imagery, respectively. Historically, satellites have been the main form of LiDAR data collection. Currently, airborne is the main form of LiDAR data collection (Haring 2021); however, there has been recent progress in utilizing ground-based LiDAR data collection for mapping purposes to help mitigate these issues (Ussyshkin and Theriault 2011). LiDAR can be combined with other datasets, such as information gathered from augmented reality (AR), to create a more comprehensive view of roadsides. These new sources of information are rapidly changing the way DOTs are able to view their landscapes.

The data included in the GIS software is commonly collected through Global Positioning Systems / Automatic Vehicle Locators (GPS/AVL). These are some of the most common technologies DOTs use (Ming-Shiun et al. 2022) as many have a GPS/AVL system in their trucks, particularly for snow and ice operations. This equipment, once installed, can track the locations of vehicles throughout the workday, enabling DOTs to ensure public safety by determining what areas of the road have been plowed or treated with either chemicals or salt. As these systems are tied to vehicles, if a vehicle is used for multiple types of jobs throughout the year, the central office would be able to track the working locations for multiple types of work orders, such as litter pickup, herbicide application, and more (Schneider et al. 2017).

All remotely collected data should have a ground-truthing aspect to ensure accuracy. Unmanned Aerial Vehicles (UAV) or drones can use LiDAR and photogrammetry to identify areas within or around assets that show signs of deterioration. The use of drones and UAVs to obtain LiDAR data has significantly increased the value of this information and enabled DOTs to quickly progress in developing 3-D imagery to manage their roadside assets. In addition, the high-resolution imagery captured by UAVs increases the usability of this technology for vegetation management. UAV technology has widespread applications. For instance, Wedegedara et al. found that these technologies allowed utility companies to proactively manage trees along power lines in Connecticut (2022). At the same time, Costello et al. found they could positively identify the noxious parthenium weed in agricultural settings utilizing red-green-blue (RGB) and hyperspectral imagery with drones and the assistance of machine learning (2022). This method had an overall accuracy of 95% for detection and 86% for classifying flowering versus nonflowering parthenium weeds. This indicates that identifying weedy species along the ROW can be completed remotely, increasing DOT staff safety. It is recommended that a staff member ground-truth, or visit the site to verify information, prior to maintenance crews being dispatched to complete any subsequent maintenance work.

New and Evolving Technology

Imagery

UAVs can utilize hyperspectral imagery. Hyperspectral imagery is an alternative to LiDAR that provides accurate vegetation mapping via aerial imagery. However, the accuracy of hyperspectral imagery (and other forms of aerial imagery in general) is highly dependent on the spatial modeling and analysis techniques that are applied to the collected imagery data. Researchers have been able to use a gradient boosting software to classify pixels from the hyperspectral data, achieving a 99% classification accuracy for parthenium weed growth stages (Costello et al. 2022). This supplements a growing body of literature detailing the use of machine learning for classifying, mapping, and monitoring invasive species. During a Montana study that sought to map the invasive leafy spurge (Euphorbia esula L.) and spotted knapweed (Centaurea maculosa Lam.), researchers were able to achieve 84% and 86% identification accuracy for spotted knapweed and leafy spurge, respectively (Lawrence et al. 2006). The team completed this identification using the Breiman Cutler classifications (BCC) and the random Forest package in R algorithm. All other analysis methods tested for classification accuracy were not nearly as successful (Lawrence et al. 2006). Thus, hyperspectral imagery combined with the appropriate analysis provides a promising method of identifying vegetation along ROWs when remote sensing imaging techniques are a viable option for DOTs.

Dashboard cameras and other vehicle-mounted cameras are an emerging technology that can allow DOTS to easily access images along the roadway. Currently, multiple organizations are looking into the use of dashboard imagery (Joshi and Witharana 2022; Li et al. 2019; Verbree et al. 2004; Creusen and Hazelhoff 2011). When used in conjunction with, or as an alternative to, aerial remote imagery such as LiDAR, which can provide 3-D mapping but often has coarse vertical resolution, dashboard videos can provide ground-level vertical profiles of trees and other vegetation to further characterize roadside forest conditions (Wedegedara et al. 2022). A Netherlands-based company began using 3-D point clouds through cameras mounted on vehicles to identify traffic signs and other structural elements and the corresponding condition of these assets (Verbree et al. 2004). The company is increasing the type of imagery it can collect and has contracts with DOTs to map vegetation such as trees to help determine where maintenance and tree removal is needed.

A device consisting of a machine-vision camera, a GPS, a remote control, and a processing platform was able to collect high-resolution images of invasive plant species at speeds up to around 80 miles per hour (130 km/hour). Invasive species are species that are not native to an ecosystem and whose introduction has already, or most likely will, cause environmental or economic harm (Hobbs and Mooney 2000). The system was trained through machine learning to classify individual species as well as perform binary classification of invasive versus noninvasive. This technology largely depends on deep convolutional neural network training. A convolutional neural network (CNN) is an artificial neural network designed for image recognition and processing. Systems are being developed that are capable of using machine learning to enhance CNN to automatically update or learn as it is used. These systems are used for classification and object detection to successfully identify an array of invasive species expected to be found in the area. These systems collected data while mounted on a car. To reduce labor costs from regular inspections, researchers proposed mounting devices to cars to complete inspections. This approach is more cost-effective for locating invasive species (Dyrmann et al. 2021). The study demonstrated that automatic detection and mapping of invasive plants along the roadside is possible at high speeds. However, devices like these are still being researched and are not currently manufactured for commercial use.

One emerging technology capable of working with both vegetation and structural assets is digital twins. Ammar et al. (2022) defined digital twins as “the connection between the physical and the digital aspects of an asset, thus, aligning with the overarching objective of asset management of leveraging the use of the asset information (i.e., digital aspect of the asset) to improve the asset’s performance throughout its life cycle (i.e., physical aspect of the asset).” Digital twins is a full-scale landscape-mapping concept where 3-D imagery is collected and used to replicate the ROW within a digital environment. This can be done with multiple types of software, including Building Information Modeling (BIM), Computer-Aided Design software (CAD), LiDAR, and photogrammetry. Digital twins can be used to manage a large array of transportation asset data that are crucial in making informed decisions relating to RVAM plans. Digital twins give transportation agencies access to digitized copies of the physical assets along the ROW, which are used to optimize the assets’ performance.

The switch to digital twins for transportation asset data management can be complicated. To be successful, DOT leadership need to provide detailed and comprehensive information regarding the data requirements necessary to implement the switch, plus staff training. Different physical assets are organized into tiers based on their complexity and the level of detail required for data collection and analysis. Automated data collection is normally scheduled every two years. However, if the data collection technology is not up to date, the data collected may not include pertinent information, such as asset type. If a DOT cannot feasibly purchase new data collection technology, staff must manually collect data more frequently to properly create and adopt digital twins. The predecessor to digital twins within a DOT is the regular use of geospatial, remote sensing, and monitoring technologies. To ensure staff can follow new procedures and to ensure comfort with the utilization of digital twins, it is recommended that DOT staff are first comfortable using geospatial, remote sensing, and monitoring technologies.

Artificial Intelligence

The research team looked into the use of artificial intelligence (AI) for both remote and manual surveys. Remote work can be completed using AR and automated machine guidance (AMG). Manual surveys can be completed with the use of AI through UAVs. UAVs can be used in conjunction with LiDAR and hyperspectral imagery to develop a map depicting the ROW.

AMG uses GPS and 3-D models of the environment to help guide construction equipment in order to improve efficiency (Tran et al. 2022). This system is helpful when state DOTs look to develop 3-D models of the ROW, such as digital twins, as it is functional with both hardscape and softscape construction. Multiple state DOTs, including Florida, Iowa, Kentucky, New York, Utah, and Wisconsin DOTs, outline applicable AMG methods specific to both ecosystem reclamation and road construction alike (Tran et al. 2022).

Although AR technology is new to the transportation construction industry, its future use is projected to rapidly increase in response to cheaper, high-quality 3-D options (Tran et al. 2022). AR headsets create a computer-generated, 3-D image of an environment that users can interact with and manipulate in real time to view various stages in the construction process (Tran et al. 2022). This type of data can be invaluable for detecting and correcting flaws in work placement prior to implementation. AR slightly differs from virtual reality (VR) in that computer-generated content is superimposed onto a real-world view, and the user’s awareness of the real environment can be maintained. DOTs can apply these uses to softscape construction, and the predicted reduction in costs and improving quality of VR/AR technology can make this a viable option for a cost-effective multiuse tracking method for hardscape and landscape construction projects.

DOT staff and contractors can drive or walk the ROW to map areas and then use AI to analyze the information to determine where vegetation and roadside safety features are located within the ROW and what areas of the ROW require maintenance. These manual surveys are labor intensive and impractical for creating large maps in a timely manner (Dyrmann et al. 2021). UAVs are a manual/semimanual way to map the ROW. These systems can be operated either manually or automatically; however, when a pilot does not receive a part 107 waiver from the Federal Aviation Administration, the pilot must be within sight distance of the equipment (FAA 2024). Beyond Visual Line of Sight (BVLOS) is becoming more common, with UAV technologies having been developed to allow for this (Davies et al. 2018). Many DOTs have not accepted this practice due to possible liabilities along the ROW. Currently, many DOTs require visual line of sight to be maintained to reduce accidents along the ROW. When DOTs gain confidence in the advanced systems, it may be possible for UAVs to be outside of the line of sight of the pilot. UAVs are a popular method for camera-assisted monitoring of invasive plants. These systems can be considered impractical for roadside monitoring as they can only travel a small portion of the roadway before needing to be charged or refueled, and they cannot be flown in high winds. When conditions allow, UAVs provide a fast and effective way to survey a habitat (De Castro et al. 2021).

Funding

Funding for RVAM is commonly provided by state sources. However, funding can be obtained from federal sources for specific purposes, such as disaster relief, initial landscaping postconstruction, and other specific purposes. Generally, maintenance activities are not allowed to be included with federal funding; however, some state DOTs are beginning to request maintenance funding during establishment periods with requests for postconstruction funding, and changes at the federal level can provide additional funding for maintenance activities. Common funding sources are state and federal gas taxes and state vehicle taxes (Dumortier et al. 2017). The funds are managed by state legislatures, the FHWA, or the United States DOT, depending on the source.

Federal funding is typically not earmarked specifically for DOT use; however, a DOT can apply for funds that are earmarked for public land and state agencies. The Infrastructure Investment and Jobs Act (IIJA) provides funds for resiliency, including those for coastal resiliency, ecosystem restoration, and weatherization, and requires the U.S. DOT to provide grants to eliminate or control invasive plants while supporting pollinators through seeding/planting native, locally appropriate species (Congress.gov, 2021). Following the passing of the IIJA in 2021, funds became temporarily available for the removal of noxious and invasive plant species, supporting fish passage, pollinator habitat installation and management, managing wildlife corridors, and more through grant programs.

Funding for RVAM may need to be pieced together through multiple sources, both federal and state, to ensure the proper quantities of funds are available for RVAM activities. DOTs can review the application requirements to ensure that the chosen funding sources are appropriate and to increase the chances of receiving the funding as desired.

As with any funding request, clear and distinct goals can increase a DOT’s chance of being selected. A state DOT that cannot provide these goals may receive less funding than state DOTs with clear goals. Any organization providing goals should have definitions that are applicable across the state and align with the definitions of the organization providing the funding. The lack of common definitions can reduce the awarded funding. During the literature review, the research team was unable to locate a standardized definition for Roadside Vegetation Asset Management. In response, the team developed a definition for use within this project with the assistance of the NCHRP project panel.