CHAPTER 10
Data and Performance Measurement

Implementing SUM services in rural areas comes with a set of unique challenges and opportunities that differ significantly from those in urban environments. One of the most critical components of successfully deploying these services is the establishment of a comprehensive data collection and performance measurement framework. In rural settings, where populations are often more dispersed, transportation infrastructure is less developed, and resources are limited, accurate and comprehensive data becomes essential. This data not only plays a crucial role in understanding the specific mobility needs of the community but also in optimizing service delivery, identifying and addressing service gaps, and ensuring that SUM services are both effective and efficient.

By gathering and analyzing data on various aspects of service delivery such as ridership patterns, trip frequency, customer satisfaction, and operational performance, agencies can gain valuable insights that inform decision-making processes. For example, understanding which routes are most utilized or identifying peak times for service demand enables better resource allocation and the potential for service adjustments that can enhance the overall effectiveness of SUM services. Additionally, this data is vital for developing KPIs, which serve as benchmarks for evaluating the success of the services and guiding future improvements. In rural areas, where challenges such as low population density and limited infrastructure are prevalent, data-driven strategies can help maximize the impact of SUM services.

The importance of data and performance measurement in the implementation of SUM services in rural areas cannot be overstated. In rural regions, where traditional public transportation options may be scarce, SUM services offer a flexible and adaptive solution to meet the diverse needs of the community. However, without a strong data-driven approach, these services risk being underutilized, inefficient, or failing to meet the communityʼs needs effectively.

Data serves as the foundation for evidence-based decision-making. It enables agencies to make informed choices about where to allocate resources, how to scale services, and what adjustments are needed to better serve the population. Regular performance measurement ensures that services remain aligned with the communityʼs expectations and evolving needs, providing a mechanism for continuous improvement. This process is particularly important in rural areas, where limited funding and resources make it imperative that every decision is backed by accurate data to maximize the impact and sustainability of SUM services. By prioritizing data collection and performance measurement, agencies can ensure that their services are not only effective in the short term but also sustainable and scalable in the long term, ultimately improving the quality of life for rural residents.

Data Management

When developing and implementing SUM services in rural areas, it is essential to address the specific data needs early in the process, particularly during the RFP and service agreement stages. Agencies should work closely with SUM partners to define the requirements for data

reports, fields, and the level of granularity needed for effective service monitoring and improvement. This includes specifying the types of data that need to be collected, the frequency of data reporting, and the formats in which the data should be delivered.

Data sharing between parties must be governed by a formal business agreement, ensuring that all participating members, including vendors, understand their responsibilities regarding data management. Such agreements should outline how data will be shared securely, including the use of secure channels for transmitting sensitive information, to protect against unauthorized access or breaches.

When managing data, especially when it involves sensitive information, agencies must carefully consider and comply with the privacy regulations of customers; an example is the Health Insurance Portability and Accountability Act in relation to medical transportation. This is particularly critical when SUM services interact with medical facilities for appointment scheduling or involve Medicaid programs. Compliance with these regulations ensures that the privacy of individuals is protected, and that the agency operates within the legal frameworks required for handling sensitive data.

Steps for data management plan development:

1. Identify data access, processing, and collection capabilities. As part of the RFP process, agencies need to outline the specific data access, processing, and collection capabilities required from bidding vendors. This ensures that vendors can meet the agencyʼs data management needs and provide the necessary infrastructure for effective data handling.
2. Specify datasets and potential analyses. Agencies should detail the specific, ideal, and additional datasets that the vendor could provide, along with an explanation of which types of analyses each dataset will facilitate. This clarity helps ensure that the data collected is aligned with the agencyʼs performance measurement goals and can be used to generate valuable insights.
3. Develop a comprehensive data management plan. A well-developed data management plan is crucial for managing any data generated by the SUM service or provided by the vendor. This plan should cover all aspects of data handling, including storage, processing, analysis, and reporting. It should also address data governance, security protocols, and compliance with relevant regulations.
4. Plan for data collection before, during, and after service launch. Agencies should create a plan for how data collection will be conducted at various stages prior to service launch, at the time of launch, and on an ongoing basis. This plan ensures that data is consistently gathered to monitor service performance over time and make informed decisions for service adjustments and improvements.

Some SUM vendors offer advanced capabilities that allow agency users to create custom reports and export raw data independently. These features can be highly beneficial, enabling agencies to tailor reports to their specific needs and conduct in-depth analyses. Additionally, integrations between GPS and computer-aided dispatch (CAD) platforms can enhance data tracking and reporting capabilities, providing agencies with more accurate and detailed insights into service performance. These integrations can support real-time data monitoring and improve the overall efficiency of SUM services by offering a more comprehensive view of operational metrics.

Integrations and Planning

Rural transportation agencies can greatly benefit from integrating online dashboards and user satisfaction metrics into their technology platforms. These dashboards not only provide real-time data but also facilitate the monitoring of service performance and customer feedback. By having access to user satisfaction information, agencies can make data-driven decisions to improve service quality and address any issues promptly.

Data integration is a crucial step in building robust rural mobility-as-a-service (MaaS) systems in the future; this involves connecting multiple trip planner applications and payment options into a unified platform, enabling seamless transitions between different transportation modes. Such integrations will help create a more cohesive and efficient service environment, allowing for better resource utilization and an enhanced customer experience across various transportation providers.

Bikeshare programs in rural areas generate a wide range of data, including sign-out sheets, raw comma-separated values (CSV) data on trip usage and registrations, and origin-destination points of trips. Additionally, “breadcrumb data,” which tracks the paths taken during trips, can be extremely useful for planning purposes if it is available from the technology. This data may be useful in identifying where additional bike lanes may be needed and optimizing the placement of bike stations to improve accessibility and convenience for users.

Performance Measurement

Framework Development

The first step in developing a performance measurement framework for SUM services is to identify the key areas of internal importance that will be evaluated. These areas should align with the agencyʼs overall mission and objectives, focusing on aspects such as operational efficiency, customer satisfaction, cost-effectiveness, and service reliability. By pinpointing these critical areas, agencies can ensure that their performance measurement efforts are focused on the most impactful elements of the SUM service.

Next, it is essential to review the goals that were established for the SUM service during the planning stages. These goals serve as the foundation for the performance measurement framework, guiding the selection of metrics and evaluation criteria. Reviewing these goals helps ensure that the performance measurement process remains aligned with the original vision and objectives of the service, providing a benchmark against which progress can be assessed.

To create a performance measurement framework that truly reflects the needs and expectations of the community, it is crucial to incorporate community input into the evaluation areas. This can be achieved through surveys, focus groups, public meetings, and other forms of engagement that solicit feedback from the serviceʼs users. By integrating the communityʼs voice into the evaluation process, agencies can identify areas that are most important to residents, ensuring that the SUM service is meeting its intended goals and providing value to the community.

Finally, once the key areas for evaluation have been identified and community input has been considered, the next step is to determine specific, measurable metrics within each topic area. These metrics should be quantifiable and objective, allowing for accurate tracking and comparison over time. Examples of measurable metrics include ridership numbers, on-time performance rates, customer satisfaction scores, and cost per trip. By establishing clear metrics, agencies can systematically monitor the SUM serviceʼs performance, identify trends, and make data-driven decisions to enhance service delivery.

Data Availability

The first step in ensuring data availability for performance measurement is to identify the specific datasets and sources that will be used to evaluate the framework metrics. This involves determining which types of data are necessary to measure each of the KPIs identified in the framework. Data sources may include internal systems such as GPS tracking, fare collection records, ridership logs, and customer service reports, as well as external sources like third-party data providers or government databases. Clear identification of these datasets ensures that the necessary information is available to accurately assess the performance of the SUM service.

Once the relevant datasets have been identified, it is crucial to specify the availability and responsibilities associated with datasets that are internal to the agency. This includes determining which departments or teams within the agency are responsible for collecting, managing, and maintaining each dataset. It also involves ensuring that these datasets are readily accessible to those who will be conducting performance evaluations. Establishing clear responsibilities and access protocols helps streamline the data collection process and ensures that accurate and up-to-date information is always available for performance measurement.

In many cases, SUM services rely on datasets that are managed by external parties, such as service providers, vendors, or partner organizations. It is important to specify the availability and responsibilities of these external datasets as part of agreements or contracts. These agreements should clearly outline who is responsible for providing the data, the frequency of data updates, and the format in which the data will be delivered. Additionally, the agreements should include provisions for data security and confidentiality, particularly when dealing with sensitive information. Ensuring that these details are addressed in contracts helps prevent delays or issues with data availability during the performance measurement process.

Incorporate Data from Public Surveys, Feedback, and Other Avenues

To create a comprehensive and well-rounded performance measurement framework, it is essential to incorporate data from public surveys, feedback, and other community engagement avenues. This type of data provides valuable insights into the customer experience, helping gauge public perception and satisfaction with the SUM service. Public input can also highlight areas that may not be fully captured by operational data alone, such as accessibility concerns, service awareness, or specific community needs. By integrating this qualitative data with quantitative metrics, agencies can gain a more complete understanding of the SUM serviceʼs impact and effectiveness.

Performance Assessment

The first step in performance assessment is to identify baselines or set targets for each metric within the performance measurement framework. Baselines are established by analyzing historical data or industry benchmarks to understand the current state of the service, while targets are set based on the desired outcomes and goals identified during the planning stages.

These benchmarks provide a reference point against which the serviceʼs performance can be measured, helping determine whether the SUM service is meeting expectations and where improvements may be needed.

To maintain consistent oversight of the SUM service, it is crucial to specify the frequency of performance evaluations. Regular evaluations, whether they occur monthly, quarterly, or annually, ensure that any issues or trends are identified early and addressed promptly. The frequency of these evaluations should be aligned with the serviceʼs operational cycles and the availability of relevant data, allowing for timely adjustments and continuous improvement.

In some cases, certain metrics or topical areas may be more critical to the serviceʼs success than others. Agencies should consider determining the weighting for these metrics to reflect their relative importance in the overall performance assessment. For example, customer satisfaction might be weighted more heavily than ridership numbers if the primary goal of the service is to improve user experience. Weighting allows for a more nuanced analysis of performance, highlighting the areas that have the greatest impact on the serviceʼs success.

As part of the performance assessment, agencies should use data-driven insights to determine whether additional technology is needed to improve SUM services. By analyzing performance metrics such as service efficiency, rider demand, and customer satisfaction, agencies can identify specific technology gaps that may be limiting the serviceʼs effectiveness. For example, if performance data reveals delays in service response times or challenges in optimizing routes, agencies may consider integrating technologies like predictive analytics, real-time tracking, or dynamic scheduling systems to address these issues. These performance-driven evaluations help ensure that technology enhancements are targeted, directly addressing identified operational inefficiencies and improving overall service delivery.

Infrastructure has a direct impact on the performance of SUM services, especially in rural areas where road conditions, connectivity, and available transportation facilities can vary significantly. During performance assessments, agencies should carefully evaluate how infrastructure limitations such as poor road quality, insufficient bike lanes, or outdated transit hubs are affecting service efficiency, safety, and reliability. By analyzing performance data, agencies can pinpoint specific infrastructure challenges that may be hindering SUM services and determine where improvements are needed. Addressing these infrastructure impacts is essential for maintaining consistent service quality and ensuring that SUM services can operate efficiently across diverse rural environments.

Finally, agencies should be prepared to determine possible actions for service adjustments based on the performance data collected. This could include altering routes, adjusting service hours, reallocating resources, or implementing new technologies. The goal is to use the insights gained from performance assessments to make informed decisions that improve the service, better meet community needs, and ensure the long-term sustainability of the SUM program. Regularly revisiting and refining the performance measurement framework will allow the service to evolve and adapt to changing conditions and demands.

Impact Communication

One of the critical aspects of performance measurement is ensuring that the public is kept informed about how the SUM service is performing. To achieve this, agencies must specify the communication channels and frequency through which SUM performance updates will be shared with the public. Common avenues include agency websites, social media platforms, community newsletters, public meetings, and local media outlets. The frequency of these updates should be consistent, such as quarterly or biannually, to maintain transparency and build trust within the community. Regular communication helps keep the public engaged and informed about the successes, challenges, and ongoing improvements of the SUM service.

It is equally important to communicate any changes to the SUM service that result from performance assessments. When adjustments are made, whether changes in routes, schedules, or service features, agencies should clearly explain the reasons behind these changes and how they are intended to improve the service. This communication should be proactive and transparent, ensuring that the community understands that their feedback and the performance data are driving continuous improvement. Providing detailed explanations of service changes not only helps manage public expectations but also reinforces the agencyʼs commitment to enhancing the SUM service in response to performance metrics and community needs.

Replicability and Scalability

The replicability and scalability of data and performance measurement strategies are critical components in the success of SUM services in rural areas. Data-driven approaches offer a blueprint for agencies to replicate and expand their services across regions with similar characteristics. Establishing a robust data collection and performance measurement framework allows agencies to monitor service effectiveness, optimize operations, and make informed decisions about resource allocation. By using data to track key performance metrics such as ridership patterns, service reliability, and customer satisfaction, agencies can identify areas of success and areas needing improvement—making it easier to scale services in a structured and evidence-based manner.

For scalability, rural SUM services must leverage technology-driven tools such as dynamic scheduling, real-time tracking, and data analytics to ensure efficient service delivery even as demand fluctuates. These tools not only support operational efficiency but also facilitate the collection of real-time performance data, which is essential for adjusting services as they expand. As agencies gather and analyze this data, they can implement changes that allow SUM services to scale without compromising quality.

Replicability of data and performance measurement strategies is key to spreading successful SUM service models across multiple rural regions. Agencies can document best practices and lessons learned from initial implementations to create replicable frameworks that other regions can adopt. By standardizing data management processes, performance evaluation methods, and the technology used for monitoring services, agencies can create a repeatable model that allows SUM services to be expanded and adjusted to meet the unique needs of rural areas. This approach helps build a more sustainable and connected transportation network, ensuring that performance-driven strategies can be easily scaled and replicated across diverse rural communities.

Key Takeaways

• Data-driven decision-making. Effective implementation of SUM services in rural areas depends on robust data collection and performance measurement frameworks. These frameworks empower agencies to make informed decisions regarding resource allocation, service adjustments, and overall service improvements, ensuring that services remain responsive to community needs.
• Technology integration is crucial. Leveraging advanced technologies such as dynamic scheduling, real-time tracking, and data analytics is essential for optimizing operations and enabling the scaling of SUM services in rural areas. These technologies also streamline data collection, making it easier to assess performance and improve service delivery.
• Performance metrics alignment. The success of SUM services hinges on the alignment of performance metrics with community goals. By establishing clear KPIs that reflect the serviceʼs objectives such as ridership, cost-effectiveness, and customer satisfaction, agencies can continuously monitor and improve performance.
• Consistency in data reporting and evaluation. Regular and consistent data reporting is crucial for long-term service improvement. Establishing regular evaluation intervals, whether monthly, quarterly, or annually, ensures that agencies can detect emerging trends, address issues promptly, and adapt to changing conditions.
• Continuous community engagement. Maintaining regular communication with the public regarding SUM performance and service changes builds trust, ensures transparency, and fosters ongoing community support. Community feedback, in turn, becomes a valuable source of data for refining services and ensuring long-term success.
• Data sharing and collaboration. In rural areas, data sharing between agencies and partners is essential for improving service coordination. Formal data-sharing agreements help facilitate this process, ensuring that agencies have access to relevant data for comprehensive performance evaluations and service optimizations.

Additional Resources

The agreement templates described in Appendix B are critical for establishing robust data and performance measurement systems for SUM services. The document templates for RFPs and Service Agreements contain examples of what data and reporting might be requested from a SUM provider and help to ensure clear protocols for secure and consistent data exchange between agencies, vendors, and partners. Additionally, SLAs outline specific performance metrics and reporting requirements, enabling agencies to monitor compliance with KPIs such as ridership, on-time performance, and customer satisfaction. These templates foster accountability and provide a framework for structured, data-driven service improvements.

The Implementation Checklist described in Appendix C offers essential guidance for integrating data and performance measurement into SUM service operations. It includes steps for developing a comprehensive data management plan, defining measurable performance metrics, and scheduling regular data reporting and evaluation intervals. These checklist items ensure that agencies have a systematic approach to collecting, analyzing, and using data to optimize services, identify gaps, and sustain improvements over time. By following these steps, agencies can build a foundation for continuous performance monitoring and long-term service success.