CHAPTER 2

Using the Fare-Free Transit Evaluation Framework

This evaluation framework presented herein provides guidance for transit agencies and other interested parties on evaluating the feasibility of various fare-free transit alternatives.

The evaluation framework has 10 steps that describe the process for getting organized, making a plan, and evaluating fare-free transit (Exhibit 2-1).

Transit practitioners can focus on evaluating the impacts of fare-free transit across four themes: access, mobility, and equity; operational efficiency; financial health; and community impacts. Exhibit 2-2 shows these themes along with example questions that can guide the evaluation. Many of the steps include examples that are organized around these themes. These examples are not intended to be prescriptive but may be used as models. More information about the impacts, categorized by theme, can be found in Chapter 3.

The framework can be used when a transit agency is interested in evaluating fare-free transit alternatives. It can be used from the beginning of this process up until the decision to implement (or not to implement) a preferred alternative. Because the framework follows the general principles of program evaluation, it may also be used after implementation to evaluate the long-term

feasibility of fare-free transit. General guidance for using this evaluation framework after implementation is provided. Whether the framework is being used before or after implementation, the project team should allocate sufficient time to complete the steps and plan the evaluation process accordingly.

Two types of callouts are provided throughout the evaluation framework to share examples of how the framework was used in practice (“Framework in Practice”) and opportunities to carry out outreach to stakeholders and the public (“Outreach Opportunity”). See Exhibit 2-3 for examples of these callouts.

Get Organized

At the onset of the process, those interested in evaluating fare-free transit alternatives should get organized with the following steps:

• Step 1: Set the context for the evaluation of fare-free transit, including identifying relevant context and forming a project team.
• Step 2: Assess the existing conditions of the transit agency.
• Step 3: Review peer transit agencies that have considered, evaluated, and/or implemented fare-free transit policies or programs.
• Step 4: Identify opportunities and challenges for fare-free transit policies and programs in your transit agency or community.

Step 1: Set the Context

Identify Relevant Context

Those interested in evaluating fare-free transit programs and policies should start by documenting relevant context that is specific to the transit agency and surrounding community. This context will help guide the scope and design of the evaluation process. The following questions should be explored:

• What goals does the transit agency have for transit service and/or fare policy?
• Has the transit agency evaluated fare-free transit in the past? What were the outcomes?
• What is the decision-making process for implementing fare-free transit?
• Who are the local advocates for fare-free transit (e.g., elected officials, transit decision makers, transit agency leadership, transit agency staff, and community representatives) (see the discussion in “Framework in Practice: MBTA Fare-Free Pilot”)?

• What are the advocates’ goals for fare-free transit (e.g., increase ridership, promote equity, reduce fare collection costs, and reduce emissions)?
• Are the advocates backing a specific fare-free transit policy or program? Are they open to a variety of fare-free transit alternatives?
• What do advocates, transit agency staff, decision makers, and community partners want to know?
• What stakeholders should be involved in the evaluation process (see the discussion in “Outreach Opportunity: Stakeholder Identification and Engagement”)?

Assemble Project Team

Based on the relevant context, the transit agency should form a team of staff and community partners with the knowledge, resources, and time to complete an evaluation of fare-free transit (hereafter referred to as the “project team”). Valuable transit agency staff may include representatives from multiple departments including operations, security, auditing, service planning, IT, communications, finance, civil rights, and labor relations. Useful community partners may include staff from municipal departments, MPOs, or neighboring transit agencies. The size and composition of the project team will be unique for each transit agency. The make-up of Los Angeles Metro Transportation Authority’s (LA Metro’s) project team can be seen in “Framework in Practice: LA Metro Fare-Less System Initiative Task Force.”

Step 2: Assess the Existing Conditions

The project team should collect relevant information on the existing conditions of the transit agency and its fare collection system. The data collected in this step can be referenced to identify transit-agency-specific opportunities and challenges in Step 4, inform the selection of fare-free transit alternatives in Step 8, and estimate the impacts in Step 9.

Potential elements of the existing conditions analysis include documenting recent ridership and performance trends, planned service or network changes, transit funding, fare structure and policy, fare usage, fare collection costs, and community demographics. The project team should determine the specific elements of the existing conditions analysis based on operating context and available data. It is easy to spend a lot of time analyzing existing conditions data, so the project team should aim to strategically identify necessary elements for the evaluation. Key existing conditions findings from Sun Tran’s fare study can be found in “Framework in Practice: Sun Tran Analysis of Existing Conditions.”

Explore Ridership, Revenue, and Cost Trends

Gathering and documenting performance data can help the project team set a baseline understanding of how many people are using the transit system, how much services cost, and how much revenue the transit agency obtains through collecting fares. An analysis of performance trends over time (e.g., 5 years) provides an opportunity to predict future performance if the transit agency continues to collect fares.

Example questions for this analysis are the following:

• What are the trends in ridership, operating costs, cost per trip, fare revenue, and farebox recovery over the last 5 years?
• What is considered farebox revenue, aside from passenger fares (e.g., local partnership contributions)?
• How do the trends differ across various routes and services?
• What is the forecasted ridership and farebox revenue for future years?
• How is ridership and farebox revenue forecasted?

Identify the Benefits and Challenges of Existing Funding Sources

Fare-free transit results in a shift in the way that transit service is funded. The project team should document the existing funding structure, including the benefits and challenges of the various sources. Assessing the benefits and challenges should involve looking at whether the funding sources are stable, dependent on system performance (e.g., ridership, productivity, and farebox recovery), and regressive (i.e., place a larger financial burden on riders with low incomes). Additionally, the team can begin to identify funding opportunities for replacing fare revenue.

Example questions for this analysis include the following:

• What are the funding sources and trends of transit agency funding aside from fare revenue (e.g., federal, state, local, and community sources)?
• What are the benefits and challenges of each of these funding sources?
• Have there been previously identified opportunities for alternative revenue sources at the federal, state, local, or community level?

Measure Operating Efficiency

Fare-free transit may result in increased transit ridership and changes in rider behavior, which may impact transit operations. These impacts may be positive (e.g., higher productivity) or negative (e.g., overloaded vehicles, late buses, and security issues). The project team should analyze existing operations data to determine where additional ridership may benefit or challenge the system. Additionally, the project team should document how the transit agency currently collects rider data (which may be impacted by the elimination of fares) and what policies exist for guiding passenger behavior.

Example questions for this analysis include the following:

• What is the existing ridership productivity on the system? Does it differ across different services or routes?
• Does the transit agency have any planned service changes?
• Is there underused or limited passenger capacity? If so, where?
• Is there operator and staff capacity to handle the potential need for increased service due to high ridership? What would be the cost of meeting operator staffing needs?
• Does the transit agency currently have on-time performance issues (i.e., are buses often late or running early)?
• How does the transit agency currently collect data related to passenger counts? Will this be affected by eliminating fareboxes?
• What are the current rider policies that guide passenger behavior (e.g., codes of conduct, destination policies)?

Consider Community Demographics

Because fare-free transit can be a tool to promote more equitable access to transit, the project team should analyze existing community demographics to understand who would benefit from fare-free transit. The team should look at both service area and ridership demographics to identify how, if at all, the representation of certain populations will vary. For example, if a larger percentage of transit riders have low incomes than the population within the service area, then fare-free policies and programs may have a positive impact on people with low incomes. If the transit agency has a Title VI plan, the team should focus on populations identified in that plan. Additional demographic and socioeconomic factors such as race, age, ability, cost of living, and spending patterns on necessities like housing and transportation, should also be included.

Example questions for this analysis are the following:

• What percentage of current transit riders belong to historically marginalized populations (e.g., people with low incomes, people of color, those with no vehicle access, older adults, and persons with disabilities)?
• Do rider demographics vary across routes and/or services?

• What percentage of the transit service area population belongs to historically marginalized populations?
• On average, what percentage of household income is spent on housing and transportation costs in the transit service area [the project team could consult the Center for Neighborhood Technology (CNT) Housing and Transportation (H+T) Affordability Index]?

Document Fare Structure and Policy

The project team should document the existing fare structure, as well as any governing fare policy that dictates goals for fare collection or guidelines for making fare changes. The project team should identify any existing groups that are eligible for fare discounts or other local programs to benefit people with low incomes.

Example questions:

• What are the transit agency’s current fare structure and policy?
• Which groups currently receive discounts or free fares?
• How much is spent to administer existing discount programs?
• Are there pass partnerships that exist that provide discounted or subsidized fares to certain riders (e.g., employer, school, and university partnerships)?
• Are there other discount programs for people with low incomes among partner agencies (e.g., municipal utility programs and neighboring transit agencies)?
• What are the fare structures of any neighboring and regional transit agencies? What are the transfer policies?

Examine Fare Usage

The project team should analyze how the different fare types are being used and their respective contribution to the transit agency’s fare revenue. This analysis will give the project team an understanding of the proportion of fares that are already subsidized, who is receiving the various subsidies, and how existing discounts are being utilized.

Example questions:

• What percentage of riders pay full, discounted, or free fares?
• What percentage of riders receive discounted or free fares from partnerships?
• What proportion of eligible riders participate in existing discounted or fare-free programs? Are there potential barriers that are blocking participation?
• What percentage of revenue comes from riders who belong to historically marginalized populations?
• How many riders are transferring between neighboring transit agencies?

Quantify Existing and Future Fare Collection Costs

Determining the full cost of existing and future fare collection costs is an important element of evaluating the costs and benefits of full fare-free transit. When calculating the full cost of existing fares, transit agencies should consider the costs of administering, selling, collecting, counting, and securely transporting fares, as well as the hardware and maintenance required for farebox equipment. Direct expenses as well as the opportunity costs of staff time spent administering fares should be calculated.

For future fare collection costs, transit agencies should calculate costs for any anticipated operating and capital needs such as updated fareboxes, ticket vending machines, fare media, software,

and maintenance. If no existing planning has been completed, transit agencies can look at how many years of useful life remain on existing fare collection technology to estimate when it will need to be replaced. To determine the costs of future upgrades, transit agencies can explore the costs of upgrading their current fare collection technology to a more recent model or look to peer transit agencies for best practices on advancing fare collection technology.

Example questions:

• What are the operating and maintenance costs of the existing fare collection system including staff time, services, and technology?
• What is the remaining useful life of the existing fare collection equipment?
• Does the transit agency have any plans to update the fare collection equipment and, if so, what are the estimated costs?
• What percentage of revenue is or will be spent on fare collection?

Step 3: Review Peer Transit Agencies

The project team should look to peer transit agencies that have evaluated or implemented fare-free policies or programs to identify relevant fare-free context, goals, evaluation, outcomes, and program funding. The project team can look to transit agencies of similar sizes and in similar operating environments (e.g., region, city size, community demographics, university town, or resort town) or those with noted best practices in fare-free evaluation. The project team can also reference the findings from the transit agency survey and case studies presented in Chapter 4 of this report. “Framework in Practice: Ride On Fare Study Peer Review” presents some of the findings from the peer review that was part of Ride On’s Ride On Zero & Reduced Fare Study (MCDOT 2021).

Example questions the project team can explore during the peer transit agency review include the following:

• What was the context that led to the evaluation or implementation of fare-free transit? Who advocated for the policy or program? What were the relevant existing conditions that facilitated or impeded fare-free transit?
• Did the transit agency identify any goals for fare-free transit?
• Did the transit agency evaluate fare-free transit before implementation? If so, did it collect any notable data, make any assumptions, or utilize any interesting tools in its evaluation? What were the results of the evaluation?
• Did the transit agency go forward with full or partial fare-free transit? What were the reasons for selecting one option or the other? If the transit agency implemented partial fare-free transit, has the transit agency thought about going fully fare-free?
• After implementation, did the transit agency experience any benefits or challenges? Were these outcomes measured with qualitative or quantitative data?
• How was the transit agency able to replace foregone fare revenue? Was funding secured before implementation of fare-free transit? Did the transit agency utilize any notable local, state, or federal funding sources? Did it form any new partnerships? Was the transit agency able to increase the local contribution from a governing municipality?

Step 4: Identify Opportunities and Challenges

The project team should identify the opportunities and challenges for the transit agency related to fare-free transit policies or programs. These opportunities and challenges should be informed by transit agency context, existing conditions, and the peer review. Example opportunities and challenges can be seen in Exhibit 2-4. The project team can engage the public in this step to ensure the team accurately reflects the needs of the community (see “Outreach Opportunity: Public Engagement”).

Make a Plan

To ensure the evaluation is appropriate for the transit agency’s context, the project team should design an evaluation plan based on the identified opportunities and challenges. This planning includes the following steps:

• Step 5: Set evaluation goals and objectives to guide the scope of the evaluation.
• Step 6: Determine performance measures for each objective.
• Step 7: Establish selection criteria for determining feasibility.
• Step 8: Select fare-free transit alternatives to evaluate.

Step 5: Set Evaluation Goals and Objectives

The project team should set appropriate goals and objectives to define the scope of the evaluation based on the opportunities and challenges identified in Step 4 and feedback from stakeholders.

When setting evaluation goals and objectives, the project team can look to external guidance from the FHWA’s Performance-Based Planning and Programming Guidebook (Guidebook) (Grant et al. 2013). The Guidebook outlines a framework for public-sector agencies to use “performance information to make decisions that are more effective and efficient and lead to improved outcomes.”

The Guidebook defines the following performance management terms that are relevant to the evaluation framework for fare-free transit:

• A goal is a broad statement that describes a desired end state. When developing goals, transit agencies should be focused on intended outcomes, rather than specific policies.
• An objective is a specific, measurable statement that supports achievement of a goal. A good objective should include or lead to development of a performance measure that can be tracked over time and is used to assess different investment or policy alternatives.
• A performance measure is a metric used to assess progress toward meeting an objective. Performance measures can be used in strategy analysis to compare different investment or policy alternatives and can be used to track actual performance over time.
• A target is a specific level of performance that is desired to be achieved within a certain time-frame. A target can be used as a basis for comparing progress over time to a desired outcome or for making decisions on investments.

Example goals and objectives related to the four themes in fare-free transit impacts can be seen in Exhibit 2-5. Examples of implemented goals and objectives used during Iowa City Transit’s evaluation of fare-free transit are in “Framework in Practice: Goals and Objectives in Iowa City’s Transit Fare Study.”

Step 6: Determine Performance Measures

For each objective, the project team should determine which performance measures they want to include in the evaluation. These should be metrics that can be measured by the project team within the evaluation timeline and with available data. Performance measures for the example goals and objectives can be seen in Exhibit 2-6. The project team can engage the public and stakeholders during this step to gather feedback on appropriate measures. See an example from the Kansas City Area Transportation Authority (KCATA) in “Framework in Practice: KCATA Evaluation Outreach.”

For each of the selected performance measures, the project team should plan how they will measure impacts across the alternatives. This planning will help the team identify additional data needs, as well as any evaluation tools or processes needed to develop or access (e.g., ridership models, operating models, financial models, or regional travel models).

Step 7: Establish Selection Criteria

The final step in designing the evaluation plan is to establish appropriate criteria for determining the feasibility of a fare-free transit alternative. This gives the project team a chance to define thresholds for the performance of the alternatives across various performance measures. The level of detail included in the selection criteria will vary based on the program context, feedback received from stakeholders, and the transit decision-making process. A range of approaches can be seen in Exhibit 2-7. More detail on how to engage stakeholders at this step can be seen in “Outreach Opportunity: Stakeholder Input into Evaluation Plan.”

Step 8: Select Fare-Free Transit Alternatives

After designing the evaluation plan, the project team should select which fare-free policy or program to evaluate. Based on findings in the previous steps and engagement with stakeholders, the project team can identify opportunities to implement fare-free transit, whether it is full or partial fare-free transit for specific riders, services, routes, times of day, or geographies. For example, transit agencies with low farebox recovery and those with a large majority of riders who have low incomes, who are older adults, who have a disability, or who are student riders may want to evaluate full fare-free transit. Transit agencies with higher farebox recovery or with significant revenue from local employer pass programs may choose to evaluate partial fare-free transit.

For some project teams, the evaluation context will lead them to evaluate only one alternative. Other project teams may be interested in exploring multiple options like fare-free transit for all and fare-free transit for riders with low incomes. Regardless of the number of alternatives, all should be evaluated against a baseline alternative. The baseline alternative should look at the impacts of no change to fare collection across the performance measures.

For all alternatives, the project team should define the details of the proposed policy or program. For transit agencies that are interested in taking a data-driven approach to measuring success after implementation, the project team should explore defining the alternative as a pilot. The length of the pilot should be determined through conversations with stakeholders and available

funding. Some transit agencies may determine that permanent fare-free transit is a viable approach for them based on the evaluation context or an identified long-term funding source. For any partial fare-free alternatives, the project team should define details for each alternative. Details should include the following:

• Who is eligible for fare-free transit? How will eligibility be verified in advance or upon boarding?
• How will a fare-free program be administered? What will be the cost to administer the program? Are there any additional technology requirements?
• Where, when, and how can riders access fare-free transit?

An overview of the process for identifying fare-free policy and alternatives at DASH in Alexandria, Virginia, is in “Framework in Practice: DASH Fare Program Scenarios.”

Evaluate

After planning, the project team should evaluate the feasibility of the alternatives. This evaluation includes the following steps:

• Step 9: Estimate impacts of the alternatives using the evaluation plan.
• Step 10: Select preferred alternative using the selection criteria and appropriate decision-making process.

Step 9: Estimate Impacts

The project team should gather credible evidence across the various performance measures for each alternative and compare the impacts across the alternatives.

Gather Credible Evidence

To compare the impacts of the alternatives, the project team should first gather credible evidence per the evaluation plan. Credible evidence includes reliable, accurate, and trusted data. This evidence may be pulled from the results of the existing conditions assessment (e.g., fare collection costs) or gathered and analyzed specifically for the evaluation (e.g., ridership model, community survey). A sample process for estimating the impacts is shown in Exhibit 2-8. An overview of the steps the Link Transit project team took to evaluate full fare-free transit can be seen in “Framework in Practice: Link Transit Analysis Process.”

Compare Alternatives

After gathering credible evidence across the various performance measures, the project team should compare the impacts of the alternatives and apply the selection criteria as established in Step 7. These results should be presented in a clear format that can be easily interpreted by various stakeholder audiences. The project team may choose to present various levels of detail to different stakeholders. For instance, higher-level decision makers may only be able to interpret the relative impacts of the various alternatives as opposed to interpreting specific quantitative data. Transit agency staff, on the other hand, may want to see detailed results of the analysis to inform decision-making. An example of how Ride On shared the results of the Ride On Zero & Reduced Fare Study (MCDOT 2021) as prepared for MCDOT (Maryland) can be seen in “Framework in Practice: Ride On Alternatives Comparison.”

Step 10: Select Preferred Alternative

In the final step, the project team should guide the selection of a preferred alternative. This process will depend on the decision-making process identified in Step 1 but may involve engaging various stakeholders to confirm the recommended approach (see “Outreach Opportunity: Sharing Findings with Stakeholders and the Public”). If a fare-free transit alternative is selected as the preferred alternative, the project team will want to solidify details before implementation (e.g., timeline, funding, eligibility, program administration). Decision makers may also choose not to implement a fare-free transit policy or program in this step (see “Framework in Practice: Houston METRO’s Decision to Maintain Fares”).

Implement and Monitor Program

After deciding to implement a preferred fare-free transit alternative, the project team should take additional steps to ensure that evaluation continues past implementation. This evaluation differs in purpose from the evaluation steps in the evaluation framework. The evaluation steps in the evaluation framework focus on determining the feasibility of potential fare-free policy and program alternatives whereas the post-implementation evaluation focuses on determining the success of fare-free transit. This evaluation is especially important if the preferred alternative is a pilot to determine long-term viability. Suggested steps include the following:

• Design an evaluation plan. Design a plan to evaluate the success of a fare-free transit policy or program. This process should follow steps similar to the previously discussed Steps 5 through 7, including setting goals and objectives, determining performance measures, and establishing decision-making criteria.
• Implement fare-free transit. Take logistical steps to implement the fare-free transit policy or program including the following:
  • Make operational changes to halt fare collection, temporarily or permanently (e.g., stop printing fare media, revise driver and customer service training, decommission fare boxes and ticket vending machines, implement alternative ridership data gathering methods)
  • Adjust transit agency policies (e.g., change fare policy, strengthen rider codes of conduct, tighten paratransit eligibility)
  • Secure alternative funding sources (e.g., apply for grant funding, pass tax referendum, build partnership program)
  • Market fare-free transit (e.g., community education about benefits, informing riders)
  • Complete Title VI fare equity analysis to assess the impacts of any fare change on people of color and people with low incomes
• Evaluate fare-free transit. Gather credible evidence to support the evaluation plan.
• Determine success and long-term feasibility. Use the findings from the evaluation to determine whether the transit agency should continue to operate fare-free or adjust the policy or program.
• Continue to monitor and adapt. Continue to collect data to measure positive and negative impacts on ridership, transit agency operations, finances, and the community. Data collection methods may need to be adjusted if the system no longer relies on fare media.