Appendix F

2014 Calibration Workshops

MRIP Calibration Workshop II

Appendix 1. Detailed Implementation Steps for the Calibration Methods Proposed During the Workshop.

Summary Report: NOAA Calibration Methods Workshop - Charleston, SC
September 8-10, 2014
Lynne Stokes, Ken Pollock, Ginny Lesser
December 18, 2014

The new MRIP Access point survey has replaced the original MRFSS Access Point Survey, A variety of design changes have been made. One major consequence is that the new survey covers the fishing day more effectively than the original MRFSS Access Point Survey. Because the time series of recreational catch rate estimates form the basis of so many important fisheries stock assessments, there is the need to develop methods which "calibrate" the original time series of MRFSS estimates to the new time series of MRIP estimates. This is a difficult statistical estimation and prediction issue because both surveys were not run in parallel in any years (except for one pilot test in NC). The new estimates can be very different from the old estimates causing an abrupt change in the time series.

The purpose of this document is to outline the steps involved in implementing several model dependent calibration approaches to re-estimate catch that were discussed at the Charleston workshop. In addition, we discuss their assumptions. The first two methods use ideas of ratio estimation and assume that the major changes between the two surveys are due to a better temporal coverage of the fishing day in the new MRIP survey. The third method is a regression prediction modeling approach that will take longer to develop. None of these methods incorporate any analysis of spatial patterns or include time series methods, which might improve estimates. This would be worth exploring to determine if time series or small area estimation techniques for this short time series might provide improved estimates.

1. Direct Catch Ratio Adjustment
   • Steps in approach (for each subregion, state, mode, species.):
     1. Define peak period for each of the domains (excluding species). Peak period is defined using two criteria: 1) the contiguous range of hours during which weighted hourly proportions of total trips in the MRFSS years (prior to 2013) were greater than or equal to the corresponding weighted hourly proportions of total trips in 2013, and 2) the peak period accounted for at least 75% of the intercept data (trips) in the MRFSS years.
     2. Estimate peak and total catch using the 2013 data based on the MRIP survey method where both the peak and total fishing periods were sampled adequately. Denote these by c_p,2013 and c_total,2013, respectively.

2. • 3. Calculate the ratio R₂₀₁₃ = c_total,w2013/c_p,2013. This estimate and its large sample variance, based on standard Taylor series methods, can be calculated from survey sampling software packages such as SAS.
     4. Denote the estimator of catch based on the MRFSS method during the peak period in earlier year y(e.g,, y = 2012, 2011, etc.) by c_P,y. Then the estimator of adjusted total catch for year y (i.e., a prediction of what would have been obtained if MRIP had been run) will be calculated as the product of the ratio from year 2013 and the catch for the peak period in year y; i.e.,

        C_tot,y = R₂₀₁₃*C_p,y.

        iv. The variance of the adjusted catch c_tot,y can be calculated using the expression for the variance of a product of two independent random variables introduced by Goodman (1960):.

        var(c_tot,y) = var(R₂₀₁₃)(c_p,y)² + var(c_p,y)(R₂₀₁₃)² - var(R₂₀₁₃)var(c_p,y)

        By substituting estimates for each of the components in this equation, the variance can be estimated.

   • Assumptions:
     1. Relative distribution of catch throughout day (i.e., between peak and total) is constant between 2013 and the year that is being adjusted for each domain
   • Advantages:
     1. Simple to apply.
   • Disadvantages:
     1. Information that is available for non-peak hours are not used.
   • Two variations of this approach:
     1. Keep a fixed peak time the same (note this will vary by state and mode)
     2. Use different peak times (allow this to vary by state, mode and year since this was allowed to vary in these groups)
3. Complex Ratio Method Based on Fishing Effort Distributions
   • Steps in approach (for each subregion, state, mode, species etc.):
     1. The 2013 daily relative distribution of total fishing effort is obtained and also the relative distribution of total fishing effort data for the year to be compared to (for example, for y = 2012, 2011, etc.). Total fishing effort is estimated as the fishing effort estimate from separate telephone surveys (CFITS, FHS) that is subsequently expanded by coverage correction factors estimated from APAIS.

3. • Advantages:
     1. Information that is available for non-peak hours are used unlike in the previous method.
   • Disadvantages:
     1. Information from non-peak hours will be limited and may be highly variable or impacted by incomplete coverage compared to information from peak hours.
     2. The assumptions under which this estimator will be consistent (that is, will provide an unbiased estimate for a sufficiently large sample size) are unknown at this time. For example, if the (strong) assumption needed for Method 1 is assumed, the estimator will still not necessarily be consistent.
   • Other ideas to consider as variations of above
     1. Recalculate catch after effort has been readjusted. Therefore, both catch and effort are readjusted. The calibration methods make use of the MRIP public-use or micro datasets. The records included in these datasets come from APAIS. However, the sample weights in these datasets include a post-stratification adjustment such that the sum of the sample weights equals the MRIP estimate of total effort in domain cells defined by year, subregion, state, wave, mode, and area. To more fully approximate the effect of temporal coverage changes on catch, the MRIP estimates of total effort must be recalculated since they also include coverage correction factors estimated from APAIS. Once total effort has been recalculated, sample weights may be post-stratified to the new effort totals, and then revised catch estimates may be calculated as weighted sums using sample weights that have been adjusted to both a prior year daily distribution of effort as well as the resultant new effort total.
     2. Apply temporal distribution either year-by-year or as an average across a range of years (say 2004-2012). Then multiply this ratio by MRFSS estimates of catch in previous years. NOTE: If use each year separately, then there is no assumption that the relative distribution of catch is constant throughout the day across years, only the two years that are compared. So if only one year violates this assumption, then conducting an aggregate analysis could bias the estimator for the other years, while if it was done separately, only it would be biased by that assumption violation. Conversely, using a multi-year average distribution may work to smooth results in cases where annual level distributions may be more variable.
4. Regression Model-Based Approach
   • Steps in approach:
     1. Develop a regression model using 2013 intercept data (perhaps other years as well) to predict and classify trips into either morning, peak, or evening as predicted from

Catch can also be added to model, but need to be careful of applying 2013 year affects to previous years.

References: