RECREATIONAL FISHERIES SURVEY METHODS
F. Jay Breidt, Ph.D.
Department of Statistics
Colorado State University
Member, Committee on the Review of Recreational Fisheries Survey Methods
Ocean Studies Board
Division on Earth and Life Studies
National Research Council
The National Academies
Subcommittee on Fisheries, Wildlife, Oceans and Insular Affairs
Committee on Natural Resources
United States House of Representatives
May 21, 2013
Good morning, Mr. Chairman, Ranking Member Sablan, and members of the Subcommittee. My name is Jay Breidt. I am a professor of Statistics at Colorado State University, where I served as the Chair of the Department of Statistics from June 2005 until December 2010. I was also a member of the National Academies’ National Research Council (NRC) Committee on the Review of Recreational Fisheries Survey Methods in 2006. The National Academy of Sciences was chartered by Congress in 1863 to advise the government on matters of science and technology.
The NRC study was conducted in response to a request from the National Marine Fisheries Service (NMFS) for a review of methods used to collect and analyze recreational marine fisheries data for application to fisheries management.
The NRC formed a committee of ten experts in fishery science and statistics. Dr. Patrick Sullivan, an associate professor in the Department of Natural Resources at Cornell University, served as the committee chair. After the study was released, Congress amended the Magnuson-Stevens Fishery Conservation and Management Act and included provisions to improve data collection regarding marine recreational fisheries. This written testimony reviews some major points from that report, titled Review of Recreational Fisheries Survey Methods, and describes progress made by NMFS on revising marine recreational data collection since 2006.
Historically, marine recreational catch in the United States has been documented through the Marine Recreational Fisheries Statistics Survey (MRFSS), which was established by NMFS in 1979. As compared to commercial fisheries, collecting data on recreational fisheries is more difficult due to the number of recreational saltwater anglers, the diverse range of places in which they fish, and the many different methods of recreational fishing.
Recreational catch is computed by multiplying the number of recreational trips by the catch per trip. MRFSS uses two complementary surveys to estimate the two terms in this product. The number of trips, or effort, is estimated using an offsite survey, consisting of telephone interviews of anglers in coastal households. The catch per trip is estimated using onsite surveys, in which anglers are “intercepted” while they are fishing or at their access points. Biological samples are also collected from these onsite intercepts.
It is now evident that for some fish stocks, the recreational fishery represents a significant component of the total catch. Since the establishment of MRFSS, marine fisheries management goals, objectives and context have changed. Management decisions are often made at finer spatial and temporal scales, the mix of recreational and commercial fishing has changed for many areas and species, and stock assessment models now make greater use of data from recreational fisheries. Accurate and timely data on catch and effort levels in recreational fisheries is imperative to ensure the sustainability of popularly-targeted fish stocks.
NMFS’s request for a study recognized the limitations of the MRFSS program and the agency sought recommendations from the NRC on potential improvements and alternative approaches.
STUDY CONCLUSIONS AND RECOMMENDATIONS
The NRC committee concluded that the MRFSS program was not adequate to meet the current demand for data, in terms of quality and timeliness, required for effective management of recreational fisheries. The committee’s review focused primarily on MRFSS, but many related surveys conducted by state agencies suffered from the same limitations, and the committee’s recommendations applied to those surveys as well. Greater coordination among federal, state, and other survey programs was recommended to help gain a national perspective on the status of marine recreational fisheries.
Sampling Issues with the Telephone Survey
The committee identified several concerns with the telephone interview surveys. First, the increasing use of cellular telephones reduces the efficiency of the random-digit-dialing (RDD) telephone surveys conducted by MRFSS. The utility of RDD surveys targeted to coastal counties is reduced because cellular telephones are not geographically restricted (unlike land lines). Telephone surveys are also problematic because they depend on the accuracy of the angler’s memory and their willingness to provide information to the caller.
The committee determined that a comprehensive, universal sampling frame with national coverage would be an efficient way to improve the data. The committee also suggested that this could be implemented in the form of a national registry of saltwater anglers or, alternatively, a license program that allows for no exemptions. Telephone surveys would then be based on this more limited sampling frame, rather than the RDD frame which includes all households, not just those with saltwater anglers. The report also recommended consideration of dual-frame surveys; for example, combining a sample from an incomplete list frame of anglers with an RDD sample of all households to ensure complete coverage.
Sampling Issues with the Intercept Survey
The committee identified various shortcomings in the intercept (onsite) survey methods. These methods do not account for anglers who have access to private fishing areas, and operate on the assumption that data from private areas would be similar to the data collected at public access sites. The committee further recommended that the onsite sampling frame, or list of access points, should be revised to account for low-activity access points.
MRFSS onsite samplers were given considerable latitude in the selection of sites and the measurement protocols followed at a selected site. The committee noted that the sampling process required greater quality control, with less latitude on the part of the samplers.
Other Sampling Issues
In addition to the suggestions on survey design, the report suggested further research to provide more reliable estimates of the number of fish caught in catch and release fisheries as well as a clearer understanding of mortality rates for fish caught and not brought to the dock.
The committee concluded that all for-hire recreational fishing operations should be required to maintain logbooks of fish landed and kept, as well as fish caught and released. They should be required to provide the information in a timely manner to the survey program in order to remain eligible for operation, and all information provided should be verifiable.
Improving Statistical Estimation
The study found that the sampling designs and data collection methods of recreational fishing surveys fell short of what was needed for management. Unverified assumptions may have interjected biases into some survey estimates. Understanding the extent of such biases would require testing the assumptions and determining the direction of bias.
The report noted that current estimators of catch rate were likely to be biased, given a mismatch between the design and estimation procedures for the onsite survey. Further, the estimators of uncertainty associated with various survey products were likely to be biased and too low. The committee concluded that these properties should be determined, enlisting the expertise of an independent and permanent research group of statisticians for ongoing evaluation and advice on the design and adequacy of the survey methods.
Incorporating Trends in Where, When, and Why People Fish
Good surveying requires tracking data on the human dimension of fishing, including the social and economic factors that might affect the number and location of fishing access sites. The MRFSS program was not designed to incorporate this information, but largely focused on biological factors.
The study recommended the implementation of a national trip and expenditure survey, which would support economic valuation studies, impact analyses, and other social and attitudinal studies. The study further recommended that the national database on marine recreational fishing sites should be enhanced to support social and economic analysis. Examples of site characteristics that should be incorporated into the database include: boat ramps, facilities, natural amenities, parking, size, and type.
Need for Better Communication and Outreach
Recreational anglers are the key source of information for the surveys and consequently their cooperation and support is essential to the success of the program. The committee concluded that if anglers understood the purpose of the surveys, the basic methodology, and the value of the data produced, they would be more likely to participate and provide reliable information.
The study recommended improving outreach by advising anglers and managers on the various uses of the data collected. Outreach and communication were identified in the report as areas that should be integral parts of the revised survey program. Lastly, angler associations should be engaged as partners with survey managers, and local knowledge, education, and community activities should be incorporated into the process.
Need for Greater Program Support
The NRC report concluded that a lack of resources had hindered the efforts of the MRFSS program staff to implement, operate, and improve the survey program. This included efforts to improve the program based on recommendations from earlier reviews. Financial resources available to the program were not sufficient to tackle the challenges associated with conducting an efficient and timely survey. Further, NMFS did not have sufficient technical expertise on its staff to handle surveys of such complexity.
In addition to a redesign of MRFSS, the study suggested that provisions be made for ongoing technical evaluation and modification as necessary. The study recognized that additional funding would be necessary to design, implement, and maintain a new program and that this might require a survey office devoted to the management and implementation of marine recreational surveys.
WHAT CHANGES HAVE OCCURRED SINCE 2006
The findings and recommendations of the 2006 NRC report were arrived at by committee consensus and were subjected to NRC’s review process, including external peer review. I will now turn to my observations and opinions regarding changes to NMFS’s recreational survey methods since the release of the NRC report. This reflects only my own experiences and does not represent either committee consensus or NRC review.
In my letter of invitation to this Subcommittee meeting, I was asked for my thoughts on whether the new program has been fully developed and implemented and whether the program is meeting the goals envisioned by Congress. It is my opinion that the revised program is now fully developed in the sense that it is a dynamic system for implementing necessary revisions, creating state-of-the-art design and estimation procedures, and adapting to evolving scientific challenges. The program is transparent, accessible, and subjected to rigorous peer review. This is exactly the sort of statistical program envisioned in the NRC report: there could not be a static, one-time fix to the problems with MRFSS. I now turn to the experiences on which my opinion is based.
In 2007, I was contacted by Dr. Dave van Voorhees of NMFS and asked to assist NMFS in their response to the NRC report, in developing a new Marine Recreational Information Program (MRIP). Since that time, I have acted as a consultant and have advised graduate students who have been supported on NMFS contracts. I have interacted with other consultants, including mathematical statisticians, survey methodologists and information technology specialists. This group includes academics and industry representatives. We have regularly collaborated with staff from NMFS and from state agencies.
The NRC report recommended the establishment of a permanent and independent research group to evaluate recreational fisheries surveys and to guide future innovations. The NRC committee’s goals in making this recommendation included building stakeholder confidence in the statistical system by involving a group from outside the federal agency and by subjecting all work to rigorous peer review. The consultant model adopted by NMFS is entirely consistent with these goals, in my opinion. Consultants in collaboration with NMFS staff are establishing nationally consistent standards for design of marine recreational fisheries surveys, producing detailed sampling designs and data collection protocols, documenting all revised design and estimation procedures, and conducting outreach to stakeholders and to the scientific community.
Improving Statistical Estimation
One of the first problems addressed through MRIP was the mismatch between design and estimation in the intercept survey, potentially affecting the estimates of catch rate and their measures of uncertainty. The effect of the mismatch on the catch estimates was unknown, while the estimates of uncertainty were known to be biased and too low. I worked with other consultants and NMFS staff to revise the weighting procedure used for the intercept data, producing software and technical documentation that was then peer-reviewed by statisticians in industry, in academia, and in the Census Bureau. Revised estimates for 2004-2011 were then computed and released after extensive review. The improved estimation method directly addressed NRC concerns, and is transferable to future onsite surveys, to some auxiliary surveys conducted by NMFS (such as the Large Pelagics Survey), and to some surveys conducted by state agencies.
Sampling Issues with the Intercept Survey
The NRC critique of the intercept survey included in particular the fact that samplers were given too much latitude in the sampling process, including the opportunity to change to alternate sites or alternate modes of sampling. Samplers also focused on the highest-activity part of the day, under the untested assumption that this would be representative of catch rates at other times during the day. To address these concerns, NMFS undertook a pilot study in North Carolina during 2010, in which new field protocols were compared side-by-side to traditional MRFSS intercept survey methods. The new protocols included time-of-day stratification, to ensure some coverage at all times of the day and night. The new design eliminated much of the sampler’s discretion in visiting sites, eliminating a source of variation that was of concern to the NRC committee. Unlike the traditional MRFSS, the survey design and estimation approach tested in North Carolina adhered closely to generally accepted statistical survey methods, while maintaining practical feasibility. Results of the pilot study were peer-reviewed and the final report was released earlier this spring. The new intercept survey protocols are now being implemented on the Atlantic and Gulf coasts.
Sampling Issues with the Telephone Survey
Implementation of the National Saltwater Angler Registry began with federal regulations in 2008. States can be exempted from the registry if they license or register their anglers and provide sufficient contact information for those anglers for use in recreational surveys. Most coastal states qualify for this exemption. Gaps in survey coverage result from exemptions to state licensing requirements and problems with the contact information. MRIP, supported by statisticians and survey methodologists, has been experimenting with dual frame surveys of fishing effort to improve survey coverage. These methods combine angler license frames with household telephone or address frames. Surveys of effort are then conducted with a combination of telephone and mail data collection. Designed experiments are underway to determine the most effective contact options, in terms of getting good response rates and high quality data in a timely manner.
Other Sampling Issues
One MRIP project has tested the use of on-board video cameras to capture data on the species, size, and release condition of recreational discards. This study is ongoing.
MRIP has studied methods for collecting catch and effort data from the recreational for-hire sector, most recently focused on electronic logbook reporting coupled with dockside validation of the logbook data. These studies are ongoing.
Establishing Nationally Consistent Standards
A key feature of MRIP is cooperation between state agencies and the federal system. I have personally observed this cooperation while taking part in the North Carolina Pilot Study, and while conducting reviews of the recreational fisheries survey methods for Oregon in 2010, Washington in 2010, California in 2011, and Hawaii in 2012. Each review included NMFS staff and a team of consultants, and each resulted in a series of recommendations to the state agency on methods to improve their recreational fisheries surveys. The agency, in turn, could apply to MRIP for grant support to address those recommendations. This helps transfer best practices being adopted at the federal level to the states, with appropriate modifications for the unique state-level characteristics of the recreational fishery.
Communication and Outreach
NMFS has embraced the NRC recommendation of better communication and outreach. Many of the MRIP projects have the active participation of the recreational angling community, including fishing club representatives and recreational angling advocates. For example, I serve on a technical working group considering redesign of the Large Pelagics Survey. Two other members of that group are charter boat captains. NMFS staffers responsible for outreach participate in technical meetings and produce press releases and educational videos explaining the revised methods to a general audience. These materials are of high quality, in my opinion. For example, one of these videos, in which my colleague and I described the statistical re-estimation procedures, was awarded a 2013 Gold Screen/Blue Pencil Award of Excellence from the National Association of Government Communicators. Materials related to the revisions, together with data, software, and technical documentation, are now readily available on the MRIP website.
WHAT HAS YET TO BE DONE
One statistical issue in the NRC report that remains to be addressed is small area estimation, in which auxiliary data are used to produce estimates at finer spatial and temporal scales than would be possible using only the weighted survey data. This is an active area of statistical research, with applications throughout the federal statistical system. It is natural that development of such estimators would come after resolving more fundamental design and estimation issues. According to the MRIP website, preliminary work on small area estimation has begun, including developing the necessary database of auxiliary information, and constructing appropriate predictive models.
In my estimation, the MRIP program has directly addressed the concerns noted in the NRC report and is now a complete statistical system with a sound scientific basis. This was not true in 2006. I do not think that all issues are resolved, or ever will be, since the problems in collecting data necessary to manage recreational fishing are continually evolving. But the system in place now is structured to adapt effectively to such changes, by developing, testing, and implementing appropriate tools.
Thank you for inviting me to testify before the Subcommittee today. I am happy to answer any questions you may have.
An archived webcast of the hearing can be found on the House Natural Resources Committee's Web site.