Appendix B

Review of Recreational Fisheries Survey Methods (NRC, 2006) Summary

INTRODUCTION

Recreational fishing in the United States is an important social and economic component of many marine fisheries. However, in some cases, recreational fishing takes more fish than commercial fishing, and in an increasing number of cases, recreational fishing is the main source of fishing mortality. In addition, current assessments indicate that some marine recreational fisheries have exceeded their quotas, raising concern because fishing effort in marine recreational fisheries is projected to increase. It is important that catch monitoring systems are adequate for timely management of these fisheries.

Marine recreational fisheries are not monitored with the same rigor as commercial fisheries. However, as concerns about the effects of all types of fishing have grown, more attention has been paid to the possible impacts of marine recreational fishing. The growing interest in the effects of recreational fishing on fish stock size and composition has led to increased demands for timely and accurate data. Although the National Marine Fisheries Service (NMFS) of the National Oceanic and Atmospheric Administration implemented the Marine Recreational Fisheries Statistics Survey (MRFSS) in 1979 to obtain statistics about marine recreational fisheries, management goals and objectives have changed since then, as has the complexity of the recreational fishing sector. The need for and use of marine recreational fishery statistics in science and management have changed as well. This committee has identified several areas in which designers of sampling programs, data collectors, and users of recreational fisheries data appear to have incomplete communication, mismatched criteria, or other obstacles.

The MRFSS has two major components: an onsite component, in which

anglers are intercepted and interviewed on the water or at sites such as marinas where they access the water, and an offsite component, in which anglers are contacted and surveyed by telephone after their trips are completed. There has been widespread criticism of the nature and use of the MRFSS information. The MRFSS was (and is) intended to be a national program, but not all coastal states participate. In some cases, states have their own surveys of recreational fish landings instead of the MRFSS; in other cases, states have surveys that complement the MRFSS. In addition to this lack of uniformity of coverage, the quality of the MRFSS data for management purposes has also been questioned.

Indeed, it is much more difficult to collect data on recreational saltwater anglers than on commercial fishing operations. There are far more saltwater anglers than commercial fishermen—approximately 14 million anglers fished annually in recent years—and they do not land their catches at specific points where there are dealers, as do commercial fishermen. In addition, there are many modes of fishing (e.g., anglers who fish from head boats or charter boats, with guides,¹ from shore, on private boats, from private property), and many anglers release fish they catch. Some anglers travel far to fish and often fish only a few times each year, which makes them difficult to encounter in surveys. Others, who live within 50 miles of the coast, are much more likely to be intercepted by the MRFSS. Finally, most surveys of anglers depend to some degree on the anglers’ recall and willingness to volunteer valid information. As a result, designing a survey that will provide accurate and timely information, with good coverage and at acceptable cost, is a major challenge.

Despite the complexity of the challenge and its importance for fishery management, the MRFSS staff have been severely handicapped in their efforts to implement, operate, and improve the MRFSS, including implementing the recommendations of earlier reviews. It is not reasonable to expect such a small staff—and one that lacks a Ph.D.-level mathematical statistician—to operate a national survey of such complexity, despite the dedication of the small staff the MRFSS does have.

In addition, the MRFSS is severely limited by the lack of a universal sampling frame for all saltwater anglers, a lack that is not of the MRFSS’s own making. To make matters even more difficult, some of the data that the MRFSS depends on are collected by states, which use a variety of data-collection and sampling protocols. Finally, the financial resources allocated to the MRFSS are modest in comparison to the challenge. This committee’s findings and recommendations should be viewed with this in mind.

___________________

¹ Head boats, also called party boats, take large groups of anglers (sometimes as many as 100) on fishing trips; the groups usually are not preformed. Charter boats (also occasionally called party boats) take smaller groups of anglers, usually four to eight, most often in preformed groups. Guided trips are trips in which a guide takes one or two anglers in a smaller boat. These different categories operate under different U.S. Coast Guard and state license requirements. Throughout this report, these sectors are collectively referred to as the for-hire sector.

THE PRESENT STUDY

To help identify solutions to some of the above problems, NMFS asked the National Academies to assemble a committee to review current marine recreational fishing surveys and to make recommendations for improvements—especially to the MRFSS—and to recommend the implementation of possible alternative approaches. (See Box B.1 for the committee’s statement of task.)

In response, the National Research Council (NRC) of the National Academies established the Committee on the Review of Recreational Fishing Survey Methods, composed of experts in survey design and statistics, biological statistics, fishery management, and the economics and sociology of recreational

fishing. The background and support for the conclusions and recommendations presented below are found in subsequent chapters.

CONCLUSIONS AND RECOMMENDATIONS

General Conclusions

• The committee agrees with conclusions of previous NRC committees that marine recreational fishing is a significant source of fishing mortality for many marine species and that adequate scientific information on the nature of that mortality in time and space is required for successful management of those species.
• Marine fisheries management goals, objectives, and context have changed since the MRFSS was begun in 1979. Management decisions are often made at finer spatial and temporal scales than they were earlier, the mix of recreational and commercial fishing has changed for many areas and species, and stock-assessment models now make greater use of data from recreational fisheries.
• The MRFSS is in need of additional financial resources so that technical and practical expertise can be added to assist in a major overhaul of the design, implementation, and analysis of data from the MRFSS. Both the telephone and access components of the current approach have serious flaws in design or implementation and use inadequate analysis methods that need to be addressed immediately. This committee’s review has focused primarily on the MRFSS, but many of the component surveys of the MRFSS conducted by state agencies (with various degrees of federal funding) suffer from the same shortcomings as does the central MRFSS. As a result, many of this committee’s recommendations apply to state surveys as well as to the MRFSS.
• Many of the independent surveys conducted by the states, as well as state-run surveys that are components of the MRFSS, are different from each other and from the central MRFSS in important ways, including sampling, data collection, and preparation of estimators.
• The committee concludes that users’ concerns about the use of the MRFSS in fishery management are justified by the above-mentioned weaknesses, but they also result from inadequate communication and outreach on the part of the MRFSS managers at NMFS.
• The for-hire sector of marine recreational fisheries (i.e., charter, guide, and head boat operations) is more like a commercial sector than it is like the private-angler sector.

General Recommendations

• The MRFSS (as well as many of its component or companion surveys conducted either indirectly or independently) should be completely redesigned to improve its effectiveness and appropriateness of sampling and estimation procedures, its applicability to various kinds of management decisions, and its usefulness for social and economic analyses. After the revision is complete, provision should be made for ongoing technical evaluation and modification, as needed, to meet emerging management needs. To improve the MRFSS, the committee further recommends that the existing MRFSS program be given a firm deadline linked to sufficient program funding for implementation of this report’s recommendations.
• A much greater degree of standardization among state surveys, and between state surveys and the central MRFSS, should be achieved. This will require a much greater degree of cooperation and coordination among the managers of the various surveys. The for-hire sector of marine recreational fisheries should be considered a commercial sector, and survey methods and reporting requirements for that sector therefore should be different from those for private anglers.

Sampling Issues Conclusions

• The committee concludes that the current methods used in the MRFSS for sampling the universe of anglers and for determining their catch and effort are inadequate. Sampling of each group of anglers (i.e., private, guided, head boat, charter boat) presents challenges that can differ across the groups. Two complementary methods of sampling are used in the MRFSS. One is onsite (i.e., intercepting anglers while they are fishing or at their access [landing] points). The other is offsite, which includes a variety of sampling techniques for contacting anglers after they have completed their trips. Both onsite and offsite methods suffer from weaknesses that may lead to biases in catch and effort estimation. Finally, the estimation procedure for information gathered onsite does not use the nominal or actual selection probabilities of the sample design and therefore has the potential to produce biased estimates for both the parameters of interest and their variances.
• Onsite methods fail to intercept anglers who have private access to fishing waters or intercept them only sporadically. It is impossible, using current methods, to obtain information on the target species of anglers who have private access. In addition, various physical, financial, and operational constraints often lead to spatial or temporal biases in onsite sampling coverage that are not adequately accounted for in the estimation equations.

• Offsite sampling methods that rely on telephone interviews are complicated by the increasing use of cellular telephones, especially in surveys of residents of coastal counties. This is because cellular telephones are not restricted to a geographic region as are landline telephones. If cellular telephones are excluded, then undercoverage of the survey will be increasingly problematic over time as the number of people who use only cellular telephones is growing. The existing random-digit-dialing (RDD) survey suffers in efficiency from the low proportion of fishing households among the general population and may allow bias in estimation from its restriction to coastal counties only.
• The existing RDD survey suffers in efficiency from the low proportion of fishing households among the general population and may allow bias in estimation from its restriction to coastal counties only.
• Reliance on fishing license–based lists of saltwater anglers is not yet feasible as a means of improving offsite sampling methods to avoid the inefficiency of RDD, undercoverage due to cellular telephone use, and restriction to coastal counties. Although many states collect angler information when a saltwater fishing license is purchased, there are license exemptions based on age, residence, access points, existence of a boat license, mode of fishing, and other factors. As a result, angler information for those states is incomplete. Some states have more complete information than others, and in the states that have no saltwater license, there is no list of saltwater anglers. The lack of a universal sampling frame (registry or license requirement) for all saltwater anglers is a major impediment to the development of a reliable and accurate survey program.
• Catch and release fishing (release of fish that survive capture) is increasingly common in many marine recreational fisheries. Although some fish survive capture and release, mortality may be high, in some cases exceeding 50 percent. The survey fails to provide a valid and reliable method of adequately accounting for fish caught and not brought to the dock (including fish released alive or dead, as well as fish caught for bait or given away before reaching the dock). This shortcoming affects estimates of catch and total removals.
• The correct identification of fish species, especially in places with diverse fish faunas, is a difficult challenge, both for many anglers and for those conducting surveys. Incorrect identification obviously has the potential to lead to incorrect conclusions from survey data.

Sampling Issues Recommendations

• A comprehensive, universal sampling frame with national coverage should be established. The most effective way to achieve this is through a national

• registration of all saltwater anglers or through new or existing state saltwater license programs that would allow no exemptions² and that would provide appropriate contact information from anglers fishing in all marine waters, both state and federal. Any gaps in such a program (e.g., a lack of registration in a particular region or mode, exemptions of various classes of anglers) would compromise the use of the sampling frame and, hence, the quality of the survey program. An updated, complete registration list would greatly improve sampling efficiency in terms of time and cost. Although these savings might not cover the entire cost of maintaining such a database, the benefit from the increased quantity and quality of the data would be worth the extra cost, especially if there is an associated increase in public confidence in the final estimates.

• Future telephone surveys should be based on the above universal sampling frame.
• Charter boat, head boat, and other for-hire recreational fishing operations should be required to maintain logbooks of fish landed and kept, as well as fish caught and released. Providing the information should be mandatory for continued operation in this sector, and all the information should be verifiable and made available to the survey program in a timely manner.
• Additional studies are needed to understand the extent to which fish are kept and inspected, as well as the extent of catch not available for inspection to improve the accuracy of catch estimates.
• Panel surveys, which contact individual anglers repeatedly over time, should be considered in recreational fishing surveys to gather angler trend data and to improve the efficiency of data collection.
• The onsite sampling frame for the MRFSS should be redesigned. The estimation procedure critically depends on the assumption that catch rate does not vary according to the nature of the access point. In particular, small or private access points that most likely are missed might have different catch rates than larger access points, which would lead to bias in the resulting estimators. In addition, the sampling process requires greater quality control (less latitude on the part of the samplers) than it has at present. (See the recommendation below for the establishment of an independent research group to investigate matters such as these.)
• Dual-frame procedures should be used wherever possible to reduce sample bias. For example, if a state has an incomplete list frame based on licenses, the use of an additional sampling frame of the state’s resi-

___________________

² There is no scientific reason that a state should not continue to allow certain groups (e.g., seniors) to fish for free, as long as everyone is required to register in the universal sampling frame or have a state saltwater license.

• dents (e.g., RDD) would reduce the bias. The existence of a universal frame described above would make this approach unnecessary for offsite sampling.

• Internet surveys should be considered for their potential use in recreational fishing surveys, especially in panel surveys, as a way for anglers to submit information.

Statistical Estimation Issues Conclusions

• The designs, sampling strategies, and collection methods of recreational fishing surveys do not provide adequate data for management and policy decisions. Unknown biases in the estimators from these surveys arise from reliance on unverified assumptions. Unless these assumptions are tested and the degree and direction of bias reliably estimated, the extent to which the biases affect final estimates will remain unknown.
• The statistical properties associated with data collected through different survey techniques differ and often are unknown. The current estimators of error associated with various survey products are likely to be biased and too low. It is necessary, at a minimum, to determine how those differences affect survey results that use differing methods.

Current analysis procedures used in the MRFSS do not exploit the current knowledge of finite population sampling theory. The current estimates are particularly deficient when applied to small areas because they do not use information in adjoining areas or time periods, nor do they consider relationships between species that occur together. Therefore, they are of lower precision than would be possible if this information were used. Improvements in these estimates would be of great use to managers who need to make quick decisions concerning spatial areas that are smaller than typical in the early years of the MRFSS.

Statistical Estimation Issues Recommendations

• The statistical properties of various sampling, data-collection, and data-analysis methods should be determined. Assumptions should be examined and verified so that biases can be properly evaluated.
• A research group of statisticians should design new analyses based on current developments in sampling theory. These examinations should include experimentation, such as specific sampling of activities like nighttime fishing or fishing from private property, whose current under-representation in the MRFSS sampling has the potential to create bias.

Human Dimensions Conclusions

• The MRFSS was not designed with human dimensions data (i.e., collection of social, behavioral, attitudinal, and economic data) in mind. The qualities of social, economic, and other human dimensions data have been compromised for many of the same reasons that the biological data have been compromised, including such issues as those related to coastal populations, telephone surveys, and sampling protocol. The human dimensions data have been further compromised by simply being added onto the biological data-collection efforts that have different sampling requirements and survey design needs. Current surveys are largely focused on biological factors (e.g., numbers, sizes, and species of fish landed) and not on human dimensions factors. The statistical and sampling problems associated with social, behavioral, attitudinal, and economic data often can be considerably different from those associated with biological factors.

If the number of marine fishing trips increases, it is likely that additional fishing access sites will be developed. In addition, social and environmental changes (e.g., changes in the distribution and numbers of people, a major hurricane) also can affect the availability and use of access sites. To ensure adequate coverage of the recreational fishery, a periodic updating of lists and descriptions of fishing locations and access sites is needed.

Human Dimensions Recommendations

• An independent national trip and expenditure survey should be developed to support economic valuation studies, impact analyses, and other social and attitudinal studies. The sampling and survey procedures of the independent survey should be designed for the purpose of social and economic, not biological, analyses.
• Add-on surveys for human dimensions should be continued but in a more focused way than currently is done to target specific management needs and to supplement the national data as needed.
• The national database on marine recreational fishing sites and their characteristics should be enhanced to support social, economic, and other human dimensions analyses. Sites should be defined at levels as fine as possible. The data set should include site characteristics that matter to anglers in making fishing choices, such as boat ramps, facilities, natural amenities, parking, size, and type (e.g., beach, pier, launch point). To account for changes in the number and patterns of trips and the changing characteristics of sites, a periodic updating of the data should be conducted.

Program Management and Support Conclusions

• A large number of complex technical issues associated with surveys of marine recreational fishing remain unsolved, and a significant investment in intellectual and technical expertise is needed.
• A greater degree of coordination between federal, state, and other survey programs is necessary to achieve the national perspective on marine recreational fisheries that is needed.
• The recommended changes to the design and operation of the MRFSS and its continued development and operation will require additional funding above current levels.

Program Management and Support Recommendations

• A permanent and independent research group should be established and funded to continuously evaluate the statistical design and adequacy of recreational fishery surveys and to guide necessary modifications or new initiatives. Human dimensions expertise should be included as well.
• Additional funding is needed for a survey office devoted to the management and implementation of marine recreational surveys, including coordination between surveys conducted in various state and federal agencies.

Communication and Outreach Conclusions

• It is difficult for individual anglers to see the effects of recreational fishing on their target species and to distinguish daily and seasonal fluctuations from trends. As a result, no matter how well designed and implemented a marine recreational survey is, it will not fully succeed without the cooperation of anglers. Unless anglers believe that the survey is well designed and implemented and that it is being used intelligently to address appropriate management issues, they are unlikely to participate.
• In particular, anglers need to have a basic understanding of the relationship between a statistically based sampling scheme and the frequency with which each of them is (or is not) contacted by a data collector.
• If anglers believe that their input is influencing the design and use of surveys, they are more likely to be satisfied with those surveys than otherwise.
• If anglers understand the basic purposes and decisions to which recreational fishing survey data are being applied and how those data are interpreted and used, they are more likely to feel confident that the ap

Communication and Outreach Recommendations

• Outreach and communication should be improved in several ways. The MRFSS managers should advise anglers and data users on the constraints that apply to the use of the data for various purposes. Managers and anglers also should be informed clearly about any limitations of the data.
• Outreach and communication should be institutionalized as part of an ongoing MRFSS program so their importance is acknowledged and appropriate expertise can be developed.
• Angler associations should be engaged as partners with survey managers through workshops, data collection, survey design, and participation in survey advisory groups. Many NRC and other reports stress the importance of using local and traditional knowledge, capacity building, and local communities in knowledge-gathering and dissemination activities. These recommendations apply, as well, to the recreational fishing community.

This page intentionally left blank.