A CRITIQUE OF THE CALIFORNIA DEPARTMENT OF FOOD AND AGRICULTURE PIERCE’S DISEASE/GLASSY-WINGED SHARPSHOOTER BOARD’S REQUEST FOR PROPOSALS

The focus of this critique is the RFP that was issued by the CDFA PD/GWSS Board in December 2022 (a copy is provided in Appendix D).² Among the proposals submitted in response to this RFP were 16 proposals that mostly address grapevine virus diseases and their vectors, which the Board asked the committee to evaluate using the evaluation criteria in the RFP.³

The committee concluded that overall, the RFP would need to be revised to be more transparent about the research areas that the Board is supportive of. The RFP should also have more explicit instructions on the content in each of the sections of the proposal and emphasize what information must be provided by the applicants to allow proposal reviewers to make a full assessment of the merits of the proposed projects. In this regard, the Board may want to consider adding language to the RFP indicating that proposals will be ineligible for review and funding if the required information is not submitted. Alternatively, the Board may want to consider providing a standard application form, as other RFP-issuing entities have done, instead of allowing applicants to submit their own Word documents (which in some cases did not necessarily follow the format outlined in the RFP or provide the information required in the RFP). Requiring the use of a standard application form would help with ensuring completeness and consistency across proposals, which would greatly benefit the proposal evaluation process.

Comments and recommendations for improving specific sections of the RFP are provided below. Note: Italicized text is from the RFP.

RFP FOCUS AREAS

The California Department of Food and Agriculture’s Pierce’s Disease and Glassy-winged Sharpshooter Board (CDFA PD/GWSS Board) is currently accepting proposals for research and outreach projects on the following serious pests and diseases of winegrapes:

• Pierce’s disease and its vectors, especially the glassy-winged sharpshooter
• Brown marmorated stink bug
• Grapevine fanleaf disease
• Grapevine leafroll disease
• Grapevine red blotch disease
• Mealybug pests of winegrapes
• Spotted lanternfly

The committee noted that one of the 16 proposals submitted to the Board was a continuation of a project (previously funded by CDFA) that aims to use a gene editing approach to create plants with resistance to a grapevine disease that is not in the RFP focus areas.⁴ If the

___________________

² The RFP is also available at https://www.cdfa.ca.gov/pdcp/grants/docs/cdfa_pd-gwss_rfp_dec_2022.pdf (accessed June 14, 2023).

³ See Activity 1 in the Statement of Task (Appendix A).

⁴ After the release of the prepublication copy of the report to the sponsor, the sponsor representatives clarified that while not stated in the RFP Focus Areas section, the funding scope of the PD/GWSS Board

Board is open to funding research that does not directly address any of the RFP focus areas, then it would be prudent to revise the RFP to include “other important non-viral diseases.” The Board may also want to consider periodically updating the RFP focus areas, for example, to include any emerging or re-emerging diseases and pests of winegrapes or to address particular knowledge gaps in these focus areas as they are identified. The Board should also make it clear in the next RFP and to the proposal reviewers/review panel whether proposals would necessarily have to address the focus areas explicitly outlined in the RFP in order to be eligible for funding.

In this section of the RFP, it is not clear what “research and outreach projects” is referring to—can applicants submit research projects, outreach projects, or research projects with an outreach component? The Board may want to revise the RFP text to clarify this.

The committee observed that, in the proposals they reviewed, those that had an outreach component varied in the amount of details provided. If outreach is important to the Board, then the RFP should provide guidance on what information about outreach should be included in the proposal. This would help reviewers to fairly assess the proposed outreach activities.

GENERAL INFORMATION

• Funding preference will be given to projects deemed likely to lead to practical solutions to winegrape pest and disease problems in California.
• Multi-disciplinary team projects are encouraged.

Aside from encouraging multidisciplinary research, the RFP could also emphasize that the research teams should have the appropriate expertise needed to carry out all the proposed work and achieve the project objectives. For some projects including vector-borne disease complexes, having a multidisciplinary team may result in greater understanding and impact for stakeholders.

ELIGIBILITY

• Any individual or group with expertise in winegrape pest and disease research and/or outreach and has affiliation with a non-profit institution is eligible and encouraged to submit proposals.

For clarity, this section could be revised by adding a second bullet that states that “industry members and organizations can be collaborators/co-PIs (and are encouraged to be part of proposals as appropriate).”

REVIEW PROCESS AND CRITERIA

Proposals will be reviewed by review panels and/or individual reviewers. In addition, the PD/GWSS Board’s Research Screening Committee will review and make recommendations to the PD/GWSS Board on the funding of proposals. The PD/GWSS Board will then make funding recommendations to the Secretary of CDFA.

___________________

includes method development research that could eventually be used to address Pierce’s disease or PD/GWSS Board designated pests and diseases that are in the RFP focus areas.

Proposals will be reviewed and evaluated in the following areas (100 points possible):

• Objectives and Relevance – Are the objectives clearly stated, justified, worthwhile, and reasonable? Are the proposed objectives likely to contribute significantly to solving the problem? Does the proposed project unproductively overlap with other projects? (20 points)
• Activities to Accomplish Objectives – Is the work plan reasonable, feasible, and capable of meeting the stated goals and objectives? For research, is the work plan of good scientific merit? (40 points)
• Project Leader, Co-Project Leaders, and Others – Do they have appropriate backgrounds, expertise, experience, and capabilities for the proposed tasks? Is the team missing any critical capabilities? (10 points)
• Resources and Likelihood of Accomplishing Objectives – Assuming the requested funds are awarded, will the Project Leader and associates have the resources, including facilities, to achieve the objectives? (10 points)
• Project Timetable – Are the milestones appropriate? Are they achievable? (10 points)
• Budget – Is the budget reasonable and appropriate? (10 points)

The committee was asked to evaluate 16 proposals using the criteria above. The committee was also asked, in addition to providing scores, to write explanatory comments that would be sent to principal investigators (PIs) to provide feedback on their proposals. During the proposal review, the committee noted overlaps in some of the proposals and in some cases, this was pointed out in the reviewer’s comments. However, no further instructions were given to the committee about addressing overlaps. In this regard, the Board may want to consider requesting future proposal reviewers/review panels to identify overlaps between proposals and recommend collaborations to streamline efforts in certain areas of research to accelerate the development of practical solutions.

In the section above, the committee suggests adding these two questions to “Objectives and Relevance:” 1) “Is there a clear explanation of how the findings would lead to practical applications in California winegrape production?” and 2) “Is there sufficient description of further steps beyond the current project that may be necessary to achieve field application?”

PROPOSAL FORMAT AND GUIDELINES

To ensure that all required information is submitted without exceeding the maximum page number, the Board could provide a standard form for all applicants, instead of allowing them to upload their own Word documents (which in some cases do not follow the format specified in the RFP). The RFP could also indicate that proposals that are missing the required information (budget, timeline, information about previously funded project, etc.) would be ineligible for review or funding.

Project Leader

Indicate the Project Leader, i.e., the person responsible for overall project management, coordination, and execution. Include institutional affiliation, address, phone number, and e-mail address.

Co-Project Leaders

Please include institutional affiliations, addresses, phone numbers, and e-mail addresses. Indicate the roles of each Co-Project Leader and make sure that each Co- Project Leader is aware of their proposed participation.

Cooperators

Please indicate the roles of each Cooperator, and make sure they are aware of their proposed participation.

To encourage clarity and facilitate the review process, the Board may want to consider asking the applicants to provide a table that contains the PI(s), co-PI(s), and Cooperator(s) names, affiliation, contact information and a brief description of their roles and expertise. For proposed projects with field studies, information about who will maintain the field site may also be provided in this table.

PROJECT HISTORY

Please indicate if this is a new or continuing project. If a continuing project, indicate when it began, the number of years of activity, and the sources of funding. Also, indicate how this project relates to other past, current, and anticipated future projects. Summarize previous work in this area.

Clarification about Progress Reports: Progress reports should not be included as part of your submission. Instead, use the sections titled “Project History” and “Summary” to briefly discuss any previous work on your project that is relevant to the present proposal.

For some of the proposals reviewed by the committee, it was not immediately clear if the proposed project is new or a continuation of a previously funded project.⁵

__________________

⁵After the release of the prepublication copy of the report to the sponsor, the sponsor representatives pointed out that a form where PIs are asked to indicate whether a project is new or continuing was also submitted by the PIs via the CDFA website, but this form was not included in the project proposal documents that the committee reviewed.

The RFP provides guidance on what information to provide for continuing projects; however, only a minority of the proposals included information on previous years of funding. In many of the proposals, the section “Project History” was used for an extended literature review or prior accomplishments review, which could be an indication of prior financial support to the applicant from CDFA.

This section does not contain guidance on what information to provide for new projects, which leaves the applicant to decide what to include.

During the proposal evaluation, the committee did not have access to the progress reports from applicants who are proposing to work on projects previously funded by CDFA or other organizations; hence, the committee members could only rely on the information provided in the “Project History” section. The proposal review was hampered when the applicant did not provide sufficient information on previous work and its relevance to the project being proposed.

For ease of review and to allow reviewers to conduct a full and fair assessment of proposals, the Board may want to consider the following recommendations:

• Require applicants to clearly indicate right after the project title whether the proposed project is new or continuing.
• In addition to the “Layperson Summary,” require applicants to provide an “Executive Summary” (400 words maximum) that explains the relevance of the proposed project, the main objectives, and the expected outcomes. This summary should also explain, in non-technical terms, the methodology, anticipated results, and how and when these results can be of benefit to growers and the winegrape industry.
• Require applicants proposing continuing projects to provide information about when the project began, the number of years of activity, and the sources of previous funding, including an explicit disclosure of whether previous funding was received from CDFA, as part of the “Project History” section. Applicants proposing new projects need not submit project history information; the “Executive Summary” should be sufficient for these applications.

OBJECTIVES OF PROPOSED PROJECT AND PATH TO APPLICATION

Please state the aim or broad goal of the proposal, followed by a numbered list of specific objectives. After the specific objectives, provide a summary of the potential impact and relevance of the proposed project. For research projects, describe how the project’s findings will lead to practical applications in California winegrape production and describe the steps that must be taken to achieve field application. Provide an estimate of the time frame involved. Describe how the overall project and each objective address the fundamental goal of protecting winegrape production in California. Cite relevant literature.

This section of the RFP asks the applicant to provide information about the project’s goals, the potential impacts and relevance of the project, and a timeframe for the application of results to the management of a disease or pest. The committee noted that in some of the proposals, information about the path to application was buried in the discussion of the objectives. For the Board and the reviewers to understand the science, have clarity on all of the steps involved in the proposed work, and understand how and when the project results can help

protect winegrape production in California, the committee suggests removing the “Objectives of the Proposed Project and Path to Application” section and splitting it into these sections:

• Background and Justification for the Proposed Project: In this section the applicant can cite relevant literature or work relevant to the proposed project and discuss the project rationale.
• Main Goals and Specific Objectives: In this section the applicant can state the main goal and provide a numbered list of specific objectives.
• Path to Application: In this section the applicant can indicate how the project can contribute to a practical solution, provide a timeline or estimate of when the tangible results or applicable solutions can be expected, and outline the solution’s anticipated long-term feasibility and sustainability.
• Deliverables: In this section the applicant should list the deliverables from the proposed project which could be turned over to growers for managing the target pathogens and pests or to other researchers for further study to generate applicable results in the future.

The “Objectives and Path to Application” section gives good guidance describing the path to application. The wording is stronger and clearer than what is in the criteria section for the objectives. The committee suggests adopting the language from the “Objectives and Path to Application” section of the RFP to improve the evaluation criteria provided to reviewers.

METHODOLOGY TO ACCOMPLISH OBJECTIVES

Discuss the methodology for each objective. For research, discuss laboratory experiment or plot design, expected results, statistical analyses, methods to be used, parameters of data collection including sampling methods, and potential pitfalls and limitations. For research that has a field component, discuss site selection and how the field component will help accomplish the stated objectives. Cite relevant literature.

The committee observed that in many of the proposals, the challenges and pitfalls of the proposed project were not discussed. The committee suggests revising the second sentence in the “Methodology to Accomplish Objectives” section to read: “For research, discuss laboratory experiment or plot design, expected results, statistical analyses, methods to be used, parameters of data collection including sampling methods, along with challenges, potential pitfalls, and limitations and plans to overcome them.”

RESOURCES AND LIKELIHOOD OF ACCOMPLISHING OBJECTIVES

Please summarize how the resources of the Project Leader and associates (i.e., dedicated financial sources, computer facilities, laboratory and field resources, and human resources) and previous work make the proposed work feasible and increase the likelihood for accomplishing the stated objectives. For field studies, Project Leaders conducting field research should indicate who will maintain the field site and include a statement describing how the site will be maintained that demonstrates awareness of good farming practices.

The last statement in this section regarding “good farming practices” did not seem to be addressed in some of the proposals. If awareness of good farming practices is important to the Board, then information about this should be included in the evaluation criteria section to encourage applicants to address this requirement. The Board may also want to consider providing an explanation of what constitutes “good farming practices” to be clear about what information applicants should provide.