The National Academies of Sciences, Engineering, and Medicine will form an ad hoc committee to review a report authored by Abt Associates, who served as contractors to the Social Security Administration. The report, "Synthesizing Information about Vocational Preparation Requirements, Occupational Tasks, and Required Functional Abilities in the Standard Occupational Classification (SOC) System" provides guidance to SSA on the mental health requirements for specific occupations.
SSA requests an evaluation of its contractor's methodology and selection of data used in accordance with the specific questions below, and to provide responsive findings and conclusions to SSA. The scope of the review is limited to the following questions:
1. Suitability of the database(s) utilized
Abt used the Department of Labor’s Occupational Information Network (O*NET) database to inform its ratings on social interactive and adaptive functional capacities.
• Was this an appropriate data source to use to identify occupations’ core tasks?
• Was this an appropriate data source to use to inform the ratings?
• Were there other data sources that would be better to inform the ratings?
2. Soundness of the methodology to connect occupational tasks to social interactive and adaptive functional capacities
Abt used the judgement of a panel of experts that they assembled for this purpose and statistical analysis of O*NET data to connect tasks to specific vocational preparation (SVP) levels, social interaction, and adaptability.
• Did Abt provide the expert work group with sufficient guidance and information to complete its task?
• Did the expert work group use the O*NET data constructs appropriately and in a manner consistent with the limitations of the data?
• Was Abt’s process for obtaining consensus ratings appropriate?
• Was Abt’s meeting process effective to provide ratings?
• Considering the instructions and guidance provided by SSA, did Abt’s methodology provide a sufficient nexus to their findings?
3. Appropriateness of the expertise gathered to perform the analyses
The Abt expert work group deliberated upon the mental requirements of occupations.
• Did the composition of the expert work group include appropriate expertise?
4. Overall Confidence
• Were there any likely sources of bias or systematic error resulting from the data or methodology?
• Was it clear how Abt and the expert work group reached its conclusions?
• Did Abt present its methodologies clearly?
In accordance with OMB guidelines, reviewers shall limit their advice to an evaluation of the soundness of the methodologies Abt used to select appropriate O*NET data and to derive its ratings. The reviewers shall also evaluate whether the report identifies and characterizes any pertinent uncertainties in its ratings.
Reviewers shall not replicate or evaluate individual ratings.