2020 Breakout Session - Siegel et al.

Title of Presentation: A Data Driven Approach Psychology Internship Selection; what happened to concepts of reliability and validly?

Presenter Information:

Wayne Siegel, Ph.D., ABPP - Minneapolis VA Health Care System

Chris Erbes, Ph.D., ABPP - Minneapolis VA Health Care System

Amanda Ferrier-Auerbach, Ph.D., ABPP - Minneapolis VA Health Care System

Abstract:

Historically, the psychology internship selection has not been a very data-driven process. The internship selection process is also an exceptionally time-consuming endeavor (Lopez Oehlert, and Moberly 1996). Training directors and program supervisors typically invest numerous hours in the file review, interview and ranking processes. Compounding these challenges, the AAPI (APPIC Application for Psychology Internship) variables used in this process have low reliability and limited evidence supporting their validity. Despite APPIC’s efforts to operationalize these variables, there is great variability among doctoral programs in how these numbers are defined and accrued, and some doctoral programs provide significant oversight to their students’ AAPIs while others do not (Rodolfa, Vieille, Russell, Nijjer, Nguyen, Mendoza, and Perrin, L, 1999; Ross, Davis, and Michael, 2010). In 1999, APPIC developed the AAPI (APPIC Application for Psychology Internship), a standardized application used by all member internship programs. The AAPI evolved to an online version in 2006 and applicant data could be downloaded and potentially used by internship programs. Decisions on which variables to include were made mostly by consensus of APPIC member programs and the APPIC Board, rather that which variables actually predicted success on internship. It should be acknowledged that APPIC’s focus at the time was to get buy-in from internship programs in order to move forward with a standard application. Regardless, data derived from the AAPI allows for some potential solutions to the problems noted in the previous paragraph. The Minneapolis VA Psychology Training Program has been implementing a systematic internship selection procedure that is designed to maximize the use of available AAPI data while minimizing the bias inherent in our prior more subjective selection processes. Objective weightings in a number of categories are derived from the APPI using set scoring rules, and then merged into composite ratings based on program defined domain weightings. Current domains include Assessment, Intervention, Diversity, Match, and Scholarly Potential. An additional Neuropsychology domain is added for our Neuropsychology Track. The proposed oral presentation will provide a brief historical context of psychology internship selection, how our data-informed (early years) and data-driven (recent years) processes have evolved, the struggles along the way, and the advantages and disadvantages of these approaches. We will review our current systematic process and compare it to prior processes in terms of efficiency (staff time), and perceptions of validity and utility. Lopez, S., Oehlert, M., and Moberly, R. (1996) Selection Criteria for American Psychological Association- Accredited Internship Programs: A Survey of Training Directors. Professional Psychology: Research and Practice, Vol. 27, No. 5, 518-52 Rodolfa, E., Vieille, R., Russell, P., Nijjer, S., Nguyen, D., Mendoza, M, and Perrin, L (1999). Internship Selection: Inclusion and Exclusion Criteria. Professional Psychology: Research and Practice. Vol.30, No. 4, 415-419 Ross, W., Davis, S., and Michael, P. (2010) An Examination of Inclusion and Exclusion Criteria in the Predoctoral Internship Selection Process. Training and Education in Professional Psychology. Vol. 4, No. 4, 213–218