Conquering the ARC-PA Pass Rate Report Volume 1: When ARC-PA requires PANCE performance analysis

We all know that meeting ARC-PA standards is required for PA programs to retain accreditation. According to ARC-PA.org, in any year that the PANCE pass rate for first-time takers, by cohort for that year, has a pass rate percentage of 85% or less, “the PA program must submit an analysis of PANCE performance to ARC-PA within six months of providing this data within the Program Management Portal, or by July 1 of the following year, whichever is sooner.” You may download the PANCE Required Report for Low First-Time Taker Pass Rates from their website.

To put it simply, if less than 85% of your students can pass the PANCE on their first try, ARC-PA wants to know why. Moreover, they want you to demonstrate that you (and your program) know why it happened, and what modifications you can make to keep it from happening again.

Readers know by now my fondness for statistics, and no statistics about your PA program exist in a vacuum. Your first-time PANCE pass rate signifies a past, present, and future situation: 

• Your program imparts the knowledge required to become a PA.
• You have used past information to adjust your program accordingly so that your pass rates stay high.
• Students in your program can trust that their time and money is being used effectively to meet their goal of becoming a PA.
• Your program prepares students for the rigors of this qualifying test.
• Your program is dedicated to providing the best possible education for students yet to come.

Goals of this Series

A lower pass rate indicates that something in your curriculum, instruction, or preparation of students may be lacking. And you, as the program director, should be equally interested. Quality applicants come to schools that can provide the education they want. Your program’s name should be a badge of honor on a graduate’s CV, as well as a desirable school to prospective students. A high first-time PANCE pass rate is a major component of that status.

Beginning this new series in the Massey & Martin, LLC newsletter is exciting, because it relates to so much of what we have already discussed: statistics, ARC-PA requirements, and reporting techniques, and the Student Success Model. We will spend the next several issues breaking down the subject of risk management in PA education, which will inform in two ways:

• how to respond to the ARC-PA’s report for analysis; and
• how to track risk predictors for your PA students, so that your program can intervene on behalf of students at risk of failing before the PANCE exam is taken.

Reporting to the ARC-PA

When you submit data to ARC-PA regarding PANCE outcomes, it should include the following details as these details correlate to PANCE outcomes:

• Admissions criteria as predictors of success
• Individual course performance
• Course and instructor evaluations
• Program instructional objectives, learning outcomes, and breadth and depth of the curriculum
• Student summative evaluation results
• Remediation practices and results
• Attrition criteria and data for the cohort being reviewed
• Feedback from students who were unsuccessful on PANCE, if available
• Preceptor and graduate feedback 
• Employer feedback (not required, but may be helpful if available to the program)

Be aware that the ARC-PA Commission reviews these reports as evidence of an intact assessment system. The inability to provide an acceptable report involving all elements required will result in the report not being accepted. If your program does not deliver an acceptable report on the second attempt, it could be put on probation. The commission will place programs on probation solely for that reason.

Most of the conclusions and correlations can be completed through descriptive analysis such as comparisons, stratifications, and case analysis. The focus of the report needs to be on graduates who failed PANCE vs those that passed PANCE.

The program is required, at a minimum, to analyze the cohort data for the graduating class that had a pass rate of < 85%. I find that three years of data provides more insight for trend analysis. Though ARC-PA does not make this requirement, I also believe that some aspects of the analysis need a “deeper look” through parametric such as Pearson Correlation, Logistic regression, Step-Wise regression, Phase 1 and Phase 2 stepwise regression, and individual variable regression.

So, let’s get started…

In the next issue of our newsletter, we’ll begin at the top of the list with “Admissions criteria as predictors of success,” and examine each of the above factors for ARC-PA consideration. We’ll discuss how to collect, analyze and present this information, and what modifications can be made based on statistically significant correlations.