In my last blog, I discussed the use of stepwise regression, in which possible predictor variables are entered into a regression model until the best statistically significant variables are determined. I want to resume this discussion today by looking at some real-life examples of predictor variables and how they were determined.
For a 2020 cohort, EORE and the number of remediations were the statistically significant (p < 0.01) predictors remaining in the model. A combination of these variables was the best predictor of PANCE scores. With enough data, you can begin to identify a Multiple Variable Risk Model. The multiple correlation coefficient (correlation between PANCE scores predicted by the model and actual PANCE scores) for this final model was high (R =0.76). The variables (EORE and number of remediations) explained 58.3% of the variance in PANCE scores
For a 2020 cohort, the EORE, History and Physical I, Clinical Medicine I, Clinical Problem...
Little is more important to a PA program than the PANCE pass rates of its graduates. A program that can boast high first-time PANCE pass rates will draw more students. Therefore, it is most desirable to discover variables that can predict passing (or failing) PANCE scores as far ahead of time as possible. With this benefit, students who need assistance can get it, and you may bolster portions of your program to improve overall PANCE scores.
Correlation and regression are ways to measure variables such as these, as predictors of PANCE scores:
The most common way I see regression performed is through Parametric Analysis to Enhance Assessment Regression.
In this case, we look at each of these specific elements as a predictor of PANCE...
In the past few blogs, we have examined various statistical concepts, so that we can derive interpretations from them. Previously, I discussed statistical significance, used as a pre-designated point at which we can say that the appearance of a correlation is probably not due to random chance.
The point of looking at correlation coefficients and statistical significance, for the PA program, is eventually to determine what courses, exams or other indicators correlate with passing PANCE scores. We use correlation and statistical significance to find, primarily, which variables can be used to predict PANCE performance.
This chart shows how R values and p values change as we correlate certain variables to PANCE scores.
In my previous blog, we discussed the meaning of correlation and how we can use this measurement to determine if variables move positively or negatively in relation to one another. Today we will expand this discussion by explaining the importance of statistical significance when we determine how strongly two variables correlate.
Descriptive statistics provide a one-dimensional perspective. They can generate correlative relationships, but they cannot determine statistical significance. I think that, for those people who are research-oriented, whenever possible, statistical significance is important, and with educational outcomes, it is certainly appropriate. So, what does it mean to say that two variables correlate with statistical significance? Let us examine this question.
The knowledge of the relationship between two variables is useful in predicting one from the other, especially if one variable is observed in advance of the other....
Thank you for joining me as I continue my blog series on utilizing advanced assessment methods for the vast amounts of data collected by your PA program. We have spent several blogs exploring the benefits of this methodology. Now we will break down the numbers so you can understand what it all means. What do applied statistics tell us in this context about correlations? What constitutes statistical significance?
Before you race for the door, let me ease your mind. This will not be a statistics course. Statistics may seem so complex that they have no practical application, but that is far from the truth. With just a few definitions and an understanding of how variables move in relation to one another, you can reap the benefits of advanced analysis.
This is a topic near and dear to my heart. I have taught research methodology and stats to graduate health science students for more than fifteen years. I have also been intricately involved with assessment in PA education for many...
Thank you for joining me through this series on responding to the requirements of ARC-PA’s 5th Edition Standards Appendix 14. Before we move on to our next subject, I offer some final insights that I hope will help you in creating a well-rounded and thoroughly supported SSR for your PA program.
The SSR is an interconnected document. Many of the appendices organically connect with each other. In responding to one appendix, you may always incorporate relevant data from other appendices. 14C (effectiveness of a program’s didactic curriculum) connects very well with 14F (presentation of PANCE outcomes).
Is it necessary to include all the same information in Appendix 14F that you have already included in 14C? My recommendation is to include an excerpt from 14C that is appropriate within the answer to 14F.
Be certain that you answer each and all of the questions asked in each appendix template and be cautious about simply...
ARC-PA’s 5th Edition Standards require that your program’s goals become part of the data you collect and review. Appendix 14H is about your success in meeting program goals. Its instructions are:
Appendix 14G looks at the sufficiency and effectiveness of your PA program’s principal and instructional faculty and staff. However, here’s a “warning” note. You still must develop, and explain, your own methodology about how you determine sufficiency. Therefore, the element about the complexity of the program itself must be addressed.
Appendix 14G expects the following elements on your SSR:
Today we continue our review of how to meet the requirements ARC-PA’s 5th Edition Standard’s Appendix 14F requirements, which include your presentation of PANCE outcomes in your PA program, looking at how admissions, course grades, test grades and other points of data correlate with these outcomes, and which of these are predictors of success or failure on the test. In this edition of our blog, we will discuss the purpose of correlating PANCE scores with 1) number of C-grades; 2) number of students remediations and 3) preceptor ratings.
When a student receives a near-failing or failing grade in a class, remediation comes swiftly. Something is clearly wrong. But C-Grades, which technically imply “average” performance in a classroom setting, tell their own story. Correlation of C-grades to PANCE scores should indicate whether the number of C grades is statistically significantly correlated with PANCE scores. We have...
Thanks for “clicking in” to my blog once again! As you know, we are deep into a review of the various requirements of Appendix 14 of ARC-PA’s 5th Edition Standards. Today we begin looking at Appendix 14F requirements, and some ideas on how the data may be presented.
In Appendix 14F, the Commission requests your presentation of PANCE outcomes in your PA program. This means looking at how admissions, course grades, test grades and other points of data correlate with PANCE scores, and which of these are predictors of success or failure on the test.
Data analysis related to PANCE outcomes is to include, but is not limited to, correlation of PANCE outcomes and: