The Purpose of Advanced Assessment Methods
Welcome back to my blog series on using advanced assessment methods in the interpretation of the data your program collects. The ARC-PA Commission requires that you collect and present this to them; advanced assessment means deriving two different benefits.
First, your SSR will be a better representation of what the committee wants to see.
Remember that advanced assessment does not “replace” the minimum requirements for the SSR.
Nor does the accrediting body require you to go deep into statistical analysis on the parametric level. Nevertheless, it can be highly beneficial to do so. One thing the committee does want is for your SSR report to show context, linkage between the appendix sections, and proof that you are aware of cause and effect of variables. It is more important to show linkage between data analysis, conclusions and actions. There are ways to show linkage between descriptive as well as parametric components.
When you generate data points from different angles, oftentimes they have connections. This reinforces the interrelated nature of assessment. I am going to give you some examples of that. ARC-PA talks about the organic connection between all elements of the self-study report, so really the best example that you can provide is, for example, the cross-reference appendix 14F into 14E, or vice versa. Or even put elements of Appendix C within F. There are ways to cross-pollinate that process. In upcoming blogs, I will demonstrate this with sample spider diagrams, showing how all the parts will intersect.
Second, your program will benefit from your advanced understanding.
The most intimidating part of this process is simply getting started – organizing the various committees and arranging the flow of data from collection to analysis to conclusions. However, once the system is in place, the amount of information you can extract is impressive and will, eventually, save your program’s and its students’ time, frustration, and even money.
For example, advanced assessment methods can tell you where in the system a critical failure is occurring. In upcoming blogs, we will take a careful look at failing PANCE scores, and tracking backward through data sets to see at what point the trouble begins. We will look at ways to assess the necessity of curriculum changes, or alterations to your admissions process.
The Assessment Model requires a carefully defined process, through which a specific committee structure or designated people analyzes the results. This reinforces the four stages of assessment and ensures the capture of critical analysis in committee documentation.
The first element is the regular and ongoing collection of data. The amount of data we must collect is significant. Looking at quantitative data in tables and charts is the best way. I invite you to look at our Fifth Edition Webinar recordings for in-depth coverage of Appendix 14. I am not going to repeat that in this blog series, except for an applied approach.
The following illustration shows what the culmination of this process might look like.
As I am sure you know by now, I advocate for a very systemized, centralized process for data. This is an example of a data assessment committee minutes. This is the template I recommend, in which you have captured discussions and the action plan. There is always a communication, timeline, and referral process.
In this case, the program determined that low PANCE pass rates were related to the end-of-curriculum exam scores. Based on their analysis, they determined that students who were scoring below 1475 on the EOC had a much higher failure rate, so they incorporated a cut score of 1475. They did the same thing for the end-of-rotation exam. They determined, based on three years of data, that students scoring in the low seventies had a higher probability of failing exams. Thus, the action plan was to incorporate this as a policy, which ended up being communication, and then finally, an implementation process.
In my next blog, we will begin applying advanced assessment theories to concrete and common PA program situations. I will provide several examples of how to analyze cause-and-effect in a data set. I hope you will join me then!
