ARC-PA 5th Edition Standards: Appendix 14H

ARC-PA’s 5th Edition Standards require that your program’s goals become part of the data you collect and review. Appendix 14H is about your success in meeting program goals. Its instructions are:

• The program must provide information in the template below for all of its published goals. Space has been provided for six program goals, but if the program has more than six goals use an additional template. Reference other appendices of the SSR as needed.
• Provide a tabular or graphic display of data collected by the program for each of its goals for the past three academic years. If already provided in another appendix, reference that data in the analysis narrative
• When creating your data display(s) please keep in mind: 
• Quantitative data must be reported in aggregate and displayed in tables or graphs that directly support the analysis. 
• Qualitative data themes used in the analysis must be reported and summarized in the narrative or displayed in an appended document.

(In other words, for these two sub-points, do not simply provide raw data and expect that to be sufficient!)

• Data should be presented in a way that allows comparison across years and appreciation of trends over time
• Reference other appendices of the SSR as needed. 

Rather than simply reiterating what is on your website, my viewpoint is that the Commission is looking for a more detailed analysis. They want not only to know what your program goals are, but the rationale behind them, and how you are tracking your progress toward those goals.

What I recommend is stating your goals in terms that are measurable. If your program’s goals are not measurable, consider updating them. Your can then clearly demonstrate your program’s progress toward, or success in meeting, such goals. Sometimes goals are complex, and they have multiple domains within them. Be certain that the data that you display addresses each element of your goal.

Recently, I spent some time visiting several PA program websites to view how their goals were displayed. In the following example, the program lists several outcome measures. Note that some of these are aspirational goals.  

Outcome Measure: 

• Five-year first-time PANCE pass rate that meets or exceeds the national average

• 100% of preceptors choose above average or outstanding in medical knowledge. 

• 100% of students were perceived as highly prepared by simulated patients in the simulation center
• Students perceive that they are highly trained to provide high-quality compassionate patient centered care with 100% of students choosing “agree” and “strongly agree.”

First time PANCE pass-rates can easily be measured against the national average. However, the remaining three goals may be “aspirational” in nature – the program strives to achieve 100% ratings, but a perfect score is hard to consistently achieve. When explaining aspirational goals in your SSR, tell what the target is (100% of students perceived as highly prepared) and the number achieved (for example, 80% of students were perceived as highly prepared) – then explain what methodology you will use to continue to improve that score. Aspirational goals can enhance your measurements, as well: what data will you observe to rate your progress toward a certain aspiration?

When displaying goals, I found a useful tactic to be the rationale statement, which tells the reader why the program considers a goal to be important.

How granular do we want to get with outcome data? This really depends on your goal. Outcome B above looks at three-year aggregate scores for preceptor evaluations. However, if you want to look at a specific element, such as “interpersonal skills,” “soft skills,” or “people skills,” (as in the abbreviated goal “students perceive that they trained to provide compassionate patient care…”), then you might indeed want to look at the component of the preceptor evaluations that look at interpersonal elements. An aggregate score might not tell the whole story. Your need for granularity depends on how you state the goal.

Here is another example:

Goal: To support and encourage service, scholarship, leadership and other activities that lead to the advancement of a personal career and the profession.

There is quite a bit to unpack in this goal. Outcomes need to address service, scholarship and leadership too. I recommend breaking goals down into the sum of their part. Address each element and then decide which outcome measure will answer the question.  Does “x” number of service hours indicate support and encouragement? How do you define scholarship? Is scholarship related to faculty? Should you provide examples of the scholarship achieved? What outcome shows a that leadership is supported and encouraged?

To clarify, I recommend that for each program goal, you define: 1) the rationale of the goal; 2) the outcome measure that supports the goal; and 3) your actual outcome.

Conclusion

With that, we have reached the end of Appendix 14! In my next blog, I will wrap up this series by giving you some final advice and takeaways for meeting the 5th Edition Standards.