Jump to content

Evidence-based assessment/Step 11: Progress and process measures

Add links
From Wikiversity
Click Here for Landing Page
Click Here for Landing Page
HGAPS New for Fall 2022: HGAPS and Psychology Conferences
Click Here for Landing Page
Click Here for Landing Page

HGAPS is finding new ways to make psychological science conferences more accessible!

Here are examples from APA 2022 and the JCCAP Future Directions Forum. Coming soon... ABCT!
~ More at HGAPS.org ~


Medical disclaimer: This page is for educational and informational purposes only and may not be construed as medical advice. The information is not intended to replace medical advice offered by physicians. Please refer to the full text of the Wikiversity medical disclaimer.



EBA Implementation
Assessment phases
Steps 1-2: Preparation phase

Preparation phase

Steps 3-5: Prediction phase

Prediction phase

Steps 6-9: Prescription phase

Prescription phase

Steps 10-12: Process/progress/outcome phase

Process phase

Process Phase: Progress and Process Measures

[edit | edit source]

Overview

[edit | edit source]

The most fine-grained level of assessment tracks change in between sessions. In the journey of therapy, this is the GPS measuring progress towards goals (how many kilometers covered?). Daily or session-level measures act as the speedometer, providing a sense of how rapidly things are progressing, or whether they are stalled or moving backwards. Practicing without systematic assessment is akin to barreling blind into space – unlikely to help you arrive at your destination on time and without significant detours. Mapping out your route before you leave and checking progress against that plan allows for recalculating when obstacles are encountered and an ongoing conversation between therapist and client about what the ultimate destination is and how best to get there.


An analysis of the treatment manuals for a set of ESTs observed that all of them included some form of progress tracking, whereas treatment as usual often tracks progress informally and impressionistically, if at all.[1] The techniques used here vary widely across interventions, from daily report cards completed by teachers, to sticker charts and token economies, or three and five column charts in CBT, diary cards in DBT, and daily life or mood charts for mood disorder, including several free examples that have been customized for children. These regular progress measures increase the effectiveness of the treatment[2], the same as how weigh-ins give a diet more traction. Giving clinicians very simple feedback about whether the client is making good progress, stalled, or worsening changes therapy outcomes, including a reduction in premature termination and better retention of difficult cases, as well as faster planned termination of cases who responded well.[3][4][5][6][7][8] Streamlined assessment tracking a few top problems on a session-by-session basis is both feasible and highly sensitive to treatment effects with youths and families.[9] Figure 5 shows an example of plotting session level tracking using the Top Problems type of scaling, juxtaposed with tracking the session progress.[10] The more frequent schedule of measurement makes these data even more useful as a harbinger of progress or setbacks, and makes more rapid course correction possible. These data also can signal appropriate times to transition to tapering and planned termination, as well as continued monitoring to maintain gains. Tracking process is where the traditional assessment measures are weakest. None of the commonly taught and used measures are suitable for weekly, let alone daily, repetition. This is a niche ripe for innovation.[11] This also is the place where technology is most rapidly transforming assessment. Mood charts have migrated from paper and pencil[12] to online[13], and thence to text messaging[14] and smartphone apps.[15] Daily report cards[16] now can be emailed or texted to teachers, and texting also makes ecological momentary assessment of mood and energy feasible[17][18][19]. Apps can now use phones as actigraphs, providing objective measures of energy and sleep.[20] Wearable devices such as the FitBit and smart watches also add to the potential for passively tracking footsteps, other motor activity, heartrate, and peripheral temperature. Apps can even monitor the cadence and intonation of speech to assess for depression or hostility.[21]


Sleep tracking example. Figure 6 shows two forms of sleep data that an inexpensive smartphone app generates. Our client would install the app on her phone, turn it on as she went to bed, and put the phone on the mattress next to her. The phone’s gyroscope detects her movements during the night, and uses algorithms to estimate when she fell asleep, when she woke, and even sleep stage (light sleep, deep sleep, and REM). The sleep onset and duration measures correlate surprisingly well with established criterion measures (REM not so well yet).[22] The phone methodology creates a low burden way for her to track sleep, revealing periods of sleep disruption that then can be discussed in treatment.


Monitoring social media can also gather information about risky behaviors or suicidal ideation, and feasibly could lead to passive assessments of other psychological constructs that do not require any additional effort on the part of the patient[23](see also the Durkheim Project, developed for the US military). These new information systems are still getting patched into the dashboard guiding therapy, and passive monitoring raises a new set of ethical considerations that need attention. Is informed consent needed before monitoring public Facebook pages for cues of risk? Is the consent of the parent required even though the information is public? Is the assent of the youth required? The questions, and workable solutions, will come into focus quickly in the next few years.

The ongoing assessment also makes course-correction possible, improving the chances of a good outcome and perhaps reducing the risk of premature drop-out as well as preventing harm. When clinicians were randomly chosen to receive ongoing progress feedback, only 13% of clients deteriorated versus 21% when the clinicians did not get the progress monitoring; and 35% of clients made significant gains in the progress monitoring condition, versus only 21% in the treatment as usual arm[24]. These rates convert into a Number Needed to Treat of 7.1, meaning that adding routine progress monitoring will create one more case with significant gains per seven cases added[25]. More provocatively, continuing with treatment as usual results in a Number Needed to Harm of 12.5, meaning that roughly one in a dozen cases would have a deteriorating course that could have been prevented by adding progress monitoring[26]. Weekly feedback to the therapist produces significantly faster response, with increased doses of feedback enhancing the patient response[27]. Measurement-based care increases the rate and speed of response, roughly doubling the rate of remission achieved with pharmacotherapy of major depression in a randomized trial versus assessment as usual[28]. It also is demonstrating good ability to identify both youth[29] and adult cases at risk of dropping out[30], outperforming clinician judgment[31]. This makes it possible to have an early warning system that could prompt the clinician to actively re-engage and discuss things with the patient – a win for both parties when it leads to better outcomes.

Rationale

[edit | edit source]

Steps to put into practice

[edit | edit source]

There are several options about how to do this:

  • Jacobson and colleagues developed a two part framework for clinically significant change, combining a "Reliable Change Index" (RCI) with a set of benchmarks defined by norms based on people with and without a clinical condition. Several technical refinements have been suggested, but the Jacobson model remains one of the most well known approaches. It also is a worthy example of how to use psychometric and nomothetic data to set goals and measure outcomes for individual cases.
  • Brinley plots
  • Graphical approaches
  • Time series analysis


Tables and figures

[edit | edit source]

References

[edit | edit source]
  1. ↑ Hunsley, J. (2007). Training psychologists for evidence-based practice. Canadian Psychology, 38, 32-42. 
  2. ↑ Guo, T., Xiang, Y. T., Xiao, L., Hu, C. Q., Chiu, H. F., Ungvari, G. S., . . . Wang, G. (2015). Measurement-Based Care Versus Standard Care for Major Depression: A Randomized Controlled Trial With Blind Raters. American Journal of Psychiatry, 172(10), 1004-1013. doi: 10.1176/appi.ajp.2015.14050652
  3. ↑ Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155-163. doi: 10.1002/jclp.20108
  4. ↑ Harmon, C., Hawkins, E. J., Lambert, M. J., Slade, K., & Whipple, J. S. (2005). Improving outcomes for poorly responding clients: the use of clinical support tools and feedback to clients. Journal of Clinical Psychology, 61(2), 175-185. doi: 10.1002/jclp.20109
  5. ↑ Hatfield, D., McCullough, L., Frantz, S. H., & Krieger, K. (2010). Do we know when our clients get worse? an investigation of therapists' ability to detect negative client change. Clinical Psychology & Psychotherapy, 17(1), 25-32. doi: 10.1002/cpp.656
  6. ↑ Lambert, M. J., Whipple, J. L., Vermeersch, D. A., Smart, D. W., Hawkins, E. J., Nielsen, S. L., & Goates, M. (2002). Enhancing psychotherapy outcomes via providing feedback on client progress: a replication. Clinical Psychology & Psychotherapy, 9(2), 91-103. doi: 10.1002/cpp.324
  7. ↑ Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The partners for change outcome management system. Journal of Clinical Psychology, 61(2), 199-208. doi: 10.1002/jclp.20111
  8. ↑ Sapyta, J., Riemer, M., & Bickman, L. (2005). Feedback to clinicians: theory, research, and practice. Journal of Clinical Psychology, 61(2), 145-153. doi: 10.1002/jclp.20107
  9. ↑ Weisz, J. R., Chorpita, B. F., Frye, A., Ng, M. Y., Lau, N., Bearman, S. K., . . . Hoagwood, K. E. (2011). Youth Top Problems: using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. Journal of Consulting and Clinical Psychology, 79(3), 369-380. doi: 10.1037/a0023307
  10. ↑ Powsner, S. M., & Tufte, E. R. (1994). Graphical summary of patient status. The Lancet, 344, 368-389. doi: 10.1016/S0140-6736(94)91406-0
  11. ↑ Slade, K., Lambert, M. J., Harmon, S. C., Smart, D. W., & Bailey, R. (2008). Improving psychotherapy outcome: the use of immediate electronic feedback and revised clinical support tools. Clinical Psychology & Psychotherapy, 15(5), 287-303. doi: 10.1002/cpp.594
  12. ↑ Denicoff, K. D., Smith-Jackson, E. E., Disney, E. R., Suddath, R. L., Leverich, G. S., & Post, R. M. (1997). Preliminary evidence of the reliability and validity of the prospective life-chart methodology (LCM-p). Journal of Psychiatric Research, 31(5), 593-603. 
  13. ↑ Bauer, M., Glenn, T., Alda, M., Grof, P., Sagduyu, K., Bauer, R., . . . Whybrow, P. C. (2011). Comparison of pre-episode and pre-remission states using mood ratings from patients with bipolar disorder. Pharmacopsychiatry, 44 Suppl 1, S49-53. doi: 10.1055/s-0031-1273765
  14. ↑ Bopp, J. M., Miklowitz, D. J., Goodwin, G. M., Stevens, W., Rendell, J. M., & Geddes, J. R. (2010). The longitudinal course of bipolar disorder as revealed through weekly text messaging: a feasibility study. Bipolar Disorders, 12(3), 327-334. doi: 10.1111/j.1399-5618.2010.00807.x
  15. ↑ Faurholt-Jepsen, M., Frost, M., Vinberg, M., Christensen, E. M., Bardram, J. E., & Kessing, L. V. (2014). Smartphone data as objective measures of bipolar disorder symptoms. Psychiatry Research, 217(1-2), 124-127. doi: 10.1016/j.psychres.2014.03.009
  16. ↑ Pelham, W. E., Jr., Fabiano, G. A., & Massetti, G. M. (2005). Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child & Adolescent Psychology, 34, 449-476. doi: 10.1207/s15374424jccp3403_5
  17. ↑ Axelson, D. A., Bertocci, M. A., Lewin, D. S., Trubnick, L. S., Birmaher, B., Williamson, D. E., . . . Dahl, R. E. (2003). Measuring mood and complex behavior in natural environments: use of ecological momentary assessment in pediatric affective disorders. Journal of Child and Adolescent Psychopharmacology, 13(3), 253-266. 
  18. ↑ Kimhy, D., Delespaul, P., Corcoran, C., Ahn, H., Yale, S., & Malaspina, D. (2006). Computerized experience sampling method (ESMc): assessing feasibility and validity among individuals with schizophrenia. Journal of Psychiatric Research, 40(3), 221-230. doi: 10.1016/j.jpsychires.2005.09.007
  19. ↑ Trull, T. J., & Ebner-Priemer, U. (2013). Ambulatory assessment. Annual review of clinical psychology, 9, 151-176. doi: 10.1146/annurev-clinpsy-050212-185510
  20. ↑ Faurholt-Jepsen, M., Vinberg, M., Frost, M., Christensen, E. M., Bardram, J. E., & Kessing, L. V. (2015). Smartphone data as an electronic biomarker of illness activity in bipolar disorder. Bipolar Disorders, 17(7), 715-728. doi: 10.1111/bdi.12332
  21. ↑ Low, L. S., Maddage, N. C., Lech, M., Sheeber, L. B., & Allen, N. B. (2011). Detection of clinical depression in adolescents' speech during family interactions. IEEE Translational Biomedical Engineering, 58(3), 574-586. doi: 10.1109/TBME.2010.2091640
  22. ↑ Lee, J. M., Kim, Y., & Welk, G. J. (2014). Validity of consumer-based physical activity monitors. Med Sci Sports Exerc, 46(9), 1840-1848. doi: 10.1249/MSS.0000000000000287
  23. ↑ Won, H. H., Myung, W., Song, G. Y., Lee, W. H., Kim, J. W., Carroll, B. J., & Kim, D. K. (2013). Predicting national suicide numbers with social media data. PLoS One, 8(4), e61809. doi: 10.1371/journal.pone.0061809
  24. ↑ Lambert, M. J. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10(3), 288-301. doi: 10.1093/clipsy.bpg025
  25. ↑ Straus, S. E., Glasziou, P., Richardson, W. S., & Haynes, R. B. (2011). Evidence-based medicine: How to practice and teach EBM (4th ed.). New York, NY: Churchill Livingstone.
  26. ↑ Straus, S. E., Glasziou, P., Richardson, W. S., & Haynes, R. B. (2011). Evidence-based medicine: How to practice and teach EBM (4th ed.). New York, NY: Churchill Livingstone.
  27. ↑ Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatric Services, 62(12), 1423-1429. doi: 10.1176/appi.ps.002052011
  28. ↑ Guo, T., Xiang, Y. T., Xiao, L., Hu, C. Q., Chiu, H. F., Ungvari, G. S., . . . Wang, G. (2015). Measurement-Based Care Versus Standard Care for Major Depression: A Randomized Controlled Trial With Blind Raters. American Journal of Psychiatry, 172(10), 1004-1013. doi: 10.1176/appi.ajp.2015.14050652
  29. ↑ Nelson, P. L., Warren, J. S., Gleave, R. L., & Burlingame, G. M. (2013). Youth psychotherapy change trajectories and early warning system accuracy in a managed care setting. Journal of Clinical Psychology, 69(9), 880-895. doi: 10.1002/jclp.21963
  30. ↑ Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155-163. doi: 10.1002/jclp.20108
  31. ↑ Hatfield, D., McCullough, L., Frantz, S. H., & Krieger, K. (2010). Do we know when our clients get worse? an investigation of therapists' ability to detect negative client change. Clinical Psychology & Psychotherapy, 17(1), 25-32. doi: 10.1002/cpp.656
Retrieved from "https://en.wikiversity.org/w/index.php?title=Evidence-based_assessment/Step_11:_Progress_and_process_measures&oldid=2287937"