An occupation-based, metacognitive approach to assessing error performance and online awareness

Doig, Emmah, Fleming, Jennifer, Ownsworth, Tamara and Fletcher, Stephanie (2016) An occupation-based, metacognitive approach to assessing error performance and online awareness. Australian Journal of Occupational Therapy, 64 2: 137-148. doi:10.1111/1440-1630.12322


Author Doig, Emmah
Fleming, Jennifer
Ownsworth, Tamara
Fletcher, Stephanie
Title An occupation-based, metacognitive approach to assessing error performance and online awareness
Journal name Australian Journal of Occupational Therapy   Check publisher's open access policy
ISSN 0045-0766
1440-1630
Publication date 2016-01-01
Year available 2016
Sub-type Article (original research)
DOI 10.1111/1440-1630.12322
Open Access Status Not yet assessed
Volume 64
Issue 2
Start page 137
End page 148
Total pages 12
Place of publication Richmond, VIC Australia
Publisher Wiley-Blackwell Publishing Asia
Language eng
Subject 3609 Occupational Therapy
Abstract Background/aim: Online awareness is the ability to self-monitor, identify and self-correct errors while engaged in an activity. Current assessments of online awareness involve observing and classifying error behaviour during structured, uniform tasks. However, during rehabilitation, practitioners typically work towards improving performance in individually meaningful tasks unique to the client. This article presents a metacognitive, task analytic approach to assessing online awareness involving observation and classification of errors during meaningful occupations determined after client-centred goal setting with two male clients with severe traumatic brain injury (aged 22 and 23). Study aims were to describe the approach, evaluate its feasibility and determine inter-rater agreement for error detection and error categorisation by two experienced occupational therapists. Furthermore, the error profiles and cognitive impairments of the participants on standardised neuropsychological assessment were examined to explore the validity of the assessment. Methods: Individualised assessment tasks included snack preparation, budgeting, timetabling, hot-drink preparation and use of a computer program, which were administered repeatedly over two to three months and audio-visual recordings taken. Independent ratings of two trained occupational therapists were compared using exact percent agreement. Results: Overall agreement about errors was 76%, for which there was 65% agreement about error categorisation and 100% agreement about error correction. Conclusions: There was fair inter-rater agreement between two trained occupational therapists of error behaviour and error correction when using the described occupation-based approach to assessing online awareness. This approach has promise, particularly when combined with standardised, neuropsychological assessments, for providing an in-depth understanding of error behaviour and awareness of errors during meaningful occupations.
Formatted abstract
Background/aim

Online awareness is the ability to self-monitor, identify and self-correct errors while engaged in an activity. Current assessments of online awareness involve observing and classifying error behaviour during structured, uniform tasks. However, during rehabilitation, practitioners typically work towards improving performance in individually meaningful tasks unique to the client. This article presents a metacognitive, task analytic approach to assessing online awareness involving observation and classification of errors during meaningful occupations determined after client-centred goal setting with two male clients with severe traumatic brain injury (aged 22 and 23). Study aims were to describe the approach, evaluate its feasibility and determine inter-rater agreement for error detection and error categorisation by two experienced occupational therapists. Furthermore, the error profiles and cognitive impairments of the participants on standardised neuropsychological assessment were examined to explore the validity of the assessment.

Methods

Individualised assessment tasks included snack preparation, budgeting, timetabling, hot-drink preparation and use of a computer program, which were administered repeatedly over two to three months and audio-visual recordings taken. Independent ratings of two trained occupational therapists were compared using exact percent agreement.

Results

Overall agreement about errors was 76%, for which there was 65% agreement about error categorisation and 100% agreement about error correction.

Conclusions

There was fair inter-rater agreement between two trained occupational therapists of error behaviour and error correction when using the described occupation-based approach to assessing online awareness. This approach has promise, particularly when combined with standardised, neuropsychological assessments, for providing an in-depth understanding of error behaviour and awareness of errors during meaningful occupations.
Keyword Awareness
Brain injury
Executive function
Patient outcome assessment
Rehabilitation
Q-Index Code C1
Q-Index Status Provisional Code
Grant ID 2012003082
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: HERDC Pre-Audit
School of Health and Rehabilitation Sciences Publications
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in Thomson Reuters Web of Science Article
Scopus Citation Count Cited 0 times in Scopus Article
Google Scholar Search Google Scholar
Created: Sat, 24 Sep 2016, 04:22:43 EST by Emmah Doig on behalf of School of Health & Rehabilitation Sciences