Examining the inter-rater and test-retest reliability of the Student Practice Evaluation Form-Revised (SPEF-R) for occupational therapy students

Rodger, Sylvia, Coleman, Allison, Caine, Anne-Maree, Chien, Chi-Wen, Copley, Jodie, Turpin, Merrill and Brown, Ted (2014) Examining the inter-rater and test-retest reliability of the Student Practice Evaluation Form-Revised (SPEF-R) for occupational therapy students. Australian Occupational Therapy Journal, 61 5: 353-363. doi:10.1111/1440-1630.12151


Author Rodger, Sylvia
Coleman, Allison
Caine, Anne-Maree
Chien, Chi-Wen
Copley, Jodie
Turpin, Merrill
Brown, Ted
Title Examining the inter-rater and test-retest reliability of the Student Practice Evaluation Form-Revised (SPEF-R) for occupational therapy students
Journal name Australian Occupational Therapy Journal   Check publisher's open access policy
ISSN 0045-0766
1440-1630
Publication date 2014-10
Year available 2014
Sub-type Article (original research)
DOI 10.1111/1440-1630.12151
Open Access Status
Volume 61
Issue 5
Start page 353
End page 363
Total pages 11
Place of publication Richmond, VIC Australia
Publisher Wiley-Blackwell Publishing Asia
Collection year 2015
Language eng
Abstract Background/aim: The Student Practice Evaluation Form-Revised Edition (SPEF-R) is widely used in Australian universities to evaluate occupational therapy students' performance in practice education. Reliable completion of the SPEF-R by practice educators is critical for students and universities from a quality assurance perspective. This study used standardised video vignettes to examine the inter-rater and test-retest reliability of practice educators when completing the SPEF-R. Methods: Nine vignettes were developed with three levels of student performance (below expectations, adequate-to-proficient, and outstanding) for each of three scenarios which depicted a multidisciplinary team meeting, a supervision session, and a home visit respectively. Seventy-five occupational therapy practice educators viewed the vignettes and completed an online survey to rate student performance in each vignette using five selected SPEF-R items and using the five-point rating scale. Twenty of these practice educators completed the rating process twice for test-retest reliability purposes. Percentage agreement, t-tests and Rasch Measurement Model were used for analysis. Results: Practice educators exhibited consistent ratings on most of the SPEF-R items, except for rating adequate-to-proficient student performance on three items for inter-rater reliability and eight items for test-retest reliability. Rasch analysis found that the majority of practice educators (96-98.7%) exhibited consistency in their use of the SPEF-R rating scales. Practice educators also demonstrated satisfactory test-retest agreement of severity/leniency in rating student performance in the multidisciplinary team meeting scenario. Conclusion: The study findings suggest that the SPEF-R could be used reliably and interpreted consistently by practice educators with diverse backgrounds and levels of experience.
Keyword Education
Occupational therapy
Professional
Reproducibility of results
Q-Index Code C1
Q-Index Status Confirmed Code
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: Official 2015 Collection
School of Health and Rehabilitation Sciences Publications
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 1 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 1 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Tue, 21 Oct 2014, 00:35:31 EST by System User on behalf of School of Health & Rehabilitation Sciences