Certainty rating in pre-and post-tests of study modules in an online clinical pharmacy course - a pilot study to evaluate teaching and learning

Luetsch, Karen and Burrows, Judith (2016) Certainty rating in pre-and post-tests of study modules in an online clinical pharmacy course - a pilot study to evaluate teaching and learning. BMC Medical Education, 16 1: 267. doi:10.1186/s12909-016-0783-1


Author Luetsch, Karen
Burrows, Judith
Title Certainty rating in pre-and post-tests of study modules in an online clinical pharmacy course - a pilot study to evaluate teaching and learning
Journal name BMC Medical Education   Check publisher's open access policy
ISSN 1472-6920
Publication date 2016-10-14
Year available 2016
Sub-type Article (original research)
DOI 10.1186/s12909-016-0783-1
Open Access Status DOI
Volume 16
Issue 1
Start page 267
Total pages 9
Place of publication London, United Kingdom
Publisher BioMed Central
Language eng
Subject 2700 Medicine
3304 Education
Abstract Background: Graduate and post-graduate education for health professionals is increasingly delivered in an e-learning environment, where automated, continuous formative testing with integrated feedback can guide students' self-assessment and learning. Asking students to rate the certainty they assign to the correctness of their answers to test questions can potentially provide deeper insights into the success of teaching, with test results informing course designers whether learning outcomes have been achieved. It may also have implications for decision making in clinical practice. Methods: A study of pre-and post-tests for five study modules was designed to evaluate the teaching and learning within a pharmacotherapeutic course in an online postgraduate clinical pharmacy program. Certainty based marking of multiple choice questions (MCQ) was adapted for formative pre- and post-study module testing by asking students to rate their certainty of correctness of MCQ answers. Paired t-tests and a coding scheme were used to analyse changes in answers and certainty between pre-and post-tests. A survey evaluated students' experience with the novel formative testing design. Results: Twenty-nine pharmacists enrolled in the postgraduate program participated in the study. Overall 1315 matched pairs of MCQ answers and certainty ratings between pre- and post-module tests were available for evaluation. Most students identified correct answers in post-tests and increased their certainty compared to pre-tests. Evaluation of certainty ratings in addition to correctness of answers identified MCQs and topic areas for revision to course designers. A survey of students showed that assigning certainty ratings to their answers assisted in structuring and focusing their learning throughout online study modules, facilitating identification of areas of uncertainty and gaps in their clinical knowledge. Conclusions: Adding certainty ratings to MCQ answers seems to engage students with formative testing and feedback and focus their learning in a web-based postgraduate pharmacy course. It also offers deeper insight into the successful delivery of online course content, identifying areas for improvement of teaching and content delivery as well as test question design.
Formatted abstract
Background: Graduate and post-graduate education for health professionals is increasingly delivered in an e-learning environment, where automated, continuous formative testing with integrated feedback can guide students’ self-assessment and learning. Asking students to rate the certainty they assign to the correctness of their answers to test questions can potentially provide deeper insights into the success of teaching, with test results informing course designers whether learning outcomes have been achieved. It may also have implications for decision making in clinical practice.

Methods:
A study of pre-and post-tests for five study modules was designed to evaluate the teaching and learning within a pharmacotherapeutic course in an online postgraduate clinical pharmacy program. Certainty based marking of multiple choice questions (MCQ) was adapted for formative pre- and post-study module testing by asking students to rate their certainty of correctness of MCQ answers. Paired t-tests and a coding scheme were used to analyse changes in answers and certainty between pre-and post-tests. A survey evaluated students’ experience with the novel formative testing design.

Results:
Twenty-nine pharmacists enrolled in the postgraduate program participated in the study. Overall 1315 matched pairs of MCQ answers and certainty ratings between pre- and post-module tests were available for evaluation. Most students identified correct answers in post-tests and increased their certainty compared to pre-tests. Evaluation of certainty ratings in addition to correctness of answers identified MCQs and topic areas for revision to course designers. A survey of students showed that assigning certainty ratings to their answers assisted in structuring and focusing their learning throughout online study modules, facilitating identification of areas of uncertainty and gaps in their clinical knowledge.

Conclusions:
Adding certainty ratings to MCQ answers seems to engage students with formative testing and feedback and focus their learning in a web-based postgraduate pharmacy course. It also offers deeper insight into the successful delivery of online course content, identifying areas for improvement of teaching and content delivery as well as test question design.
Keyword Formative assessment
Formative feedback
E-learning
Certainty based marking
Pharmacy
Postgraduate
Q-Index Code C1
Q-Index Status Provisional Code
Grant ID 2012001278
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: HERDC Pre-Audit
School of Pharmacy Publications
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in Thomson Reuters Web of Science Article
Scopus Citation Count Cited 0 times in Scopus Article
Google Scholar Search Google Scholar
Created: Sun, 30 Oct 2016, 20:42:51 EST by Karen Luetsch on behalf of School of Pharmacy