A new method for assessing content validity in model-based creation and iteration of eHealth interventions

Kassam-Adams, Nancy, Marsac, Meghan L., Kohser, Kristen L., Kenardy, Justin A., March, Sonja and Winston, Flaura K. (2015) A new method for assessing content validity in model-based creation and iteration of eHealth interventions. Journal of Medical Internet Research, 17 4: e95.1-e95.8. doi:10.2196/jmir.3811

Author Kassam-Adams, Nancy
Marsac, Meghan L.
Kohser, Kristen L.
Kenardy, Justin A.
March, Sonja
Winston, Flaura K.
Title A new method for assessing content validity in model-based creation and iteration of eHealth interventions
Journal name Journal of Medical Internet Research   Check publisher's open access policy
ISSN 1438-8871
Publication date 2015-04-01
Sub-type Article (original research)
DOI 10.2196/jmir.3811
Open Access Status DOI
Volume 17
Issue 4
Start page e95.1
End page e95.8
Total pages 8
Place of publication Toronto, ON, Canada
Publisher Journal of Medical Internet Research
Language eng
Formatted abstract
Background: The advent of eHealth interventions to address psychological concerns and health behaviors has created new opportunities, including the ability to optimize the effectiveness of intervention activities and then deliver these activities consistently to a large number of individuals in need. Given that eHealth interventions grounded in a well-delineated theoretical model for change are more likely to be effective and that eHealth interventions can be costly to develop, assuring the match of final intervention content and activities to the underlying model is a key step. We propose to apply the concept of "content validity" as a crucial checkpoint to evaluate the extent to which proposed intervention activities in an eHealth intervention program are valid (eg, relevant and likely to be effective) for the specific mechanism of change that each is intended to target and the intended target population for the intervention.

Objective: The aims of this paper are to define content validity as it applies to model-based eHealth intervention development, to present a feasible method for assessing content validity in this context, and to describe the implementation of this new method during the development of a Web-based intervention for children.

Methods: We designed a practical 5-step method for assessing content validity in eHealth interventions that includes defining key intervention targets, delineating intervention activity-target pairings, identifying experts and using a survey tool to gather expert ratings of the relevance of each activity to its intended target, its likely effectiveness in achieving the intended target, and its appropriateness with a specific intended audience, and then using quantitative and qualitative results to identify intervention activities that may need modification. We applied this method during our development of the Coping Coach Web-based intervention for school-age children.

Results: In the evaluation of Coping Coach content validity, 15 experts from five countries rated each of 15 intervention activity-target pairings. Based on quantitative indices, content validity was excellent for relevance and good for likely effectiveness and age-appropriateness. Two intervention activities had item-level indicators that suggested the need for further review and potential revision by the development team.

Conclusions: This project demonstrated that assessment of content validity can be straightforward and feasible to implement and that results of this assessment provide useful information for ongoing development and iterations of new eHealth interventions, complementing other sources of information (eg, user feedback, effectiveness evaluations). This approach can be utilized at one or more points during the development process to guide ongoing optimization of eHealth interventions.
Keyword Telemedicine
Stress Disorders
Secondary Prevention
Q-Index Code C1
Q-Index Status Confirmed Code
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: Faculty of Health and Behavioural Sciences -- Publications
Official 2016 Collection
School of Psychology Publications
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 4 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 4 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Sun, 24 May 2015, 11:01:29 EST by System User on behalf of School of Psychology