Comparison of four expert elicitation methods: for Bayesian logistic regression and classification trees

O'Leary, R. A., Mengersen, K., Murray, J. V. and Low Choy, S. (2009). Comparison of four expert elicitation methods: for Bayesian logistic regression and classification trees. In: R. S. Anderssen, R. D. Braddock and L. T. H. Newham, 18th World IMACS Congress and MODSIM09 International Congress on Modelling and Simulation. Proceedings. The 18th World IMACS Congress and MODSIM09 International Congress on Modelling and Simulation, Cairns, QLD, Australia, (4276-4282). 13-17 July, 2009.

Author O'Leary, R. A.
Mengersen, K.
Murray, J. V.
Low Choy, S.
Title of paper Comparison of four expert elicitation methods: for Bayesian logistic regression and classification trees
Conference name The 18th World IMACS Congress and MODSIM09 International Congress on Modelling and Simulation
Conference location Cairns, QLD, Australia
Conference dates 13-17 July, 2009
Proceedings title 18th World IMACS Congress and MODSIM09 International Congress on Modelling and Simulation. Proceedings
Journal name 18Th World Imacs Congress and Modsim09 International Congress On Modelling and Simulation
Place of Publication Christchurch, NZ
Publisher Modelling and Simulation Society of Australia and New Zealand
Publication Year 2009
Sub-type Fully published paper
Open Access Status
ISBN 9780975840078
Editor R. S. Anderssen
R. D. Braddock
L. T. H. Newham
Start page 4276
End page 4282
Total pages 7
Language eng
Formatted Abstract/Summary
In the ecological field, expert opinion has been acknowledged as providing valuable information in modelling, particularly when the observed data are limited or unreliable. Indeed international recommendations are that expert-driven models are to be preferred to statistical models for habitat modelling (Langhammer et al., 2007). However expert-driven models are not calibrated to empirical data and therefore rely entirely on the credibility and expertise of the experts. Bayesian statistical modelling provides a useful bridge between purely expert-driven models and data-driven statistical models. Several methods are available for eliciting expert knowledge into Bayesian statistical models in a variety of contexts, in general (O’Hagan et al., 2006) and in ecology (Low Choy et al., 2009). For example logistic regression (LR) is a common choice for modelling the probability of presence or absence for a species and how this relates to a number of habitat covariates, e.g. vegetation, geology, topography and climate (Guisan and Zimmermann 2000).

Recently the authors compared three elicitation methods for Bayesian regression in the context of habitat modelling (O’Leary et al., 2008a). These included a questionnaire-based method (similar to Kuhnert et al., 2005; Martin et al., 2005), which simply asks experts whether each covariate xj increases, decreases or has essentially no effect on the response y. Alternatively using a software tool (Kynn 2005), experts could also be asked to draw a species response curve showing how the probability of presence (on the y-axis) changed with a particular habitat covariate such as geology type (on the x-axis), with all other covariates held at their optimum. Finally an elicitation tool embedded within a GIS (Denham and Mengersen 2007) could be used to help experts select sites on a map, inspect the habitat characteristics at and surrounding each site, and then assess the probability of presence at each site. These three approaches were compared for habitat suitability modelling of the threatened Australian brush-tailed rock-wallaby Petrogale penicillata (O’Leary et al., 2008a). This comparison found substantial differences in the three elicitation approaches in how the expert knowledge translated into the Bayesian statistical model.

In this paper we extend this comparison to consider a method, newly developed by the authors (O’Leary et al., 2008b), for elicitation of expert opinion into Bayesian classification trees. Logistic regression and classification trees are obvious contenders for modelling the relationship between a binary response (e.g. presence/absence) and several covariates. Indeed classification trees are another statistical modelling approach often applied in the habitat modelling context (Murray et al., 2008), popular since they provide an easily understood graphical representation of a decision tree. Until recently, however, no method was available for incorporating expert knowledge into classification trees. Using the new approach, elicitation questions focus on the size of the tree representing the number of decisions; the relative importance of the covariates; and the splitting rules for the most important covariates which quantify how decisions relate to variables (O’Leary et al., 2008b).

Hence this paper compares four elicitation approaches for modelling the habitat suitability of the rock-wallaby, using the same dataset: three Bayesian logistic regression methods and one Bayesian classification tree method. We found that there were some dissimilarities between the expert informed priors formulated using the different methods, but all approaches identified that northern aspects have the highest probability of presence. This paper demonstrates that combining expert informed priors with limited observed data using one or more of the elicitation approaches may improve scientific understanding and therefore contribute to conservation management planning.
Subjects 1703 Computational Theory and Mathematics
2605 Computational Mathematics
2611 Modelling and Simulation
Keyword Bayesian classification and regression trees
Bayesian logistic regression
Expert elicitation
Informative priors
Q-Index Code E1
Q-Index Status Provisional Code
Institutional Status UQ

Document type: Conference Paper
Collection: School of Biological Sciences Publications
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in Thomson Reuters Web of Science Article
Scopus Citation Count Cited 0 times in Scopus Article
Google Scholar Search Google Scholar
Created: Wed, 27 Nov 2013, 22:55:17 EST by System User on behalf of School of Biological Sciences