Hybrid rule-extraction from support vector machines

Diederich, Joachim and Barakat, Nahla (2004). Hybrid rule-extraction from support vector machines. In: 2004 IEEE Conference on Cybernetics and Intelligent Systems. 2004 IEEE Conference on Cybernetics and Intelligent Systems, Singapore, (1271-1276). 1-3 December 2004. doi:10.1109/ICCIS.2004.1460774

Attached Files (Some files may be inaccessible until you login with your UQ eSpace credentials)
Name Description MIMEType Size Downloads
CIS1197_0.pdf CIS1197_0.pdf application/pdf 355.04KB 579

Author Diederich, Joachim
Barakat, Nahla
Title of paper Hybrid rule-extraction from support vector machines
Conference name 2004 IEEE Conference on Cybernetics and Intelligent Systems
Conference location Singapore
Conference dates 1-3 December 2004
Proceedings title 2004 IEEE Conference on Cybernetics and Intelligent Systems
Journal name 2004 Ieee Conference On Cybernetics and Intelligent Systems, Vols 1 and 2
Place of Publication Piscataway, NJ, U.S.A.
Publisher IEEE
Publication Year 2004
Sub-type Fully published paper
DOI 10.1109/ICCIS.2004.1460774
Open Access Status File (Author Post-print)
ISBN 0-7803-8643-4
Volume 2
Start page 1271
End page 1276
Total pages 6
Language eng
Abstract/Summary Rule-extraction from artificial neural networks(ANNs) as well as support vector machines (SVMs) provide explanations for the decisions made by these systems. This explanation capability is very important in applications such as medical diagnosis. Over the last decade, a multitude of algorithms for rule-extraction from ANNs have been developed. However, rule-extraction from SVMs is not widely available yet.In this paper, a hybrid approach for rule-extraction from SVMs is outlined. This approach has two basic components: (1) data reduction using a logistic regression model and (2) learning based rule-extraction. The quality of the extracted rules is then evaluated in terms of fidelity, accuracy, consistency and comprehensibility. The rules are also verified against the available knowledge from the domain problem (diabetes) to assure correctness and validity.
Subjects 280200 Artificial Intelligence and Signal and Image Processing
Keyword Data reduction
Knowledge acquisition
Regression analysis
References [1] T. Joachims, "Transductive Inference for Text Classification using Support Vector Machines", International Conference on Machine Learning (ICML), 1999. [2] A.B. Tickle, R.Andrews, M.Golea, and J.Diederich, "The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural network",IEEE Transactions on Neural Networks, vol. 9(6), pp. 1057-1068, 1998. [3] R. Andrews, J. Diederich, and A.B. Tickle, "A Survey and Critique of Techniques For Extracting Rules From Trained Artificial Neural Networks", Knowledge Based Systems, 1995, vol. 8, pp. 373-389. [4] R. Mitsdorffer, J. Diederich, and C. Tan, "Rule-extraction from Technology IPOs in the US Stock Market", ICONIP02,Singapore, 2002. [5] R. Davis, B.G. Buchanan, and E. Shortcliff, "Production Rules as a Representation for a Knowledge Based Consultation Program",Artificial Intelligence, 1977, vol. 8(1), pp.15-45. [6] S. Gallant, "Connectionist Expert System", Communications of the ACM, 1988, vol. 31 (2), pp. 152-169. [7] S. Sestito, and T. Dillon, "Automated Knowledge Acquisition",Prentice Hall, Australia, 1994. [8] K. Saito and R. Nakano, "Medical Diagnostic Expert System Based on PDP Model", IEEE International Conference on Neural Networks (San Diego CA), 1988, vol. 1, pp. 255-262. [9] S. Sestito and T. Dillon, "Automated Knowledge Acquisition of Rules With Continuously Valued Attributes", 12th International Conference on Expert Systems and their Applications(AVIGNON'92), Avignon -France, 1992, pp. 645-656. [10] M.W. Craven, and J.W. Shavlik, "Using Sampling and Queries to Extract Rules From Trained Neural Networks", Proceedings of the 11th International Conference on Machine learning, NJ, 1994,pp.37-45. [11] L.M Fu. "Rule Learning by Searching on Adapted Net" Proceedings of the Ninth National Conference on Artificial Intelligence,Anaheim CA,1991, pp 590-595. [12] G. Towell, and J. Shavlik. "The Extraction of Refined Rules From Knowledge Based Neural Networks", Machine Learning, 1993, vol. 131, pp.71-101. [13] M.C. Mozer, C. McMillan, and P. Smolensky, "The Connectionist Scientist Game: Rule Extraction and Refinement in a NeuralNetwork", Proc of the 13th Annual Conference of the Cognitive Science Society, Hillsdale NJ, 1991. [14] V. Tresp, J. Hollatz, and S. Ahmad, "Network Structuring and Training Using Rule-Based Knowledge", Advances in Neural Information Processing, 1993, vol. 5, pp. 871-878. [15] S. Thrun, "Extracting Provably Correct Rules From Artificial Neural Networks", Technical Report IAI-TR-93-5, Institut fur Informatik III Universitaet Bonn, 1994 [16] M.W. Craven, and J.W. Shavlik, "Extracting Tree-Structured Representation of Trained Networks", Advances in Neural Information Processing Systems, MIT Press, Cambridge, 1996, vol. 8, pp.24-30. [17] A. Tickle, A, M. Orlowski, M, J. Diederich, "DEDEC: A Methodology for Extracting Rules from Trained Artificial Neural Networks. "In: Andrews, R.; Diederich, J. (Eds.): Rules and Networks. Brisbane, Qld.: QUT Publication 1996, 90-102. [18] H. Nunez, C. Angulo, and A.Catala, "Rule-extraction from Support Vector Machines", Proceedings of European Symposium on Artificial Neural Networks, Burges, 2002, ISBN 2- 930307-02-1, pp.107-112. [19] N. Barakat, and J. Diederich, "Learning-based rule-extraction from support vector machines", The 14th International Conference on Computer Theory and applications ICCTA'2004, Alexandria, Egypt, Sept, 28-30, 2004. [20] http://www.rulequest.com/
Q-Index Code E1

 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in Thomson Reuters Web of Science Article
Google Scholar Search Google Scholar
Created: Thu, 07 Apr 2005, 10:00:00 EST by Joachim Diederich on behalf of Centre for On-Line Health