Finding small high performance subsets of induced rule sets: Extended summary

Models consisting of decision rules ? such as those produced by methods from Pawlak?s rough set theory ? generally have a white-box nature, but in practice induced models are too large to be inspected. Here, we investigate methods for simplifying complex models while retaining predictive performance. The approach taken is rule filtering, i.e. post-pruning of complete rules. Two methods for finding high-performance subsets of a set of rules are investigated. One method is to use a genetic algorithm to search the space of subsets. Another method is to create an ordering of a rule set by sorting the rules according to a quality measure for individual rules. A rule set with a particular cardinality and expected good predictive performance can then be constructed by taking the first rules in the ordering. Algorithms for the two methods have been implemented in the ROS E T TA system. Predictive performance is estimated using ROC analysis, and compared using statistical hypothesis testing. Ten different formulae from the literature that can be used to define rule quality are compared. Experiments on real-world data show that models often can be dramatically simplified without significant performance loss.