The common use of decision trees (DTs) as a data mining method is discussed in this paper. Olaru and Wehenkel describe a fuzzy DT technique, soft decision tree (SDT), which combines not only tree growth and pruning, but also refitting and backfitting (in order to improve generalization ability).

SDT is a variant of classical DT inductive learning, which utilizes fuzzy set theory. The elasticity of fuzzy set formalism is said to improve classification instability resulting from training data that exhibits high variance. Moreover, the greatly reduced model variance of SDT leads to improved prediction accuracy.

The performance of SDT is compared initially with Quinlan’s C4.5, Breiman’s CART, and Wehenkel’s own DT method, called ULG. This performance comparison utilizes three data sets from the University of California, Irvine (UCI) repository (omib, twonorm, and waveform) along dimensions of accuracy, model complexity, central processing unit (CPU) time, choice of stopping parameter, global variance, and parameter variance/bias. Further experiments were conducted using 11 more UCI data sets. SDT performance (with and without backfitting) is once again compared with that of C4.5, CART, and ULG. The authors conclude that “overall, for small datasets, backfitting may not be necessary, since with refitting, which is faster, we obtain already significantly better results than C4.5, CART, or ULG.”

Olaru and Wehenkel conclude their paper with several suggestions as to how this initial work on SDT could be expanded.