Faruk Bulut, “Construction And Performance Analysis Of Locally Adaptive Base And Ensemble Learners”, June 10, 2015 (Wednesday)

Computer Engineering Department Seminar #56:

	Title:	Construction And Performance Analysis Of Locally Adaptive Base And Ensemble Learners
	Speaker:	Faruk Bulut, Ph.D.
	Date:	June 10, 2015 (Wednesday)
	Time:	15:30-16:15
	Venue:	Computer Networks Lab (New building, Level B2.)
	Language:	English

Abstract: In this study, construction and performance analysis of locally adaptive base and ensemble learners have been proposed by using Meta and Ensemble Learning techniques. The characteristics and meta-features of the discretized sub regions in a dataset have been analyzed for the purpose of better learning performance. A detailed performance analysis of a local base learner over any type of dataset is firstly performed in order to understand the reasons of both failure and success in classification. Additionally, the discrete sub regions are learned by using the Mixture of Experts model to enhance the overall prediction accuracy. Furthermore, a localized lazy base learner using a dynamic parameter creator mechanism is established to gain better performance. Firstly, prediction of the performance of a local base learner (e.g., decision tree) is proposed by using Meta Learning methods. We have selected some datasets and some extracted geometrical complexity measures from the datasets so as to use in Meta Learning. The extracted features and the real accuracy rates of these classifiers have been accepted as attributes and class labels respectively, and they are placed into the Meta Learning dataset. With this training set, it becomes possible to predict the accuracy of a decision tree on upcoming datasets. Moreover, by using the new meta-learning dataset, a feasible linear regression model has been built for the purpose of predicting the performance of a decision tree classifier. As a consequence, some meaningful reasons have been determined why decision trees outperform or fail on any dataset. Secondly, a new approach in Mixture of Experts using hard clustering techniques is presented for accurate prediction and classification. Mixture of Experts, as one of the popular ensemble methods developed in recent years, is used to have higher prediction performance in classification and regression problems. In this technique, a dataset is divided into sub regions through a soft clustering procedure. An expert for each region is assigned and trained with the corresponding data points. The decisions of the experts are combined by a gate function in order to predict the class label of a query point. In contrast to the traditional Mixture of Experts method, in this study, a dataset is divided into regions by a hard clustering technique and the class prediction method is performed by four different types of proposed gate functions: cooperating, competitive, commensurative, and Borda count. In the experiments, better performances have been obtained with the proposed cooperating gate function due to its mechanism that gives different weights to the experts in the network. Finally, a locally adaptive parameter selection mechanism for nearest neighbor classifiers using clustering methods is suggested for more accuracy. The k Nearest Neighbors classification technique has a worldwide fame due to its simplicity, effectiveness and robustness. As a lazy learner, k Nearest Neighbors used in numerous fields is a versatile algorithm. In this classifier, the k parameter is generally chosen by the user and the optimal k value is found by experiments. The chosen constant k value is used during the whole classification phase. The same k value used for each test sample during the validation step might decrease the overall prediction performance. The optimal k value for each test data point should vary in order to have more accurate predictions. In this study, a dynamic k value selection method for each instance is proposed. This improved classification method employs a simple clustering procedure. In the experiments, more accurate results have been found. The reasons of success have also been understood and presented.