Automatic feature extraction and selection for machine learning based intrusion detection

As the advances in mobile technologies and IoT-enabled devices have been integrated into our daily lives, significant increases in wireless network traffic generate a large scale of high dimensional network log data. This has led to challenges in security of Wi-Fi network systems that have to analyze such complex big data for intrusion detection. Many Wi-Fi network systems commonly employee machine learning based Intrusion Detection Systems (IDS). Such IDS usually adopt supervised methods that heavily depend on observations of human experts for feature extraction, feature selection, and labeling processes of training data for classification. In this study, using the recently collected Aegean Wi-Fi Intrusion Dataset (AWID) which contains real traces of different network attacks types, we propose an unsupervised approach with automatic feature extraction and selection process to replace human intervention and manual labeling process for analyzing a large scale high dimensional data to improve the prediction accuracy of classification to detect 3 most common network attack types - Injection, Flooding, and Impersonate attacks in an IDS with a large scale of high dimensional data. The experiment results showed the effectiveness of our approach for feature extraction and selection. The quality of the selected features and the accuracy of intrusion detection of the three attack types are compared and analyzed.