A Systematic Comparative Evaluation of Supervised Machine Learning Algorithms for High-Dimensional Engineering Datasets

Priyanka Sharma, Dr. Anirudh Kulkarni
Department of Emerging Technologies

Mahatma Gandhi Institute of Technology
Gandipet, Hyderabad

Abstract

In modern engineering research and applications, datasets characterized by a large number of features compared to samples have become widespread. These datasets are common in fields like materials science, structural health monitoring, sensor networks, and systems optimization. Supervised machine learning (ML) is a prevalent method for creating predictive models from such intricate data. Nonetheless, the “curse of dimensionality” presents notable obstacles, such as overfitting, computational inefficiency, and reduced generalization. This paper offers a thorough and systematic comparative analysis of different supervised ML algorithms applied to high-dimensional engineering datasets. The analysis includes traditional algorithms (e.g., k-Nearest Neighbors, Support Vector Machines, Decision Trees), regularized linear models (e.g., LASSO, Ridge Regression), ensemble methods (e.g., Random Forests, Gradient Boosting), and advanced kernel and deep learning techniques. By employing rigorous performance metrics, robust validation methods, and interpretability considerations, this research highlights the strengths, weaknesses, and practical guidelines for choosing algorithms in engineering scenarios. Experimental findings reveal that algorithm performance significantly depends on dataset characteristics, with ensemble and regularization-based models generally excelling in accuracy and robustness. The study concludes with suggestions for practitioners and researchers dealing with high-dimensional engineering data.

In this evaluation, feature selection methods are utilized to address dimensionality challenges and improve the interpretability of models. To ensure unbiased performance estimation and avoid data leakage, both cross-validation and nested validation frameworks are applied. The study’s insights are intended to inform the creation of more efficient ML pipelines that address the specific challenges posed by high-dimensional engineering datasets. Additionally, incorporating feature selection techniques like recursive feature elimination and principal component analysis greatly enhances model efficiency and clarity. The comparative analysis sheds light on the balance between model complexity and computational expense, stressing the significance of selecting algorithms based on context. Ultimately, the results highlight the necessity for adaptive ML frameworks capable of dynamically adjusting model parameters to suit various high-dimensional engineering datasets.

 Keywords

Data with many dimensions, Supervised learning algorithms, Categorization and prediction, Choosing relevant features, Assessing model performance, Combining multiple models, Engineering data collections, Model overfitting, Challenges of high dimensionality