12/11/2024
By Danielle Fretwell

The Francis College of Engineering, Department of Mechanical Engineering, invites you to attend a Doctoral Dissertation Proposal defense by Elyas Irankhah on: Assessing Robust Machine Learning Performance Metrics to Classify Traumatic Brain Injury Using Baselines from Real User Data

Candidate Name: Elyas Irankhah
Degree: Doctoral
Defense Date: Friday, December 13, 2024
Time: From 10 a.m. - 12 p.m.
Location: Perry Hall, Room 215

Committee:
Advisor: Kelilah Wolkowicz, Assistant Professor, Department of Mechanical and Industrial Engineering, UMass Lowell

Committee Members
David Claudio, Associate Professor, Department of Mechanical and Industrial Engineering, UMass Lowell
Sashank Narain, Assistant Professor, Miner School of Computer & Information Sciences, UMass Lowell
Alessandro Sabato, Assistant Professor, Department of Mechanical and Industrial Engineering, UMass Lowell

Brief Abstract:
Machine learning (ML) opens new opportunities for advancing the classification of traumatic brain injury (TBI). Effectively classifying TBI cases remains a challenge due to the complexity of cognitive task data and the limitations of traditional computational approaches. There has been limited work performed that addresses the use of existing ML algorithms using real-world cognitive task data to accurately classify TBI cases and differentiate levels of severity. Motivated by this need, this work rigorously evaluates the performance of three ML algorithms: Decision Trees, Logistic Regression, and K-Nearest Neighbors (KNN) to accurately assess their robustness in TBI classification. In our preliminary work, we used a real-world task-based dataset from a separate study, which included data from 54 participants: 31 with orthopedic injuries (OI) and 24 with TBI. This dataset offers valuable cognitive task performance metrics to evaluate classification models, including inhibitory control, working memory and cognitive flexibility. Virtual reality (VR) also played a role by providing immersive environments to collect task-based data which enhanced the dataset’s quality and robustness by providing controlled, standardized task conditions for data collection. The proposed analysis focuses on key performance metrics to assess classification effectiveness, including accuracy, precision, recall, F1-score, and Area Under the Curve (AUC). The Decision Tree model preliminary results achieved an average accuracy of 70.4%, precision of 100%, recall of 50%, and an F1-score of 66.7%, with an AUC of 0.65. KNN and Logistic Regression demonstrated similar results, with average accuracies of 64.8% and 63.0%, respectively. Both models achieved a precision of 100%, recall of 50%, F1-scores of 66.7%, and AUCs of 0.62 (KNN) and 0.60 (Logistic Regression). These models were selected for their simplicity, interpretability, and suitability for small to medium-sized datasets. The proposed goal of this work is to assess the existing ML algorithms for their ability to classify TBI cases. The techniques explored are not intended for diagnostic use. Preliminary results indicate limitations in their performance and emphasize the need for clearly defined performance metrics and detailed statistical analysis. These findings provide a foundation for further research into improving ML techniques for classifying different severity of TBI.