This page features products that are the result of the UA-1 DECISIVE: Development and Execution of Comprehensive and Integrated Systematic Intelligent Vehicle Evaluations project by the University of Massachusetts Lowell. This project is sponsored by the Department of the Army, U.S. Army Combat Capabilities Development Command Soldier Center (W911QY-18-2-0006 and W911QY-20-2-0005).
DECISIVE Testing Handbooks
The Phase 2 DECISIVE handbook includes 9 test methods developed and results from performance benchmarking for evaluating small unmanned aerial systems (sUAS) operating in dense urban environments:
Test Methods
- Beyond-Line-of-Sight (BLOS) / Non-Line-of-Sight (NLOS) Outdoor Communications
- Automated Return to Home (RTH)
- Station Keeping In Wind
- Land and Takeoff In Wind
- Perched Field of Regard
- Exterior Building Clearing
- Fly Through Confined Outdoor Spaces
- Outdoor 2D Mapping Accuracy
- Outdoor 3D Mapping Accuracy
The test methods and benchmarking performance data handbook for dense urban environments is available on arXiv:
- Adam Norton, Brendan Donoghue, and Peter Gavriel. UA-1 PH2 DECISIVE Testing Handbook: Test Methods and Benchmarking Performance Results for sUAS in Dense Urban Environments. arXiv preprint arXiv:2502.01648, February 2025.
The Phase 1 DECISIVE handbook outlines 20 test methods developed for evaluating small unmanned aerial systems (sUAS) operating in subterranean and constrained indoor environments:
Test Methods
- Non-Line-of-Sight (NLOS) Communications
- Non-Line-of-Sight (NLOS) Video Latency
- Interference Reaction
- Runtime Endurance
- Takeoff and Land/Perch
- Room Clearing
- Indoor Noise Level
- Logistics Characterization
- Operator Control Unit (OCU) Characterization
- Obstacle Avoidance and Collision Resilience
- Position and Traversal Accuracy
- Navigation Through Apertures
- Navigation Through Confined Spaces
- Indoor Mapping Resolution
- Indoor Mapping Accuracy
- Non-Contextual Autonomy Ranking
- Contextual Autonomy Ranking
- Characterizing Factors of Trust
- Interface-Afforded Attention Allocation
- Situation Awareness (SA) Survey Comparison
The test methods handbook and benchmarking data report for subterranean and constrained indoor environments are available on arXiv:
- Adam Norton, Reza Ahmadzadeh, Kshitij Jerath, Paul Robinette, Jay Weitzen, Thanuka Wickramarathne, Holly Yanco, Minseop Choi, Ryan Donald, Brendan Donoghue, Christian Dumas, Peter Gavriel, Alden Giedraitis, Brendan Hertel, Jack Houle, Nathan Letteri, Edwin Meriaux, Zahra Rezaei Khavas, Rakshith Singh, Gregg Willcox, and Naye Yoni. "DECISIVE Test Methods Handbook: Test Methods for Evaluating sUAS in Subterranean and Constrained Indoor Environments, Version 1.1." arXiv preprint arXiv:2211.01801, November 2022.
- Adam Norton, Reza Ahmadzadeh, Kshitij Jerath, Paul Robinette, Jay Weitzen, Thanuka Wickramarathne, Holly Yanco, Minseop Choi, Ryan Donald, Brendan Donoghue, Christian Dumas, Peter Gavriel, Alden Giedraitis, Brendan Hertel, Jack Houle, Nathan Letteri, Edwin Meriaux, Zahra Rezaei Khavas, Rakshith Singh, Gregg Willcox, and Naye Yoni. "DECISIVE Benchmarking Data Report: sUAS Performance Results from Phase I." arXiv preprint arXiv:2301.07853, January 2023.
Publications
- Zahra Rezaei Khavas, Edwin Meriaux, Amin Majdi, and Paul Robinette. "Validating Drone Trust Testing in Navigation Deviation Cases in Simulation." arXiv preprint arXiv:2310.04794, October 2023.
- Edwin Meriaux, Jay Weitzen, and Adam Norton. "Simulation of the Effect of Correlated Packet Loss for sUAS Platforms Operating in Non-Line-of-Sight Indoor Environments." Drones, Volume 7, Issue 7, July 2023.
- Zahra Rezaei Khavas, Amin Majdi, S. Reza Ahmadzadeh, and Paul Robinette. "Human Trust After Drone Failure: Study of the Effects of Drone Type and Failure Type on Human-Drone Trust." In Proceedings of the 20th International Conference on Ubiquitous Robots (UR) 2023, pp. 685-692. IEEE, June 2023.
- Edwin Meriaux, Jay Weitzen, Naye Yoni, and Adam Norton. "Test Methodologies for Evaluating the Effectiveness of sUAS Communication Links for Operation in Indoor and Subterranean Environments." ASTM Journal of Testing and Evaluation, Volume 51, Issue 6, June 2023.
- Minseop Choi, and Thanuka L. Wickramarathne. "On the Use of Real-Time Multi-Object Detection and Classification for Automated Assessment of Operator Visual Attention Allocation." In Proceedings of the 26th International Conference on Information Fusion (FUSION) 2023, pp. 1-7. IEEE, June 2023.
- Ryan Donald, Peter Gavriel, Adam Norton, and S. Reza Ahmadzadeh. "Contextual Autonomy Evaluation of Unmanned Aerial Vehicles in Subterranean Environments." In Proceedings of the 9th International Conference on Automation, Robotics and Applications (ICARA) 2023, February 2023.
- Edwin Meriaux, and Kshitij Jerath. "Evaluation of Navigation and Trajectory-following Capabilities of Small Unmanned Aerial Systems." In Proceedings of the IEEE International Symposium on Technologies for Homeland Security (HST), IEEE, November 2022.
- Minseop Choi, John Houle, and Thanuka L. Wickramarathne. "On the Development of Quantitative Operator Situational Awareness Assessment Methods for Small-Scale Unmanned Aircraft Systems." In Proceedings of the 25th International Conference on Information Fusion (FUSION) 2022, pp. 1-8. IEEE, July 2022.
- Brendan Hertel, Ryan Donald, Christian Dumas, and S. Reza Ahmadzadeh. "Methods for Combining and Representing Non-Contextual Autonomy Scores for Unmanned Aerial Systems." In Proceedings of the 8th International Conference on Automation, Robotics and Applications (ICARA) 2022, pp. 135-139. IEEE, February 2022.
- Adam Norton, Peter Gavriel, Brendan Donoghue, and Holly Yanco. "Test Methods to Evaluate Mapping Capabilities of Small Unmanned Aerial Systems in Constrained Indoor and Subterranean Environments." In Proceedings of the IEEE International Symposium on Technologies for Homeland Security (HST) 2021, IEEE, November 2021.
- Minseop Choi, John Houle, Nathan Letteri, and Thanuka L. Wickramarathne. "On The Use of Small-Scale Unmanned Autonomous Systems for Decision-Support in Subterranean Environments: The Case of Operator Situational Awareness Assessment." In Proceedings of the IEEE International Symposium on Technologies for Homeland Security (HST) 2021, IEEE, November 2021.
Advancing and Evaluating UAS for Defense
The capstone event for the DECISIVE project, "Advancing and Evaluating UAS for Defense," was held on October 12, 2022. The event feature research briefs on the UA-1 and UA-2 projects funded by the UMass Lowell (UML)/DEVCOM HEROES program. All presentations given at the event are available for download via a publicly accessible Google Drive folder. Below is a list of the presentations given:
- HEROES Program Overview, Christopher Drew
- UA-1 Project Review: DECISIVE: Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations, Adam Norton
- UA-2 Project Review: Drone-Based Wireless Sensor Network for Multimodal Underground Situational Awareness, Xuejun Lu
- UMass Lowell UAS Project Transition, Adam Norton
- Test Methodologies to Evaluate UAS Communication and the Effects of Packet Loss in NLOS Indoor Environments, Jay Weitzen
- Embedding IR Sensors to Increase UAS Situational Awareness, Xuejun Lu
- Artificial Intelligence for Image Recognition During UAS Operations, Yuanchang Xie
- Methods for Combining and Representing Autonomy Ranking Scores of UAS, Reza Ahmadzadeh
- Characterizing and Evaluating Field Readiness of UAS for Subterranean and Indoor Environments, Adam Norton
- Evaluation of Indoor Navigation and Obstacle Avoidance and Resilience Capabilities of UAS, Kshitij Jerath
- Field Testing of UAS Navigation Through Confined Spaces and Apertures, Adam Norton
- Test Methods to Evaluate Indoor and Subterranean Mapping Capabilities of UAS, Adam Norton
- Trustworthy and Safe Human-UAS Interaction for Subterranean Mapping, Paul Robinette
- Assessing Operator Situational Awareness of UAS for Decision-Support in Subterranean Environments, Thanuka Wickramarathne