Date Approved

9-5-2024

Embargo Period

9-6-2024

Document Type

Thesis

Degree Name

Master of Science (M.S.)

Department

Electrical and Computer Engineering

College

Henry M. Rowan College of Engineering

Advisor

Dwaipayan Chakraborty, Ph.D.

Committee Member 1

John Schmalzel, Ph.D.

Committee Member 2

Ying Tang, Ph.D.

Keywords

low-power computing systems, high-speed target detection, physiological monitoring

Subject(s)

Wearable technology; Wireless sensor networks; Internet of Things

Disciplines

Electrical and Computer Engineering | Engineering

Abstract

Threat detection and physiological monitoring of soldiers from fused sensor data collected in real time is currently limited to running deep neural networks with substantial computing needs. The lack of data acquisition from sensor readings and efficient detection of novel enemy signatures motivates the need for a low-power, low-cost, wireless multisensor fusion computing system. We propose the current trends in Internet of Things to deploy a chargeable, wireless multi-channel acquisition system that can be interfaced to a high speed, Single Board Computer (SBC) such as the NVIDIA Jetson Orin capable of running object detection models, such as YOLOv7-tiny to enable high speed target detection, and monitoring of soldier’s health and weapon states at a low-cost, and low-power. Target detection and data fusion was achieved at ∼60 FPS with a YOLOv7-tiny model trained on a custom drone dataset with a NVIDIA Jetson Orin equipped with a USB camera, and connected to a custom wireless Wearable Multi-Channel Physiological Device (WMCPD) centered around the ESP32 and interfaced to two physiological sensors: Pulse Oximeter, and GSR sensor that synchronized heart rate, Oxygen saturation (SpO2), and Galvanic Skin Response (GSR). Based on the power metrics required for the ESP32 and the interfaced sensors, a multi-channel acquisition system was designed that features a USB-C battery charging interface capable of charging a Li-Ion (500 mAh) to power the system. This Low Power, wearable multi-channel, physiological device (WMCPD) can transmit the sensitive fused, physiological data to wireless devices such as the Edge device, or the HoloLens 2, to not only keep track of the soldier’s physiological state, but to further exemplify efficient processing at the edge and capture soldier feedback from real-time sensitive data collection, and its affect on integrating to the external devices in the mixed reality setting (TGGSE) such as the Orin or Hololens. Additionally, a custom weapon sensing device was developed that monitors the real-time ammunition, and recoil of a soldier’s gun, and was designed to be serially compatible with military devices and vehicles over MilCAN.

Download

Share

COinS