Firefighters risk their own lives to keep everyone else's safety; they deserve a tool to make their jobs faster, easier and safer. For this reason, we developed BrickThru mobile application that helps them detect and locate people in burning buildings. We built the solution around the idea that humans have a unique effect on the signal they interfere with within a specific wi-fi network range. There is a good amount of research that shows how we can apply this concept to achieve Human Detection and Human Activity Recognition (E.g. walking, running, falling, breathing). In this project, we collect Wi-fi Channel State Information (CSI), perform necessary filters to remove noise from the signal and utilize IBM's powerful machine learning tools to predict human existence and track their location in real-time.
When firefighters arrive at fire scenes, they only have a few seconds to make decisions. Sometimes they don't have enough givens, they may not know whether there are people in the flames to rescue or not. They may not know how many people are there or where they are. They carry the fear of not being fast enough through the mission and the guilt of missing anyone after it.
Research surrounding Device-free Wifi-Sensing (DFWS) is becoming more prevalent and showing excellent results in achieving accurate sensing with minimal infrastructure. Wifi signals are everywhere around us and can be used to detect humans and their different activities, like falling, walking, or even breathing. So, utilizing them provides firefighters with an affordable, portable, and convenient way to locate people inside burning buildings. Such live data could help them respond faster, keep safer, and rescue more lives.
Firefighters arrive at fire scenes wearing heavy gears and barely seeing or breathing from all the smoke. They have to make decisions a few seconds after they arrive. The pressure is immense knowing they're responsible for peoples' lives, and there might be people inside relying on them. By closing the data gap to firefighters (using IBM's powerful machine learning tools) in fire scenes, they can do their jobs faster, safer, and better without the extra pressure and guilt. We provide them with live dashboards to track people who need to be rescued within a fire scene and allocate them to the nearest firefighters. Additionally, we help the crew look after one another by tracking their health metrics through Samsung smartwatches and notifying the captain in case something doesn't look right.
Channel state information (CSI) indicates carrier signal strength, amplitude, phase, and signal delay, which vary according to the transmission distance. Such variance reveals the signal scattering, reflection, and power attenuation phenomena within the carrier, and can be used to measure the channel status of the wireless network in Wi-Fi communication. By analyzing and studying the changes in CSI, we can conclude the changes in the physical environment that cause the changes in the channel state and achieve non-contact intelligent sensing. CSI is extremely sensitive to environmental changes caused by large movements (E.g. walking/ running) and subtle movements (E.g. breathing/ chewing).
Previously, mobile phones supported measuring the CSI; but they don't anymore for security reasons and the numerous applications of CSI. For this reason, the below existing tools use ESP32 to measure the needed CSI values:
2.3) CSI Applications Found Here
Select high-sensitivity sub-carrier combinations and signals from non-line-of-sight path directions in different multipath propagation environments, thereby enhancing the sensitivity of passive person detection and expanding the detection range. This method can form "no blind spot" intruder detection in security applications. The multipath propagation characteristics of wireless signals indoors make wireless perception have natural advantages in sensing range and directionality.
You can learn from the RSSI method and use CSI as a more informative fingerprint (including information on signal amplitude and phase on multiple subcarriers), or rely on a frequency selective attenuation model for more accurate ranging.
Use CSI's high sensitivity to environmental changes to recognize human movements, gestures, breathing and other small movements and daily activities.
CSI can be used to detect fires as studied and implemented here. However, in BrickThru we are currently working on reversing that research methodolgy in the filtering process so that we can completely exclude fire in a fire scene.
- https://dl.acm.org/doi/abs/10.1145/3310194
- https://www.hindawi.com/journals/misy/2020/3185416/
- https://github.com/espressif/esp-csi
- https://www.huangyongzhi.com.cn/data/WiFire.pdf
This video demonstrates what we our progress and the result of running this repo so far.
In the setup we use a Samsung Android Phone connected to an esp32 development board using an OTG cable. We utilize a Router or an Access Point on the other side. The router and esp32 board exchange packets in a regular manner and the esp32 extracts CSI data during that process.
CSI data can be collected directly on a smartphone, however, most companies do not give developers access to it. We can partner with Samsung to collect CSI from mobile directly, and in this case we can get rid of the ESP32 and the OTG cable. The Firefighter would only need his phone, and such a feature would exclusvely be available on Samsung devices.
The Router could be replaced with ESP32 with strong antenna or a Wi-Fi directed antenna.
CSI data collected is pre-processed by the Android phone. Pre-processing include filtering, de-noising and running detection helper algorithms to trigger the real-time machine learning predictions. We train the model using a dataset in different environments and using different human activities which makes it able to accurately detect human presence.
- To run the app you can follow the steps in the BrickThru react-native directory.
- In order to prepare and flash the ESP32, you can follow the steps of the ESP CSI Toolkit we used.
- Machine Learning
- Object Storage
- Watson Studio
- AutoAI Experiment
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
-Ahmed Hisham -Salma Medhat
- We used this library this to collect CSI.