Though the conventional frame-based cameras have advanced in a lot of ways, they suffer from data redundancy and temporal latency. The bio-inspired artificial retinas eliminate the data redundancy by capturing only the change in illumination at each pixel and asynchronously communicating in binary spikes. In this work, we propose a system to achieve the task of human activity recognition based on the event-based camera data. We show that such tasks, which generally need high frame rate sensors for accurate predictions, can be achieved by adapting existing computer vision techniques to the spiking domain. We used event memory surfaces to make the sparse event data compatible with deep convolutional neural networks (CNNs). We leverage upon the recent advances in deep convolutional networks based video analysis and adapt such frameworks onto the neuromorphic domain. We also provide the community with a new dataset consisting of five categories of human activities captured in real-world without any simulations. We achieved an accuracy of94.3% using event memory surfaces on our activity recognition dataset.
The research can be found here paper
Sample Imags