CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing
Background
High-frame-rate cameras are capable of capturing videos at frame rates over 100 frames per second (fps). These devices were originally developed to characterize rapid events. Some high-frame-rate cameras can record high resolution still images of ephemeral events such as a supersonic flying bullet or an exploding balloon with negligible motion blur and image distortion artifacts. However, due to the complex sensor hardware designed for high sampling frequency, these types of equipment are extremely expensive. Furthermore, the high transmission bandwidth and the large storage space associated with the high frame rate challenges the manufacture of affordable consumer devices. Therefore, a method which addresses the large storage space requirements while maintaining speed and quality is required.
Invention Description
Researchers at ASU have designed an enhanced recurrent convolutional neural network (RCNN) to solve the aforementioned issues. RCNN’s have shown astonishingly good performance for video recognition and description. However, simultaneously improving compression ratio and preserving visual details for high-fidelity reconstruction of ephemeral images is a challenging task. Therefore, the researchers have developed an elegant RCNN algorithm to extract spatial-temporal features, including background, object details, and motions, to significantly improve the compression ratio and recovery quality trade-off. These improvements will relax the memory requirements as well as increase the applicability of high speed camera technology.
Potential Applications
- Reconnaissance
- Recreational Video Cameras
Benefits and Advantages
- Compression and Decompression- The Novel RCNN allows for the efficient compression and decompression of images captured by high-frame-rate cameras. This is a huge advantage considering most high-frame-rate cameras do not compress images at all. Essentially, this technology will allow users to take more images without sacrificing the quality of those images.