Mobile capture of volumetric point clouds enables spatial information in 3D to be streamed to multiple parties. A point cloud is simply a set of 3D points capable of representing volumes of captured data. With the rapid development of both depth sensor hardware and visual computing software, point cloud data can be captured at high precision improving the utility across a variety of applications. However, point clouds are expensive to stream and process, especially compared to the limited network bandwidth and power of mobile devices. And, existing solutions for volumetric point cloud capture are too bandwidth-intensive and/or compute-intensive to support real-time streaming operation for dense, high-quality volumetric data.
Researchers at Arizona State University (ASU) have developed an edge-assisted point cloud live-capture and streaming framework to enable real-time high precision point cloud visualization on mobile client devices. The system uses intelligent spatial agents capable of executing spatiotemporal filters on the data, dynamically adjusting the resolution and representation of the data across the streaming pipeline. Additionally, the system’s orchestrator components are responsible for distributing computational processes across different compute nodes and issuing workload balance mitigation techniques.
Related publication: Adaptive 5G systems for interactive volumetric sports analysis in augmented reality
Potential Applications:
- Real-time streaming of point clouds on mobile devices for:
- Live data capture
- Immersive sports broadcasting
- Augmented education and coaching
- Human-scene interactions
Benefits and Advantages:
- ASU platform enables:
- Dense point cloud to be captured and streamed to various mobile devices across fluctuating networks in a live-setting
- Real-time interactivity for users to interact with the point clouds (i.e., users are able to define what regions to be captured at a specific resolution)
- Balanced tradeoff among visual quality, power consumption, and performance