Background
Recent years have seen a growing interest in systems and technologies that rely on vision and wireless systems to provide certain capabilities. Examples include vision-aided wireless communications and wireless-vision activity recognition. However, these technologies have a common shortcoming; they do not directly address the association issue between the objects appearing in the visual data and the received radio signals. Hence, this is expected to be a major limitation to their effectiveness in real-world settings.
Invention Description
Researchers at Arizona State University have developed a system for multimodal learning for wireless communication and sensing. This technology enables (a) radio beam blockage prediction in 5G/6G systems using camera data, (b) identification of cars and people in a surveillance camera feed using joint visual and wireless data processing, and (c) efficient communication for autonomous vehicles relying on camera and wireless data. More specifically, given some visual data, algorithmic prediction can detect which object is emitting which type of signal. These capabilities allow more seamless adaptation of the communication system operation to its changing environment.
Potential Applications
• Millimeter wave 5G communication systems
• Terahertz 6G communications systems
• Smart surveillance systems
• Autonomous vehicles
• Electronic warfare
Benefits and Advantages
• Enables communication systems to efficiently adapt to sensed changes in the environment
• Automates transmitter identification in visual scenes
• Allows for dynamic reallocation of resources to ensure high reliability and low latency in wireless networks