Image analysis techniques are invaluable in helping physicians better diagnose and treat diseases and expand the utility of medical imaging. Self-supervised learning, in particular, is one of the most practical paradigms in deep learning for medical image analysis. Self-supervised learning methods are characterized by training deep models directly from unannotated images, which removes the need for manual labeling. Those models are used as a starting point for training a target model for a specific application. However, the mainstream self-supervised methods do not utilize the domain knowledge from medical imaging. Medical images render anatomical structures, which are associated with rich semantics about the human body. That being said, anatomical structures offer unique potential for developing powerful models for medical applications, but they are not exploited by current self-supervised methods.
Researchers at Arizona State University have developed a self-supervised learning framework that enables the capture of semantics-enriched representation from unlabeled medical image data, resulting in a set of powerful pre-trained models, called Semantic Genesis. These models learn visual representation by self-discovery, self-classification, and self-restoration of the anatomy underneath medical images. Semantic Genesis yields a generic and transferable visual representation that can improve the performance of various medical tasks across diseases, organs, and imaging modalities. Another unique property of Semantic Genesis is that it can readily serve as an add-on to dramatically boost the performance of existing self-supervised learning approaches.
This Semantic Genesis consistently surpasses not only state-of-the-art self-supervised learning counterparts but also existing fully supervised pre-trained 3D models in all four target 3D applications; thus, it can serve as a primary source of transfer learning for 3D medical imaging applications.
Potential Applications
• Medical imaging analyses across many different diseases (e.g. nodule, embolism, tumor, etc.) in any organ and using any imaging modality (e.g. CT, X-Ray, MRI, etc.)
o Classification tasks – e.g. determine healthy vs diseased tissues
o Segmentation tasks – e.g. identify region of interest vs background
Benefits and Advantages
• Self-supervised – requires no expert annotation in pre-training
• Generic – can yield diverse target applications across diseases, organs, and modalities
• Enables the capture of semantics-enriched representation from unlabeled medical image data
• Discovers a set of anatomical patterns, associated with semantically meaningful labels from unlabeled medical images
• Learns semantics-enriched representation via self-classification of recurrent anatomical patterns
• Learns different sets of visual representation via self-restoration of recurrent anatomical patterns
• Serves as an add-on to enrich existing self-supervised methods, boosting target task performance dramatically
• Outperforms learning 3D models from scratch and other existing 3D pre-trained models
• Accelerates the training process of deep learning models
• Reduces annotation costs dramatically
• Surpasses any 2D approaches
For more information about this opportunity, please see
Haghighi et al – MICCAI – 2020
For more information about the inventor(s) and their research, please see
Dr. Liang’s departmental webpage