Development of an Artificial Intelligence System for Intelligent Pathological Diagnosis and Therapeutic Effect Prediction Based on Multimodal Data Fusion of Common Tumors and Major Infectious Diseases in the Respiratory System Using Deep Learning Technology.

The main aims are as follows:

1. To establish a medical big data platform for multi-modal information fusion of common tumors and major infectious diseases (lung cancer/pulmonary nodules, tuberculosis, and COVID-19) based on the existing pathological image features and clinical multi-omics information database: The medical big data platform supports the acquisition of the patient's clinical electronic medical records (including routine clinical detection), full view digital section of pathological image data, medical imaging (CT, MRI, ultrasound, nuclear medicine, etc.), multiple omics data (genome, transcriptome, and metabolome, proteomics) omics data, etiology, pathology, and associated graphic data reports and multimodal medical treatment data. We aim to realize the storage, sharing, fusion computing, privacy protection, and security supervision of multi-modal and cross-scale biomedical big data. Our work will open up key business processes and links across regions, across hospitals, between different terminals, between hospitals and doctors, and between departments, so as to promote continuous data accumulation and knowledge precipitation in hospitals and promote medical collaboration.

2. To create a multimodal information fusion database with pathologic features, imaging features, multi-omics (pathologic, genomic, transcriptome, metabolome, proteomics, etc.), and clinical information of patients at different stages of lung cancer/pulmonary nodules, tuberculosis, and COVID-19. The database scale includes multimodal data of at least 600 lung cancer/pulmonary nodules, 200 tuberculosis, and 200 COVID-19 patients. Moreover, there will be more than 10 biomarkers significantly related to the diagnosis and treatment of patients with lung cancer/pulmonary nodules, tuberculosis and COVID-19 were excavated through association analysis, providing parameters for artificial intelligence model construction.

3. We will make use of artificial intelligence technology to create the multi-modal medical big data cross-analysis technology and the above disease individualized accurate diagnosis and curative effect prediction models. In order to solve the three key problems of multi-modal data fusion mining, such as unbalanced, small sample size, and poor interpretability, we will establish an ARTIFICIAL intelligence recognition algorithm for image images and pathological images, and use image processing and deep learning technologies to mine multi-level depth visual features of image data and pathological data. In addition, we will use bioinformatics analysis algorithms to conduct molecular network mining and functional analysis of molecular markers at the level of multiple omics technologies (pathologic, genomic, transcriptome, metabolome, proteome, etc.).