Clinical Performance Evaluation of the Artificial Intelligence (AI)/ Machine Learning (ML) Technologies Utilized by the Origin Medical EXAM ASSISTANT

Study Brief:

A multicenter, prospective observational study shall be conducted for the performance assessment and validation of the AI/ML technologies used in OMEA for the automated assessment of the first-trimester standard fetal ultrasound examinations. A prospective dataset of at least n=289 fetal ultrasound examinations shall be collected from pregnant participants with 11 weeks 0 days to 13 weeks 6 days weeks of gestational age (first trimester).

Study Objectives:

This study aims to evaluate the performance of the Artificial Intelligence (AI) / Machine Learning (ML) technologies utilized in OMEA for the:

1. Automated detection of standard diagnostic views in accordance with practice guidelines;

2. Automated verification of quality criteria required for the interpretation of diagnostic views in accordance with practice guidelines;

   Note: Quality criteria can pertain to the following:

   1. Presence/Absence of anatomical landmarks/structures identified within the diagnostic views detected

   2. Verification of imaging parameters (e.g., magnification)

   3. Verification of clinical features (e.g., orientation of the fetus)

3. Automated caliper placements to obtain measurements in accordance with practice guidelines;

Compliance with HIPAA Guidelines:

All data obtained will be de-identified according to the Health Insurance Portability and Accountability Act (HIPAA) guidelines. The sponsor will be responsible for the storage, management, and security of the de-identified data collected. To protect patient privacy, all data collected for the study undergoes de-identification, ensuring the removal of any identifiable patient information. Each data entry is assigned a unique patient number, which serves as the sole identifier for the study. The link between patient numbers and patient identifiers is securely maintained and accessible only to the principal investigator (PI) and research staff at the study site location. This link is strictly confidential and is not shared with other individuals involved in the study. Its purpose is solely for the site's reference, enabling follow-up with medical records if required. By implementing these measures, the study maintains a high level of confidentiality, safeguarding patient identities while allowing for essential record-keeping and potential future reference.

Sample Size Considerations:

Approximately 500 participants will be recruited for the study, the details of which will be captured in a statistical analysis plan that will be submitted to the FDA.

Study Design and Workflow:

The data for the study is collected in line with the Data Collection Plan and the predefined inclusion and exclusion criteria. The ARDMS performing the routine first-trimester ultrasound scan will be trained on the Image Acquisition Protocol and the Maternal Fetal Medicine (MFM)/reading physicians performing clinical benchmarking will be guided through the Reading Physician Training Manual for ensuring standardized data capture and clinical benchmarking processes for evaluating the standalone performance of the AI/ML technologies used in OMEA. All the above mentioned documents will be submitted to the FDA as part of the premarket submission review.

Phase 1: Data Capture:

At each study site, informed consent will be provided and obtained from eligible participants, and the following information will be collected.

Patient Details:

1. Fetal gestational age

2. Maternal age

3. Maternal BMI

4. Race/Ethnicity

5. Confirmation of diagnosis of fetal anomaly/syndrome prior to the study exam and post the study exam

Site Details:

1. Site location

2. Sonographer name

3. Ultrasound scanner manufacturer and series

Images and cines captured on the ultrasound machine (IUS): Registered diagnostic medical sonographers shall conduct routine first-trimester scans as per the Image Acquisition Protocol.

Images and cines captured through the capture card (ICC): A screen capture/recording of the entire exam performed by the sonographer as per the Image Acquisition Protocol will be obtained, and the images/cines required for the study that correspond to IUS will be obtained. The independent research coordinator from Origin Medical for the study will review the screen recording and identify the frame/cine for each diagnostic view (ICC) that corresponds to IUS based time stamps.

An independent quality reviewer from Origin Medical will verify whether the corresponding pair (i.e., captured on an ultrasound machine vs. obtained through screen recording) of frames/cine for each diagnostic view has been extracted or not.

Phase 2: Clinical Benchmarking and Statistical Analysis

All images/cines (IUS) from all patient exams that meet the study eligibility criteria will be pooled and randomized to prepare the ground truth by an independent Reading Panel (MFM physicians).

AI/ML technologies of OMEA interpretation of ICC The frozen AL/ML technologies used in OMEA shall interpret the images/cines (ICC).

For the sake of clarity, the ICC refers to the images/cines extracted from screen recordings using a capture card that correspond to the same images/cines as obtained by the ARDMS on the ultrasound machine.

The following tasks shall be performed by the AI/ML technologies on ICC:

1. Automated detection of standard diagnostic views;

2. Automated verification of quality criteria required for interpretation of diagnostic views ;

3. Automated caliper placements to obtain fetal measurements;

The performance of the AI/ML technologies used in the OMEA (on all ICC images/ cines that meet the study eligibility criteria) shall be compared against the ground truth for statistical analysis, i.e., against the majority consensus obtained from the Reading Panel for detection of diagnostic views, verification of quality criteria, performing fetal biometry measurements and ACEP grading of the images/cines.