Participants and Setting This will be a 12-month, within-subject design, single center
study of 300 children 6 to 24 months of age presenting to their primary care providers at
Children's Community Pediatrics offices with upper respiratory symptoms. Exclusion
criteria are children with tympanostomy tubes or purulent otorrhea, absence of upper
respiratory symptoms, or who are currently taking antimicrobials.
Design and Outcomes
Using a double blind, within-subject design, each child's ear will be assessed by the AI
app and a standard clinical exam. The primary outcome measure will be antimicrobial
prescription rates derived from 150 paired images that each have an AI app and clinician
diagnosis. We will secondarily assess acute otitis media (AOM) diagnosis rates. If the AI
app diagnoses AOM it will always prescribe an antimicrobial. Secondary outcomes are
described below:
Implementation challenges reflected by the proportion of uninterpretable images
(cerumen, uncooperative patient, poor technique).
Symptom resolution and side effects of antimicrobial use for 10 days after
enrollment, regardless of whether children are diagnosed with AOM or prescribed
antimicrobials. We will monitor symptoms of AOM daily using a validated symptom
scale entered every evening by parents in electronic diaries which we have used in
many other studies. If any participant increases their symptom score by >20% at any
time, we will contact them and offer a visit. Rates of protocol-defined diarrhea and
diaper rash which are the most common side effects of antimicrobial use in this age
group will be assessed.
AOM recurrences by reviewing the medical record for 3 months following enrollment.
Sample Size Calculation Using paired observations (AI app vs clinician antimicrobial
prescription recommendations), an estimated 300 children 6 to 24 months of age presenting
to primary care practices with upper respiratory symptoms will need to be enrolled to
derive 150 paired interpretable images to detect a 10% difference in AOM diagnosis and
subsequent antimicrobial prescription rates between the AI app and clinicians, assuming a
power of 80% and two-sided p-value <0.05. This assumes 15% of children in the target
population will truly have AOM, and clinicians will diagnose and treat AOM at a 10%
higher rate (25%) compared to the AI app. If clinicians reconcile and follow the app
diagnosis and treatment recommendations, this will equate to a 40% reduced/avoidable
antimicrobial prescription rate. This estimate accounts for two ears per child and
estimates 50% of images will be uninterpretable by the app, and 50% of clinician exams
will not result in a diagnosis.
Statistical analysis All analyses will be conducted by a statistician in the General
Academic Pediatrics Division, Department of Pediatrics.
The primary outcome of antimicrobial prescription rates will be assessed by the McNemar's
Test. Differences in secondary outcomes will be assessed by generalized estimating
equation (symptom score) and chi square test (antimicrobial side effects and recurrent
AOM).
Descriptive outcomes (uninterpretable images, clinician ability to make a diagnosis, AOM
diagnosis, antimicrobial prescriptions) will be reported as rates (%).
Study Procedures:
Screening: Office schedules will be screened to identify children in the eligible age
group who are presenting with upper respiratory symptoms.
Consent: Once age-eligible children have checked in to the office their parents will be
approached by research personnel to assess their interest in the study and eligibility.
Informed consent will be obtained after rooming.
Study Procedures:
The duration of the study procedures, not including standard clinical care, will be about
10 minutes. After consent, each participating child will have two ear exams.
The first exam will be done before the clinician enters the examination room, by
study personnel using the AI app (research intervention). The device is a standard
otoscope head attached to an iPhone. Only the ear speculum touches the child. The
child may be held by the parent or staff either prone or upright. There will be two
attempts per ear and attempts will stop at parent request. Study staff may need to
remove cerumen (ear wax) using gentle irrigation or curette. Study staff are
clinicians experienced in and qualified to remove wax. We expect a significant
proportion of unusable images and have accounted for this in our sample size
calculation and study flow chart. If there is an uninterpretable image by the app
(cerumen, poor image, or poor cooperation) in both ears, the participant will exit
the study, and the clinician will be informed before entering the room so that
normal care can ensue (cerumen removal for example). If an interpretable image is
captured in at least one ear, the diagnosis ("app diagnosis") will be recorded in
the data form and blinded to the parents and the clinician. (The app only records a
video initially, then the user selects a brief section of the highest quality video
to render the diagnosis, which can be done outside of the room).
Study personnel will then step out of the room and the clinician will enter and do
the second exam (standard care). (Clinicians may remove cerumen also by irrigation
or curette, if necessary, even if the app obtained an interpretable view. Sometimes
cerumen can fall into the canal with entry or exit of an exam speculum.) Parents
again will be blinded to the clinician's findings - there will be no discussion
about the diagnosis or treatment.
The clinician will then step out of the examination room and document a diagnosis
("clinician diagnosis") and treatment decision without consulting the app ("app
diagnosis").
The clinician will then view the app video and diagnosis, and then record their
"reconciliation diagnosis" and treatment decision.
Clinicians will then reenter the exam room and inform the parents of their diagnosis
and treatment.
Clinicians will complete a "final outcome" indicating whether antimicrobials were
prescribed and the reason for prescription.
Data will be recorded in an electronic database.
If participants have a paired image (app and clinician rendered a diagnosis on the
same ear) they will complete an AOM symptom score scale (duration: 60 seconds) and
instructed how to enter the same symptom score in an electronic diary (duration: two
minutes).
Demographic Information:
Research staff will obtain demographic information after the study procedures (duration:
60 seconds).
Follow-up:
All participants will be followed for 10 days to assess symptom resolution and side
effects of antimicrobial use. AOM symptoms will be monitored daily for 10 days using a
validated symptom scale entered once every evening (duration: 60 seconds), whether on
antimicrobial therapy or not, by parents in electronic diaries which have used by our
team in many other studies. If any participant increases their symptoms score by >20% at
any time, they will be contacted and offered a visit. Rates of protocol-defined diarrhea
and diaper rash which are the most common side effects of antimicrobial use in this age
group will also be assessed. Finally, AOM recurrences will be monitored by reviewing the
electronic medical record for 3 months following enrollment.
Duration:
The app will only be used once, before the clinician exam.
The ear symptom score will be done at the enrollment visit and then for the next
10-11 days in electronic diaries.
Text message reminders to complete the diaries will go for 10-11 days after the
enrollment visit.
Medical record review will proceed for 90 days after enrollment.