Caregiver Training to Support Children With Social Communication Challenges

Background Pediatric speech-language pathologists (SLPs) report that a substantial proportion of their caseloads now include children with diagnosed or suspected autism. This aligns with persistent increases in autism diagnoses worldwide and underscores the importance of effective, feasible, and resource-efficient support services. For children with diagnosed autism, or children with social communication challenges who may be waiting for diagnostic assessments, identifying effective services targeting language and social communication is particularly important because development of these skills is among the most important contributors to long-term outcomes in autistic children. Furthermore, supporting social communication aligns with parents' treatment priorities for their autistic children.

SLPs provide early support services that focus on promoting engagement and communication within meaningful contexts. For young children, their parents are often their primary communication partners. Therefore, parent coaching interventions are commonly used by SLPs to support social communication development in young autistic children. Parent coaching interventions allow clinicians to capitalize on parents' expert knowledge of their child. These interventions increase the potential for children to generalize skills learned in sessions to their daily life, thus increasing treatment dosage without increasing the cost. Furthermore, by reducing the frequency of direct contact hours with the therapist, public systems can optimize resource use and make these interventions accessible to a greater number of parents.

The past decade has seen an increase in the number of randomized control trials (RCTs) examining the efficacy of parent-delivered early interventions for autistic children; however, most were conducted in specialized research clinics that often do not reflect how treatment is offered by community SLPs. This can negatively impact the generalizability of the findings to real-world clinical practice. While explanatory trials determine the efficacy of an intervention (in a highly controlled setting), pragmatic trials determine the effectiveness of an intervention in a real-world context, that is, everyday clinical practice. Pragmatic RCTs embrace the complexity, unpredictability, and inconsistency inherent in routine clinical practice by including diverse patient populations and allowing flexibility and variability in how the intervention is delivered. Additionally, few clinical trials have included examination of for whom the intervention works (moderators of treatment outcome), why the intervention works (mediators of treatment outcomes), or how implementation processes could have impacted treatment effects. This leaves clinicians with limited information to effectively individualize and refine interventions to fit child, family, and system needs.

One parent coaching intervention commonly used by SLPs is More Than Words® - The Hanen Program® for Parents of Autistic Children or Children Who May Benefit from Social Communication Support (MTW). In fact, a recent survey identified MTW as the preschool autism intervention program most widely used (61%) by SLPs in Ontario, Canada. The two existing RCTs on MTW involved university-based program delivery and report mixed findings that included positive effects in parents' responsiveness and aspects of social communication for some children. Non-randomized observational studies of the MTW program reported improved parent responsiveness, self-efficacy, and interaction style; and improved child language and social communication skills post-program. When asked about perceived outcomes, parents from community-delivered MTW programs highlighted the new strategies they learned, their child's growth in functional communication skills, overall improvement in their relationship with their child, and the social and professional support network they gained. Studies of parents' experiences participating in MTW suggests they value the program and the opportunity to connect with other parents, but seek improvements in navigating program content, the numerous forms associated with participation, parent-to-parent discussion, and the amount of individualized time with the SLP. Although these results are promising, as is the case for parent-delivered interventions more broadly, community-based RCTs of the MTW program are lacking, and little evidence exists for the mediators and moderators of effective MTW delivery in real-world practice. Pragmatic RCTs are needed to build this understanding.

For a pragmatic RCT of the MTW program (and other parent coaching interventions) to be useful, it is critical to ensure that the methods used to evaluate treatment effects and mediators/moderators are not only sound but also ecologically valid and feasible for use in the community practice contexts in which the study is to be conducted. Given that real-world practice can be highly variable and unpredictable in ways that are outside of the control of the researcher, large-scale RCTs can be at risk for implementation failure without careful attention at the methodological planning stages. One solution is to first conduct a pragmatic methodological feasibility trial to identify and develop mitigation plans for possible issues in recruitment, retention, data collection, and analysis. A second associated solution is to employ principles of integrated knowledge translation, that is, ensure that co-construction and collaboration between researchers and community partners begins right from the planning stages. This can ensure that outcome measures selected are ideal from both a reliability, validity, and sensitivity perspective and practical for clinicians and families to use during real-world program delivery. In sum, for a full-scale pragmatic RCT to be successful and for results to be meaningful to end users, foundational work is needed that focuses on (a) systematic co-development of a comprehensive evaluation protocol, (b) testing of the acceptability and feasibility of the evaluation protocol within real-world community settings, and subsequently, (c) pilot testing of a pragmatic RCT protocol.

Rationale With an overarching objective to conduct pragmatic RCTs of the MTW program, the investigators first needed a comprehensive evaluation protocol of the MTW program as implemented in community practice that would be both feasible for clinicians and families and sensitive enough to measure treatment effects and moderators and mediators of child and family outcomes. Therefore, the investigators partnered with program developers and expert SLPs from The Hanen Centre to co-construct a logic (causal) model of the MTW program that links specific program objectives with evaluation tools (outputs, outcomes) and procedures. In brief, key areas integrated during development of the logic model and evaluation plan included: (a) theories and research guiding MTW program development, (b) outcome measures and treatment mediators and moderators identified in previous research, (c) expert SLP views on MTW objectives and outcomes, and mediators and moderators affecting treatment responses, and (f) organizational structures, barriers, and facilitators for SLPs delivering the program. The result of this co-construction process was a logic model of the MTW program and an associated evaluation protocol with proposed outcome measures.

The investigators recently completed the second aim, which was to assess the feasibility of the selected outcome measures and proposed evaluation procedure ("A Single Arm Feasibility Study of a Caregiver-Delivered Intervention for Preschool-Aged Children with Social Communication Challenges"). The following dimensions of feasibility were assessed: (a) recruitment capability, (b) feasibility, acceptability, and practicality of data collection and analysis procedures, and (c) preliminary effectiveness based on the selected outcome measures. A pragmatic, single-arm, pre-post methodological feasibility study design was employed to implement our co-constructed evaluation protocol during MTW programs delivered by community-based SLPs as a part of standard care in the Ontario Preschool Language Program (PSL), a publicly-funded service. In brief, the results revealed that of those approached, 45% of SLPs and 57% of parent-child dyads were willing to participate. None of the 21 enrolled parent-child dyads withdrew from the study; however, outcome measure completion ranged from 48-81%, with 4 dyads providing little to no data. SLPs viewed study methods and measures as acceptable and feasible and researcher coding of video-recorded outcome measures was reliable and practical. Preliminarily effectiveness analyses showed post-program gains in children's early communication skills and communicative participation by parent report, and in 10 of 11 child, parent, and dyadic joint engagement behaviours coded by the researchers. All parents agreed or strongly agreed that their child benefited from the MTW program and most reported increased self-efficacy in supporting their child's communication.

Collectively, the results of the feasibility trial supported use of the investigators' co-constructed evaluation protocol in a future RCT of the MTW program. The current pilot RCT is the next step. The estimations of recruitment, retention, and adherence rates noted in the feasibility trial will provide valuable insights, allowing for observation of how new adjustments and procedures developed in response to these findings alter success in these components of the study procedures. This pilot RCT will also provide critical information on recruitment, retention, and data collection for the waitlist control group, which was not assessed in the previous study and for whom methodological uncertainties remain. Additionally, findings from the feasibility trial indicated that securing a sufficient number of community sites willing and able to randomize their patients was not feasible, as the program is already considered standard care despite limited definitive RCT data to support its use. Consequently, this pilot RCT has been designed to be as pragmatic as possible, aiming to simulate standard care while employing MTW program sessions specific to this study (i.e., sessions not already part of community SLPs' existing schedules).

Objectives The objective is to gather data that will inform the decision on whether to conduct, and refine the design of, a larger definitive RCT. This will be done by evaluating the feasibility of recruitment, randomization, retention, and adherence of parent-child dyads and exploring potential trends in the preliminary effectiveness of the virtual MTW program through a pilot pragmatic randomized waitlist control trial.

Methods The Virtual MTW Program in Ontario MTW is a 13-week parent-implemented program designed to enhance communication, play development, and parent-child interaction for children under 4 years of age who are autistic and/or have social communication challenges. MTW is delivered by SLPs certified by The Hanen Centre, a global not-for-profit organization based in Toronto that developed and routinely updates the program and trains SLPs in its delivery. Traditionally, MTW has been delivered in-person, but since the onset of the pandemic in 2020, a virtual version aimed at replicating the in-person experience was added. The program delivery model to be used in this study will be the virtual MTW program. Virtual delivery includes a group orientation session, a pre-program consultation appointment, eight 2½ hour group training sessions (typically with 6-8 parents per group), and three individual video feedback sessions. At the orientation session, the SLP provides an overview of the program and provides the parent with forms to complete (i.e., parent report of their child's communication, video consent form). At the pre-program consultation, the SLP conducts an informal assessment of the child's communication and records a video of the parent-child interaction. Parents and the SLP co-develop communication goals based on the child's social communication needs and ability. During group training sessions, parents learn support strategies to target goals (e.g., joining a child's idea in play, rather than directing the play). Group virtual education sessions include SLP-led didactic presentations, video examples, group discussions, and opportunities for practice (e.g., role play). During virtually delivered, individual video feedback sessions, parents demonstrate the use of communication facilitation strategies with their child, while the SLP records the interactions. The parent and SLP then review the videos together, analyze use of the strategies and how the child responded, and formulate plans to foster the development of new skills.

In Ontario, parents who have concerns about their young child's speech-language development can self-refer for services through the Ontario Preschool Speech and Language (PSL) program, a publicly funded provincial program supporting over 60,000 children from birth through to entry into senior kindergarten annually. For children with diagnosed or suspected autism in the Ontario PSL program, one common standard of care includes offering the MTW program to families. Some Ontario families elect to pursue privately funded SLP services instead. Although some private SLPs in Ontario may deliver the MTW group program as a part of their private services, this is less common.

For this pilot pragmatic RCT, registered SLPs employed by The Hanen Centre who routinely deliver MTW programs will deliver the program virtually according to their standard clinical delivery. For this study, recruitment of the parent-child participants and allocation to group program schedule will be independently conducted by the investigators

Participant Recruitment and Sample Size Recruitment will be complete through community SLPs across Ontario who provide services to children under 4 years. As such, most children will be recruited through their SLP in the Ontario PSL; however, SLPs in private practice who support this age range will also be sources of recruitment. The investigators will contact SLPs via email, directly for those in private practice or through regional managers for those working within the PSL program. A study flyer will be shared with SLPs for them to facilitate the recruitment of eligible parent-child dyads from their caseload for the study. Families will then self-refer if interested in participating.

Since pilot studies are not confirmatory trials and are hence not hypothesis driven, no formal sample size calculation will be carried out. Based on many literature studies, a sample size ranging from 12-35 is commonly recommended for conducting a pilot study. In the investigators' previous feasibility study, a recruitment rate of 57% was observed with 21 out of 37 parent-child dyads approached by SLPs enrolled in the study. There was no attrition; however, completion rates for pre-post outcome measures range from 60-70%. Based on these findings, the investigators aim to recruit 36 parent-child dyads for each of the two groups to target at minimum 15 complete datasets in each of the treatment and control groups.

Eligibility Pre-Screening. SLPs will share the study flyer with potential participants whom they believe to be eligible. Interested families will then be pre-screened for eligibility. Either they will complete an eligibility screening survey (delivered using REDCap, a secure, online data collection platform) by following the survey link from the flyer or they will contact the Study Coordinator by email/phone and answer pre-screening questions over the phone with the study coordinator or other study team member. As a part of this screening, parents are asked to confirm they are available to attend group sessions on either Saturday mornings or Monday evenings (or both).

Informed Consent. Families who screen eligible will be provided with a copy of the study consent form to download and review within the survey or will be sent a copy via email. A member of the research team will schedule a phone or Zoom call with the parent in which they will meet with a member of the research team to review the study information and consent form before e-signing it (in REDCap).

Measures Feasibility Outcome Measures. To examine the primary research questions related to the feasibility of the proposed RCT method, the following areas will be evaluated.

1. Recruitment. The number of participants who complete and pass the screening survey will be recorded. The number of participants who enroll (sign the consent form) will also be recorded.

2. Retention: To assess retention in the MTW program, parent's attendance over the duration of the program will be recorded by the Hanen SLP delivering the program. Study withdrawal and loss to follow-up will also be recorded.

3. Randomization: The randomization method will be assessed by comparing the treatment and waitlist control groups pre-program to ensure no significant difference between them in the baseline measures including child's age, sex, or CFCS Level.

4. Adherence: Completion rates for each baseline and treatment outcome measure at each time point will be recorded for both groups

5. Fidelity: A SLP fidelity measure was developed for this pilot RCT, to assess the Hanen SLP's ability to implement the program components as intended. The individual items in this measure represent content that SLPs delivering the program attend to, but do not necessarily explicitly document. As such, the self-administered, session-specific checklists in this measure aim to document what the SLP covers when delivering the group and individual sessions (in accordance with the MTW Program Leader's Guide that manualizes program delivery). After each session, the SLP indicates Yes (2), Partly (1), or No (0) about their ability to complete each program component associated with that session, which will be used to generate an overall session fidelity percentage score (total score/ total possible score x 100).

Demographic/Baseline Measures. These measures will be collected to describe sample characteristics, confirm group equivalence after randomization, and conduct exploratory mediator/moderator analyses.

1. Child and Family Questionnaire. This form was developed by the investigators to collect demographic information of the children and parents participating in this study. It includes details about the parent's relationship to the child, cultural and ethnic background, home language, education, and household income. For the child, it gathers information on age, sex assigned at birth, health conditions, developmental diagnoses, past or current interventions, and participation in an early learning environment. Additionally, it explores family history of speech, language, or learning difficulties.

2. Communication Function Classification System (CFCS). The CFCS is a valid and reliable tool used to classify a child's everyday communication abilities into one of five levels based on how they send and receive messages. The SLP assesses the child's current level of communication based on function not chronological age or developmental stage using the CFCS flow chart, which includes five levels ranging from Level I (Effective Sender and Receiver with unfamiliar and familiar partners) to V (Seldom Effective Sender and Receiver with familiar partners). As such, a lower level is indicative of greater effectiveness as a communicator.

3. MTW Social Communication Stage. The Social Communication Checklist is a Hanen Centre-developed MTW program form that is completed by SLPs in collaboration with parents. It is used to identify the stage that best describes the child's current communication skills according to stages addressed by MTW program and coined by The Hanen Centre as: Explorer, Requester, Early Communicator, and Partner. The number of checkmarks at each stage is tallied, with more checkmarks at higher stages indicative of better outcomes. The number of skills children gain at each stage from before to after the program will be counted.

Treatment Outcome Measures. A variety of child and parent treatment outcome measures will be recorded at baseline (T1 for TG; T1 and T2 for WCG), post-program (T2 for TG and T3 for WCG) and after 3 months of receiving the intervention (T3 for TG and T4 for WCG). These measures are described in the Outcome Measures section of the trial registration.

Following enrolment, the study coordinator will assign each parent-child dyad a unique study identification number, which will be linked to identifying information gathered through the consent process in a separate Master list.

Baseline Demographic Information Collection. Parents will receive an email with a link to Child and Family Questionnaire (a REDCap survey) to collect demographic and baseline information about the child, parent, and family. Additionally, an investigator who is (or is supervised by) a registered SLP will conduct the parent-child appointment remotely at a time convenient for parents. The purpose of this appointment is threefold: (a) to ask parents about their child's involvement in an early learning environment (a demographic data point for the Child and Family Questionnaire that requires discussion), (b) to discuss and observe the child's communication abilities in order to determine their current level of communication function, and (c) to review instructions for the parents to video record interactions with their child during the study. A semi-structured interview guide will be followed to ensure all three components are completed, and the research team member will code responses on a REDCap survey linked to the Child and Family Questionnaire previously completed by the parent.

Random Allocation. Once the above demographic/baseline information has been collected, parent-child dyads will be stratified according to their MTW group program availability (Saturdays and/or Mondays) in preparation for random allocation within these two strata. A maximum of 9 families can attend MTW program group sessions according to Hanen Centre requirements. As such, once 18 families enroll who are stratified into the same MTW group session day (Saturday or Monday), they will be randomly allocated to the TG (to receive the MTW intervention immediately) or the WCG (to receive the intervention after a 3-month waiting period). Given the relatively small sample size, blocked randomization stratified by session day availability will be employed. The randomization order will be generated in 3 blocks of 6 (3 TG and 3 WCG). To minimize the likelihood of more than three consecutive parent-child dyads being assigned to the same group, only 14 of the 20 possible block permutations will be used, with sequences TTWWWT, TWWWTT, WWWTTT, TTTWWW, WWTTTW, WTTTWW being eliminated. The study coordinator will use the following random sequence generator that follows these criteria for group allocation: https://www.online-python.com/GZQV5NHF6r. The same process will be repeated as each group of 18 families per stratum enroll. Depending on the rate and frequency of dyad recruitment, this means that overlapping Saturday and/or Monday MTW programs cohorts could occur during the study.

Once randomly assigned to groups, the study coordinator will contact families to inform them of their start dates, along with their scheduled time frames for completing study activities. A spreadsheet containing the names and contract information of the parent-child dyads for the TG will be provided to the Hanen SLP assigned to deliver that program session so that clinical contact and planning can begin. The same information for the WCG will be provided to the Hanen SLP assigned to that session ~3 weeks prior to its start date so as to minimize clinician awareness of families' allocation to TG versus WCG, recognizing that it may not be possible to fully mask order from the Hanen SLPs.

Outcome Measure Assessment Points. At each time point, parents will receive a reminder email that will include a link to the secure OneDrive folder specific to their family that will have a copy of two fillable PDF forms to be completed (FOCUS-34, CSBS-DP CQ) and instructions on how to record and upload their video interaction. Parents will collect 10-minute videotaped naturalistic interactions of themselves playing with their child and then upload them to their OneDrive folder. Included in the email will also be a link to the Parent and Services Evaluation (a REDCap survey), which includes questions about the past three months related to (a) parent's reported self-efficacy and (b) services and supports accessed for the child.

MTW Program. Once the family starts the MTW program, they will not be contacted by the investigators until after the program ends. During the pre-program consultation, the Hanen SLP will record the child's MTW social communication stage. Throughout the program, the Hanen SLP will record parent attendance and complete the corresponding SLP fidelity checklist to record their own fidelity to treatment for that session. These data will be uploaded by the Hanen SLP to a secure OneDrive folder unique to that SLP. The study coordinator will then abstract the social communication stage, adherence, and SLP fidelity into a case report form that uses only de-identified numbers for the dyads before submitting the data for analysis.

Video Coding. Following submission of all parent-child interaction videos, the video filenames will be de-identified by the study coordinator to mask both the group allocation and the time point of each video. Two trained and blinded research team members will code the JERI and Parent Fidelity from these video recordings, rating of each variable using REDCap forms. For all three measures, two coders will independently code 20% of videos to determine reliability using Cohen's Weighted Kappa.

Planned Analyses Analyses will primarily use descriptive statistics and confidence interval (CI) estimation to evaluate the study procedures and outcomes. Descriptive statistics will be used to characterize feasibility outcomes and participant measures at baseline and Times 1-4. Continuous variables (e.g., CSBS Total Raw Score, FOCUS-34) will be expressed as means and standard deviations (M ± SD), while categorical variables (e.g., JERI variables, CFCS levels, self-efficacy) will be presented as frequencies and percentages. Descriptive and statistical analyses will be conducted using JASP (Version 0.19.0).

A) Primary Feasibility Outcomes The summary below outlines the one overarching and five key progression criteria [hat will be evaluated using a traffic light system to determine whether a definitive RCT is feasible and what changes would be required (Green - RCT is feasible & no changes are needed, Amber - RCT is feasible following minor changes, Red - RCT is not feasible without major changes.) To assess baseline equivalences between the two groups, independent t-tests will be conducted in continuous measures and Fisher's exact test for categorical variables.

Objective 1: To evaluate overall feasibility of study procedures (recruitment, retention, randomization, data collection, data management, follow-up)

1. Green: Each component of the study procedures runs smoothly without serious problems or obstructions that were able to stop the study. Study procedures are judged strongly feasible and suitable to proceed by the study team and community collaborators.

2. Amber: Study procedures are judged to be likely feasible by the study team and community collaborators, but minor problems in one or more components of the study procedures need to be amended before proceeding

3. Red: The study team and community collaborators identified major problems in one or more components of the study procedures that must be remedied in order for the study to be feasible and to be able to proceed

Objective 2: To evaluate parent-child dyad recruitment rates

1. Green: 72 dyads are recruited (100%) within 9 months, with at least 30 dyads recruited within the first 4 months

2. Amber: 72 dyads are recruited but it takes longer than predicted (e.g.,12 months)

3. Red: Unable to recruit at least 36 (50%) dyads within 9 months

Objective 3: To evaluate parent-child dyad retention through to study completion

1. Green: ≥ 80% dyads retained to study completion (< 20% dropped out) in both the Treatment and Waitlist Control groups

2. Amber: 60-79% dyads retained to study completion in both the Treatment and Waitlist Control groups

3. Red: < 60% dyads retained to study completion in either the Treatment or the Waitlist Control group

Objective 4: To evaluate ability for randomization to yield baseline equivalence of groups

1. Green: Children in the Treatment and Control groups did not differ on baseline characteristics

2. Amber: Children in the Treatment and Control groups minimally differed in a few baseline characteristics

3. Red: Children in the Treatment and Control groups differed in multiple baseline characteristics

Objective 5: To evaluate adherence to outcome data completion.

1. Green: ≥ 80% completion rate (< 20% missing data) for all outcome measures at the pre- and post-treatment data collection points; and ≥ 70% completion rate for all measures at follow up

2. Amber: 60-79% completion rate for one or more outcome measures at the pre- and post-treatment data collection points; and/or 50-69% completion rate for one or more measures at follow up

3. Red: < 60% completion rate for one or more outcome measures at the pre- and post treatment data collection points; and/or < 50% completion rate for one or more measures at follow up

Objective 6. To evaluate SLP fidelity to treatment

1. Green: ≥80% of SLP sessions meet MTW fidelity standards (≥ 75% overall session fidelity percentage score)

2. Amber: 70-79% of SLP sessions meet MTW fidelity standards

3. Red: < 70% of SLP sessions meet MTW fidelity standards

(B) Secondary (Exploratory) Outcomes Although this study is not designed or powered to detect statistical effects, repeated measures ANOVA will be used to explore preliminary effectiveness, that is, possible changes within each group (TG and WCG) over time according to the treatment outcome measures. Post-hoc pairwise comparisons will explore differences between specific time points and groups. Given sample size limitations, p-values will not be examined. Effect sizes for time, group, and interaction effects will be calculated using Eta Squared (η²). The effect sizes will be reported, with thresholds interpreted as small (η² ≥ 0.01), medium (η² ≥ 0.06), and large (η² ≥ 0.14). 95% CIs will also be reported for the effect sizes. The Intention-To-Treat principle will be followed for these analyses.

Exploratory analyses aimed at identifying potential moderators and mediators of treatment effects will be conducted by combining data from the waitlist group after receiving the MTW program with the treatment group (maximum possible N=72). Potential moderators such child factors (e.g., age, functional communication level), parent factors (e.g., self-efficacy, level of education), and family context (e.g., household income, engagement in prior services, as well as mediators (e.g., parent self-efficacy, parent fidelity, early adoption of strategies) will be drawn from theoretically informed selection of variables including child factors and family context. Analyses will include visual inspection as well as examination of estimated effects, and 95% CIs.