Treating Complex Grammar Knowledge Deficits in School-Age Children With Developmental Language Disorder

AIMS For later-developing language skills, including those involving complex sentence forms, there are not only very few treatment studies, but effect sizes for the few that exist are quite modest. The investigators propose that the unexceptional treatment outcomes may be attributed to two issues: 1) the reliance on treatment methods that were developed primarily for preschoolers and that lie on the more explicit end of an implicit-to-explicit treatment continuum and 2) treatments that ignore the mapping of semantic roles to the main nouns that express the agent-patient relationship. The investigators hypothesize that explicit treatment methods in which children are taught to consciously apply a simple semantic mapping rule will result in strong in-treatment performance but at the cost of building implicit mental representations of semantic-syntactic mapping necessary to automatically and unconsciously apply this knowledge to untrained linguistic contexts.

The investigators will test this overarching hypothesis by using implicit and explicit intervention methods in two treatment studies. Two randomized clinical trials, one targeting subject-object relative sentences and the other targeting passive sentences, will test the degree to which results for the implicit and explicit therapy approaches replicate across different syntactic structures. Importantly, the investigators will hold critical aspects of treatment (e.g., dose number, session duration and spacing) and stimulus parameters constant across treatment methods to isolate effects to differences in treatment method. Finally, the clinical relevance of the previously modeled relationship between complex sentence comprehension and memory in the context of treatment outcomes that target these sentence types will be assessed. The two overarching aims are:

Aim 1. To determine whether explicit and implicit treatments will improve complex sentence knowledge and use in 8 to 11-year-old children with DLD.

Hypothesis 1a. Both treatments will be superior to no treatment. Hypothesis 1b. Children receiving the Explicit treatment will perform well on a proximal outcome measure that is structured similarly to the Explicit treatment format, but relatively poorly on distal outcome measures of language knowledge that require a strong mental representation of the syntactic form. Children receiving the Implicit treatment will show the opposite pattern of results.

Hypothesis 1c. The shift in mental representation for the syntactic template will affect children's ability to generalize the trained sentence form in a narrative context.

Hypothesis 1d. Child age, SES, and overall language severity, but not sex, will affect outcomes.

Aim 2. To provide an important experimental test of causal components of the proposed mediator model of sentence comprehension.

Hypothesis 2a. The degree of success in treatment will predict the ability to chunk words within clauses.

Hypothesis 2b. Working memory will mediate the effect of treatment on knowledge and use outcomes controlling for vocabulary and pretreatment sentence comprehension.

METHOD The essence of this clinical trials project is to compare the relative effectiveness of two treatments to improve the complex grammar knowledge of children with DLD. Children will receive one of two treatments (implicit, explicit) to improve the knowledge of either the subject-object relative structure (The zebra that the lion chased was running fast) (Study 1) or passive structure (The zebra was chased by the lion) (Study 2). Children will be randomly assigned to either the implicit treatment condition or the explicit treatment condition and to either immediate treatment or delayed treatment. Children will complete four outcome measures (see below) prior to treatment and twice after treatment, immediately and five weeks later.

Participants. A total of 300 children ages 8;0 - 11;11 with DLD will be enrolled in the project (150 for Study 1, 150 for Study 2). Children will come from public, private, and charter schools around Athens Ohio, Cache County Utah, Tucson Arizona, and Morgantown, West Virginia. All children will demonstrate language impairment but normal-range nonverbal IQ as well as hearing and vision or corrected vision; they also must be native English speakers. Prior to standardized language testing, children will complete a sentence comprehension task and a sentence chunking task to determine whether participants show a deficit in one or both of the sentence structures targeted for treatment. Participants must perform < 50% correct on one or both screening measures to move to formal language testing. Both inclusionary and exclusionary criteria appear elsewhere.

Outcome Measures. Five outcome measures will be administered prior to the start of intervention. The same measures will be re-administered prior to the start of treatment for the children assigned to the delayed treatment group. The measures will also be administered twice after treatment, immediately and 5 weeks later.

Treatments. There are 2 treatments, implicit and explicit. Children will receive just one of the treatments. Across both treatments, the same images will be used and the sentences that have been prepared for the treatment of the passive structure have been refashioned into object relative items, thereby creating sentences across the two treatments that contain the same words. The same number of exposures will be delivered to the children in each treatment. Each treatment includes 20 training sessions (20 training items per session) delivered over 10 weeks. As a result, both treatments will deliver high density exposure (n = 800) to a targeted sentence pattern.

Implicit Treatment. The implicit treatment is an entirely novel application of conversational recasting techniques for complex syntax. Recasting has been successful for treating morphology deficits. The method has also been successfully used to increase the production of relative clauses, providing proof of concept for this approach. The approach uses clinician modeling (i.e., a syntactic prime) through focused recasting. Focused recasting involves eliciting child utterances that obligates use of a target grammatical form followed immediately by a clinician model that corrects any incorrect elements. Recasts can also follow correct child utterances, confirming their grammaticality. Recasting does not include explicit instruction or feedback other than the recast. On each trial, children see an initial image and ~2s later hear a clinician sentence. Next, a second image is presented and then ~2s later the examiner says, "Now you tell me about the [Noun] (patient)." The prompt focuses the child's attention on the patient to obligate them to produce it as the first noun phrase. Each child attempt is followed by a clinician recast. The recast serves as an important second exposure to the target structure the child is learning.

Explicit Treatment. In explicit teaching intervention approaches, children are provided with explanations of rules that govern language structure (e.g., "When there are two of something, an [s] is put on the end on the end of the word. One book. Two books."), employ explicit feedback after child responses, and often use drill-like formats. Clinicians tend to default to treatments that fall on the explicit end of the implicit-explicit continuum, particularly with school-aged children. As such, children are asked to remember rules governing language use and apply them in treatment, rather than implicitly building mental representations over the repeated exposures as used in implicit approaches. In the proposed explicit approach, the investigators repeatedly provide a simple rule, presented auditorily, that represents the semantic mapping of agent and patient/recipient to their appropriate nouns. Pictures are used to provide the semantic context for the sentence, a factor that is absent in other treatments. This approach links sentence form to meaning rather than to shapes and icons. Like other explicit treatments, the investigators will explicitly teach the rule that maps semantic roles to syntactic elements (nouns), and the children must remember and apply the mapping rule.

Task Administration Fidelity and Reliability. All project personnel at each performance site have been (and will continue to be) trained on the faithful administration of all measures (standardized entrance tests, screening measures, outcome measures, treatments). Training videos on the administration and scoring of all measures have been prepared and viewed by all personnel who will work with the children. In addition, mock administration and scoring of all measures has been conducted with all personnel at each site. Furthermore, the same training videos and exercises will be repeated three times per year to ensure high fidelity and reliability across all staff.

Task administration fidelity and protocol scoring reliability will be assessed for all screening measures and outcome measures for all participants. This includes an independent observer viewing the examiner administering the various measures live or from the session video to ensure the examiner has administered each measure exactly as has been prescribed on the protocol sheet. Any deviations from the protocol will be noted on the fidelity form. A percent value for "substantive deviations" and "non-substantive deviations" (e.g., minor word change or addition) will be calculated. If substantive deviations are > 10%, examiners will undergo further training for task administration. Similarly, the reliability of the examiner's scoring of each measure will be calculated by an independent observer. If examiners' scoring reliability is < 90%, they will undergo further training for task scoring.

Fidelity of treatment administration will also be assessed for 25% of all participants' training sessions. Also, examiners' reliability in scoring of all participants' responses will be assessed during 25% of the sessions. Examiners who show administration fidelity containing greater than 10% deviation and/or reliability scoring below 95% will undergo further training for the administration and scoring of the treatments. It should be noted that the examiners delivering the treatments will be different from the examiners who administer the standardized and outcome measures. Moreover, the treatment providers and the "assessors" will be blind to each other, i.e., assessors will be blind to the treatment results and the treatment providers will be blind to the participants' performance on the outcome measures.

Data Entry Reliability Across Sites. A master Xcel sheet has been created to be used at all performance sites focused on the reliable entry of all participant data in our Master ACCESS database containing all participants' standardized/screening scores, scores on outcome measures, and scores for all in-session treatment performance. The Xcel for each site contains columns to enter project staff, measures, reliability values for each task, date for when each score for each measure has been entered into ACCESS, and date for when data entry has been verified. All data Entry into ACCESS will be completed by two staff members simultaneously. Initially, one person reads off each score and the other enters the score. This step is repeated twice to ensure all scores are entered properly.

Scientific Data: Storage, Sharing, Codes, Access, & Management. The raw de-identified interview data, standardized data, and experimental data will be stored in Box files that are restricted to the site PIs and key site personnel during data collection. At the conclusion of the study, the data will be preserved in an OSF repository to enable sharing data to validate and replicate research findings described in the Aims. Once uploaded, the data will be stored on their cloud-based platform and will be shared with anyone who registers with the website, which provides a free text search function.

The Open Science Framework repository will include a detailed user guide, a codebook with univariate statistics for each variable, and study-level metadata. Each variable in the codebook will include the variable name, a brief description of the item, the variable label, value labels, and standard codes for missing values.

Documentation in the form of .pdf documents will include a description of each task, and the task instructions. The full protocols will not be provided because they contain proprietary information that may be used in the development of a standardized test.

The clinical and experimental data will be analyzed with custom R code written using packages freely available on CRAN. Fully transparent and reproducible scripts will be available in .rmd and .pdf formats. Fully organized (paginated table of contents) and annotated output will be available in .pdf format.

Sarah Schwartz at Utah State University in coordination with professional staff at each site, working under the guidance of the three PIs - James Montgomery, Ronald Gillam, and Elena Plante. In addition, the Office of Sponsored Programs at Utah State University has created a data management and sharing plan compliance system as part of their process for submitting the annual progress report.

Assignment to Treatment Conditions and Study Design. Two studies will be conducted to test the degree to which findings replicate across different sentence types. Study 1 will target subject-object relative sentences and Study 2 will target passive sentences. Children will be enrolled in one or the other study, but not both, as treatment for one sentence type may influence response to a subsequent treatment. The investigators will evaluate this possibility by testing both sentence types pre- and post-treatment. Within each treatment study, there will be four treatment conditions. The Explicit vs. Implicit conditions test the relative efficacy of each treatment condition. The two delayed-onset Explicit vs. Implicit conditions will be used to rule out maturational factors as a reason for change. Each treating clinician will treat children in all four conditions so that clinician effects are held constant across conditions.

A stratified random sampling procedure will be used to assign children to the treatment conditions. Children will be stratified into 2 age bands (8-9, 10-11), with male and female subdivisions. As children enroll into the study, they will be categorized into 1 of the 4 strata and then randomly assigned within the strata to either the Immediate Implicit, Immediate Explicit, Delayed Implicit, or Delayed Explicit treatment condition (probabilities = .33, .33, .17, .17) using probability-based randomization (R package "randomizr"). This will permit us to control for age and sex as biological variables. Random assignment is the gold standard for controlling for other, unknown factors that may affect treatment outcomes.

Statistical Power. For each study, a sample of 100 participants yields 80% power for the treatment effect (between-within subjects' interaction) in a 2x3 mixANOVA with a .05 significance level, for a relatively small effect size of f = .13 (GPower 3.1, [149]). The more complex MLM power analysis was conducted with GLIMMSE 3.0 and applies the "Hotelling-Lawley Trace" for between two group multivariate test of mean differences. A sample of 100 participants (16 clinicians with about 6 participants each) yields 80% power for the same hypothesis provided the intraclass-correlation (ICC) for participants within clinician is 0.10 and the difference in the two groups is 0.75 standard deviations initially and maintained at follow-up. These values are conservative, reflecting relatively minimal effects for ICC and group differences. The immediate vs. delayed treatment analysis (3x2 mixANOVA) provides that 80% power will be achieved for a sample of 100 participants for a similarly small effect size of f = .16. Finally, the investigators have planned for 150 children per study to protect against uncertainty inherent in power estimates and to adequately power the structural equation models. Klein's recommendation for a minimum of 10 participants per path in a structural equation model was followed. The model includes 9 paths (detailed under Analysis Plan, below), for a minimum of 90 subjects. Therefore, 150 participants will exceed this minimum recommended level, adding stability to the estimates.

Data Analyses Plan Treatment Effects. The effect of treatment vs. the 'no treatment' period of the Delayed-treatment condition will be assessed using a 3x2x2 mixed design Analysis of Variance (mixAnova). This analysis will use 3 groups (Implicit treatment, Explicit treatment, Delayed treatment, n=50 per group) and 2 repeated observations on the proximal and distal outcome measures (Agent-Patient task, Priming task). For those randomized for immediate treatment (Implicit or Explicit) pre-treatment and post-treatment observations will be used. For those randomized for Delayed treatment, initial and pre-treatment observations will be used, covering the period during which participants receive no treatment. This analysis will determine whether either or both treatments exceed natural maturation over a 10-week time frame to address Hypothesis 1a. A significant effect of sentence type would indicate treatment effects are specific to the sentence structure targeted by the treatment.

To address the issue of whether Implicit and Explicit treatments produce differential effects on the trained items from the proximal and distal language outcome measures (Hypothesis 1b), the investigators will combine children in the immediate and Delayed treatment groups within the Implicit and Explicit treatment conditions (n=75 per treatment condition). Three repeated observations (within-subject: pre-treatment, post-treatment, and follow-up) will be used for two outcome measures (within-subject: proximal and distal measures) compared between the two treatment types (between-subject: implicit and explicit treatments). A 3x2x2 mixed design Analysis of Variance (mixANOVA) will test if type of treatment influences the growth in each dependent variable. Follow-up contrasts will allow for comparisons between treatment type immediately following treatment (post-treatment), as well as if the gains are retained (follow-up). Support for Hypothesis 1b would consist of an interaction effect consisting of better performance on the Agent-Patient task for children in the Explicit condition and better performance on the Priming task for children in the Implicit condition.

Parallel mixANOVAs will be conducted using scores from the Narrative Comprehension and Retell task as dependent measures (n=75 per treatment condition). This analysis will address treatment outcomes using a translational language task that reflects both a common childhood skill and one with curricular relevance. Support for Hypothesis 1c would consist of a significant main effect in which children in the Implicit condition out-perform those in the Explicit condition post-treatment.

To further investigate the roll of working memory and to control for child characteristics (age, race, sex, SES, overall language (TILLS score)) and treatment delivery (zoom vs. in-person), Multilevel Modeling (MLM), also known as Hierarchical Linear Modeling (HLM), will be utilized as a follow-up to each of the mixANOVAs described above. The MLM model is able to expand the mixANOVA to additionally incorporate continuous moderators and covariates, as well as control for additional lack of independence between participants due to nesting within clinician. Unlike mixANOVA, MLM incorporates all collected data from participants who may lack all observations (incomplete cases due to attrition) and relaxes the reliance on the stringent ANOVA assumptions of homogeneity of variance and sphericity, both of which are likely to be violated. Formally, the MLM will include fixed effects the interaction between the between- and within-subjects factors, and covariates, as well as random intercepts for both participant and clinician. Separate MLMs will be run for Explicit and Implicit treatment conditions (n=75 per model). A finding of a significant moderating effect for working memory or age, race, sex, and SES would support Hypothesis 1d concerning age and SES. Based on the investigators' previous treatment research, an effect for treatment delivery method is not expected.

Theoretical questions. Input chunking is crucial to sentence comprehension, an idea that the investigators put forward based on the results of a previous memory-mediated model of sentence comprehension. For this reason, the investigators expect that chunking of sentences into clauses should be related to performance on the treatment outcome measures (Hypothesis 2a). The investigators will test this prediction using linear regression using the proximal and distal outcome measures to sentence chunking scores. Separate models will be run for each treatment condition (n=75 per treatment).

The investigators also hypothesize that working memory will mediate the effect of the intervention (x) on student outcome (y) (Hypothesis 2b). The investigators will test this using structural equation modeling, with separate models for each Treatment conditions to allow for the possibility that different learning strategies are used for the two treatments (n=75 per model). The path from x to m(time2) will be labeled a. The path from m to y will be labeled b. The indirect effect from x to y through m(time2) will be the product of a and b (a*b). Following Muthen et al., the investigators plan to include working memory (m(time1)), vocabulary know ledge (C1time1) and syntactic comprehension (C2time1) as control variables in the model. An indirect effect that differs statistically from 0 indicates that m mediates the effect of x on y. If the indirect effect completely accounts for direct effect from x to y, then m fully mediates the relationship of x and y. In complete mediation, the product of a and b entirely displaces the c path. The investigators plan to compare the model fit statistics of 3 mediation models, one with working memory (Woodcock-Johnson Working Memory subtest) as the mediator, one with language long-term memory as the mediator (Narrative Retell task, Clause Completion score), and one with Vocabulary Knowledge (PPVT-5) as the mediator. Each of the models has a maximum of 9 paths, with 11 participants per path for each study.