Changing Youth Narratives on Firearm Violence ("Run It Up") Intervention

The overall design for the intervention evaluation is a two group, quasi-experimental community cohort design with a pretest (baseline) and multiple post-tests (follow-ups, see Shadish et al., 2002; Cook & Campbell 1979). A baseline survey and two follow-ups of 12-16-year-old youth and one parent/guardian per youth will be collected in the intervention and comparison communities, separately measuring a number of resilience, risk-related and demographic variables, hypothesized mediators/moderators linked to the intervention and its theory of change, intervention and related media exposure, and self-report outcomes. The comparison community (Marshall Heights) has been matched to the intervention community (Washington Highlands) as closely as possible on demographic and violence data (e.g., 95% African American in the intervention community, vs 90% African American in the comparison community; both communities have high poverty rates). A preliminary analysis will be conducted at each follow-up, with full analysis after the last follow-up. We will also collect firearm and related violence data as well as place-based aggregated data from designated community Census tracts for two years pre-intervention to the final intervention year for both communities, either monthly or quarterly, depending on whether there are enough samples in each period, in order to conduct interrupted time series analysis (ITSA) to assess the intervention effect on community-level outcomes.

Sampling Because there is no sampling frame from which to randomize, we are using a targeted snowball method (see Valerio et al., 2016; Bernard, 2018) through community partners (the CSC) who are familiar with most community families. Recruitment will occur through these interpersonal networks, using flyers with study information and eligibility requirements, prompting a snowball process where initial contacts recruit others. This method is more productive, facilitates the consent-assent process and recruitment of parents/guardians, and may avoid inherent safety concerns compared to sending staff to knock on doors for recruitment. Further, it will likely increase awareness and trust in the effort. It will also support a community cohort design, in which the same samples are followed up over time, with an improved ability to make causal inferences.

To increase rigor, we have increased the annual survey sample size proposed in the funding application from 200 youth/200 parents to 250 youth and the same number of parents (in both communities, for a total n of 500 youth, 500 parents each wave), in order to achieve enough power in the supplemental propensity score analyses, and for dose-response analyses. Along with that, we will implement recruiting strategies to maximize direct youth participation in the intervention activities, to reach a goal of approximately 100-150 youth participating in the intervention activity tracks. These participants assessed at baseline will be followed up at 12 and 24 months after the baseline survey, separately.

Administration The baseline and follow-up surveys will be administered via mobile phone and tablets to a community sample of youth and parents/guardians. In the intervention community, data collection will not be limited to youth who participate in the alternative trajectory intervention activity tracks, because the goal is to assess intervention effects on community youth as a whole (including those directly participating in activities). However, to improve linkage between survey data and actual intervention participants, we will link survey and intervention participant IDs by participant names, which will be maintained in a separate password-protected secure file and not appear on any survey or intervention process data. We will create variables indicating dose to intervention exposure including participation in the intervention activity tracks and " and "media exposure". In this way, we can assess the dose-response effect among the youth in the intervention community. The survey and data storage will use REDCap software for security and efficiency.

Process and Qualitative Measures Extensive process data will be collected, e.g. attendance of each session and satisfaction with the session for each youth participating in the intervention activity tracks. In addition, each program year, we will randomly select 15 youth who are involved in any program activities, and conduct a one-hour interview with them about their experience with program media/messaging and any changes they feel have occurred for them. We will also conduct interviews with two adults named by the youth as program-related influencers or mentors, to assess any changes they have seen in the youth, and their perceptions of factors leading to the changes. These qualitative interviews (n=45) in Years 3-5 will be conducted and analyzed using the same methods outlined for the Phase One formative research.

Analyses Originally planned analyses include descriptive analyses, Difference-in-Difference (DID) analysis to test intervention effect, sensitivity analysis to assess contamination, mediation analysis and moderation analyses based on survey data, and interrupted time series analysis (ITSA) based on community data. A number of changes have been made in the original analytical plan to increase rigor, recognizing the complexity of a quasi-experimental design with one intervention and one comparison community. First, the interrupted time series component has been strengthened in several ways. This component entails tracking a set of data at the community level that we would expect to change (in a positive direction) over time as a result of the intervention. As a minimum set of community data, each occurrence of any crime, gun-related crime, non-gun-related crime in the study Census tracts will be derived from data reported through the website crimecards.dc.gov, tracked monthly from two years before the intervention and then across the intervention period. The data will be aggregated over time and over the specific Census tracts in each community (by matching addresses to Census tract) to calculate monthly or quarterly crime rate (number of crimes divided by the population), depending on how many crimes in each community per month. In addition, we are seeking community-level placebo measures that will not be changed by the intervention. These will most likely include: (1) Temporary Assistance for Needy Families (TANF) data from the DC Department of Health; and (2) child poverty data from the DC Office of Planning -- across time in each Census tract and then aggregated for analysis. The ITSA with placebo outcomes will increase the robustness in the analyses. Similarly, for DID using survey data, we have added placebo measures in the survey that we would not expect to change as a result of the intervention - for example, diet, or self-reported height and weight -- as an additional means to test the robustness of DID.

In addition, we have added propensity score matching as a supplemental analysis. For this, we will match the youth based on their baseline characteristics (sociodemographic and outcome variable at baseline) from the intervention community and comparison community to mimic an RCT for assessment of the intervention effect. Propensity score analysis will be implemented with "teffect psmatch" using STATA software. (https://www.stata.com/manuals/teteffectspsmatch.pdf).

Finally, we are collecting survey IDs, participant IDs, and tracking short-term outcome and process data collected by intervention component, social media/community event exposure, and brand equity data , allowing for analysis of intervention effect by type and degree of exposure - because we will be able to link participant IDs and survey IDs.