Study design
This was a mixed-method convergent-parallel study. Both quantitative and qualitative data were separately collected, analyzed, and presented in the results section. The qualitative and quantitative data have been merged in the discussion section and organized by two main themes i.e., perceptions of youth on the intervention and perceptions of staff and peer providers on the intervention.
The evaluation used a single-arm, open-trial design.
Study setting
The pilot study was conducted at a youth clinic (Rafiki clinic) run by the Academic Model Providing Access to Health Care (AMPATH) [22]. AMPATH is a large chronic disease program in western Kenya and is a partnership between Moi Teaching and Referral Hospital, North American Universities, and the Kenyan Ministry of Health [22].
Rafiki clinic has a total enrolment of 799 youth. Of these, 80% are living with HIV and 99% are aged 15–24 years. Four peer providers work full-time at Rafiki. The peer providers are selected to work at Rafiki based on age (they should be aged 18–24 years), HIV status (must be HIV positive, virally suppressed, and ready to disclose HIV status), and willingness to support youth wellness. Experience with substance use is not considered when hiring peers at Rafiki clinic.
Before the pilot study, all peer providers had received a 5-day training (about 30 h) in HIV adherence counseling, and basic counseling techniques, but none on substance use screening and brief intervention. The training was facilitated by the Pediatrician, Clinical Officers, Psychologists, and Nursing Staff stationed at Rafiki Clinic.
At each clinic visit, the youth first consult with a peer provider in a private room before proceeding for clinician review. During the consultation, peer providers perform antiretroviral therapy adherence counseling, and or offer basic counseling on mental health-related issues that the youth may have such as dealing with stressful situations at school or home. The clinic sees about 300 youth monthly and is staffed by 10 staff directly involved in patient care: 1 pediatrician, 2 clinical officers, 1 psychological counselor, 1 nutritionist, 2 social workers, 1 pharmacist, and 2 nurses.
Rafiki Clinic was set up to address the unique health-related needs of youth and adolescents. For a long time, staff at Rafiki encountered youth with substance use challenges but could not address this problem, hence the need to implement this intervention at the clinic.
Screening and brief intervention program
The Screening and Brief Intervention program comprised of (i) screening using the ASSIST-Y (Alcohol Smoking and Substance Involvement Screening Test—Youth) questionnaire [23]; and (ii) a single session brief intervention.
We used the ASSIST-Y questionnaire to screen for the level of substance use involvement for all substance types including tobacco products, alcohol, cannabis, cocaine, amphetamine-type stimulants, sedatives, hallucinogens, inhalants, opioids, and ‘other’ drugs [23].
The first question of the ASSIST-Y asks about lifetime substance use for each of the substances listed above. Endorsement of lifetime use is then followed by an assessment of substance use in the past 3 months. The level of substance involvement is categorized as moderate or high risk and cut-off scores vary for each substance. Scores corresponding to moderate risk substance use are as follows: tobacco products [2-11], alcohol [5-17], cannabis [2-11], cocaine [2-8], amphetamine-type stimulants (ATS) [2-8], sedatives [2-6], hallucinogens [2-8], inhalants [2-8], opioids [2-6] and ‘other’ drugs [2-6]. Scores corresponding to high-risk substance use are as follows: tobacco products [12 +], alcohol [18 +], cannabis [12 +], cocaine [9 +], amphetamine-type stimulants (ATS) [9 +], sedatives [7 +], hallucinogens [9 +], inhalants [9 +], opioids [7 +] and ‘other’ drugs [7 +].
Following the screening, we administered a brief intervention that included either a 5–10-min session of positive reinforcement delivered to youth with no history of substance use over the past 3 months, or a 20–30-min brief motivational interviewing session that was delivered to youth with moderate and high-risk substance use.
Positive reinforcement consisted of personalized feedback on the ASSIST-Y scores, a statement that praised the youth for not using substances, and advice to keep away from substance use in the future. The youth were further given booklets with information on the harmful effects of substances to take home with them.
The brief motivational interviewing intervention was adapted from the WHO ASSIST-linked brief intervention manual. This treatment is based on the FRAMES model (i.e., providing feedback on screening results; ensuring responsibility on the part of the youth; giving clear advice to stop/cut down substance use; giving a menu of options on alternative healthy behaviors to engage in; expressing empathy; and encouraging self-efficacy); and motivational interviewing techniques (creating discrepancy and ambivalence, using open-ended questions, rolling with resistance, reflective listening and summarizing) [11].
The brief motivational interviewing intervention was delivered in 11 steps over a single session as follows: 1. Asking clients if they are interested in seeing their ASSIST-Y scores; 2. Providing personalized feedback to clients about their ASSIST-Y scores; 3. Giving clients advice about how to reduce risk associated with substance use; 4. Allowing clients to take ultimate responsibility for their choices; 5. Asking clients how concerned they are by their ASSIST-Y scores; 6. Weighing up the good things about using the substance against the; 7. less good things about using the substance; 8. Summarizing and reflecting on clients’ statements about their substance use with emphasis on the ‘less good things’; 9. Asking clients how concerned they are by the ‘less good things’; 10. An assessment of readiness or confidence to initiate change using the readiness steps; 11. Giving clients take-home materials to bolster the brief intervention. Youth with high-risk use received a referral to specialist care at the MTRH, Child Psychiatry Out-patient clinic in addition to the brief motivational interviewing [11].
Adapting the screening and brief intervention program
Before implementation, we adapted the ASSIST-Y and the World Health Organization (WHO) ASSIST-linked brief intervention using the ADAPT-ITT framework [24]. The framework is made up of 8 steps including Assessment, Decision-making, Adaptation, Production, Topical Experts, Integration, Training, and Testing of the evidence-based intervention. The framework has been utilized successfully in adapting a mental health intervention for youth in sub-Saharan Africa [25]. We conducted adaptations to the ASSIST-Y and the WHO ASSIST-linked brief intervention to contextualize them to the Kenyan context and for peer delivery. The adaptations were largely surface-level and comprised of simplifying the language to make it more understandable to the youth, adding instructions to make the manual easy to navigate for the peer providers, and adding street names for the substances to the ASSIST-Y. We maintained the core components of the intervention. Details of the adaptation process have been published elsewhere [26].
Peer training
In December 2021, we invited all four peer providers who work full time at Rafiki clinic to a training on how to deliver the screening and brief intervention program. Out of these, three completed the training. The training was conducted over 5 days using lectures, quizzes, and role-plays. On each of the days, the training was conducted between 8.00 a.m. and 4.00 p.m. with a 30-min tea break and a one-hour lunch break. We therefore allocated 6 h 30 min for the training on each day (total of 32 h 30 min over the 5-day period). During the training the facilitators delivered lectures that focused on the following areas: Introduction to substance use (types of substances, burden of substance use among youth), myths related to substance use, rationale for screening, screening using the ASSIST-Y, counseling skills, motivational interviewing principles, positive reinforcement, and brief motivational interviewing. The rest of the time was spent on role-plays that helped the peer providers to practice the screening and brief intervention, and counseling skills, with both hypothetical cases and real youth. Details of the training sessions, including content and time allocations have been provided in Supplementary file 1. The training was facilitated by psychologists and psychiatrists on the research team including E.K., W.R., F.J., J.B., G.A., and M.K.
At the end of the training, we conducted exams using standardized role-plays. Each peer provider was examined using 5 standardized role-plays and one real case (a young person with substance use) (Supplementary file 1). To assess competence, three facilitators rated the exam role-plays for each peer, using a fidelity checklist of the main elements of the interventions (see Sect. 2.7.1 of this manuscript for a detailed description of the fidelity checklist). Average scores were obtained for each peer.
Two of the three peer providers achieved satisfactory competence based on assessments by the facilitators during the training. The average fidelity scores were 90%, 86%, and 48% for peer providers 1, 2, and 3 respectively. Peer-providers 1 and 2 were consented and recruited into the study.
Study participants, recruitment, and study procedures
Youth
We recruited 100 youth aged 15–24 years between January and February 2022. We excluded youth who were ill during the appointment, those unable to speak fluently in English, and those who declined to assent or consent. The sample size of 100 was arrived at based on the number of youths seen at the clinic and budgetary considerations. Moreover, the sample size of 100 was large enough to inform us about the feasibility and acceptability of the screening and brief intervention program.
Of the 110 youth who were eligible to participate, ten youths, i.e., four females and six males, declined to participate. Eight out of the nine youth declining to participate were above the age of 18 years. The reasons for declining included: being in a hurry to leave and therefore not having enough time to participate (n = 5); declining without any explanation (n = 2); and not being comfortable with the content of the study (n = 3).
Before data collection, we piloted the study procedures and data collection tools and made adjustments as necessary. A trained research assistant approached all youth presenting for any form of service at the clinic and confirmed eligibility. The research assistant then explained the study procedures and sought assent or consent in English. Consenting or assenting was done in a private room within the clinic. The research assistant collected socio-demographic data from the assenting or consenting youth. The youth then completed quantitative measures of depression (Patient Health Questionnaire-9) and generalized anxiety disorder (Generalized Anxiety Disorder -7). Thereafter the peer providers administered the screening and brief intervention program to youth with moderate and high-risk substance use. Youths with high-risk substance use were additionally referred for specialist mental health treatment. Youth who had no history of lifetime substance use or those who had not used any substance in the past 3 months received verbal positive reinforcement, brief advice on the harmful consequences of substance use, and booklets with content on the harms of substance use.
One peer-provider delivered the intervention to 52 youth, and the other to 48 youth. The youth were conveniently assigned to the two peer providers. This is because the goal was to explore feasibility, so the intervention was integrated into routine clinic procedures and participants were assigned to peer providers based on whomever was available to deliver the intervention to the youth.
Immediately after the screening and brief intervention program, each youth completed quantitative measures of intervention acceptability. Overall, of the 100 youth screened, 63 received positive reinforcement, 35 received brief motivational interviewing, and 15 received brief motivational interviewing with referral to specialized treatment. Two youths who were to receive the brief motivational interviewing declined to see their ASSIST-Y scores and to continue with the brief motivational intervention. They were thanked for their time and given the substance use education booklets. Each youth was compensated USD 5.00 for the time they spent at the clinic.
Youth with moderate to severe depression and anxiety were referred to the Psychologist within Rafiki Clinic for further assessment.
We held five supervision sessions during the screening and brief intervention implementation period. The sessions were facilitated by two psychologists i.e., W.R., and J.B. During supervision, the peer providers gave feedback on the sessions, and challenging areas were addressed through role-plays. Fidelity to the screening and brief intervention program was assessed by audio-recording all the sessions and rating them using a checklist of key elements of the intervention. The peer providers were reimbursed USD 5.00 for every participant recruited.
Peer providers
We recruited two peer providers who had achieved competency at the peer provider training into the study. We obtained informed consent from the peer providers before data collection began.
Clinic leaders
We purposively identified four clinic staff who were involved in major decision-making at the clinic and recruited them into the study. We obtained informed consent from the clinic leaders before data collection began.
Implementation of the peer-delivered screening and brief intervention and peer provider supervision
The peers implemented the screening and brief intervention program between January and February 2022 for the recruited youth.
Quantitative data collection and analysis
Quantitative data collection
Researcher-designed questionnaires were used to collect socio-demographic data from the study participants. Youth socio-demographic data included: age, sex, marital status, living arrangement, level of education, and parental status. Peer provider socio-demographic data included: age, gender, level of education, and duration of work at Rafiki clinic in years. Clinic leaders’ socio-demographic data included: age, gender, cadre, highest level of education, and duration worked at Rafiki clinic in years.
The Patient Health Questionnaire-9 (PHQ-9) was used to collect data on depression. The PHQ-9 is a valid and reliable tool for measuring the severity of major depression. It is a 9-item tool that examines symptoms over the past two-week period. Each of the 9 items is rated as follows: 0 – “not at all”, 1 – “Several days”, 2 – “More than half the days”, 3 – “Nearly every day”. In a study conducted among Kenyan adolescents, the PHQ-9 was found to be a reliable measure of depression (Cronbach’s alpha 0.73) [27]. In that study, the cut-offs determined by Kroenke 2001 [28] were used i.e., 0–4 minimal depression, 5–9 mild depression, 10–14 moderate depression, 15–19 moderately severe depression, and 20–27 severe depression.
We measured levels of Generalized Anxiety Disorder using the Generalized Anxiety Disorder -7 (GAD-7) scale. It is a valid and reliable tool for measuring the severity of GAD. It is a 7-item tool that examines symptoms over the past two-week period. Osborn et al. [27] examined the psychometric properties of GAD-7 among Kenyan adolescents and reported that the reliability was adequate (Cronbach’s alpha for the present study was 0.78). GAD-7 has been used to evaluate anxiety among Kenyan youth [27]. In that study, cut-offs determined by Kroenke et al. were used i.e., mild anxiety [5,6,7,8,9], moderate range [10,11,12,13,14], and severe range [15,16,17,18,19,20,21, 29].
We conducted surface adaptations on the PHQ-9 and GAD-7 (e.g., simplifying the language) to make the items more understandable to the youth.
We obtained youth feedback on the acceptability of the screening and brief intervention program using the Dissemination and Implementation Measures—Consumer tool, acceptability module. The tool assesses intervention acceptability from the perspective of the recipient of the intervention. The module has 15 questions rated on a 4-point scale as follows: 1- Not at all; 2- a little bit; 3- a moderate amount; 4- a lot.
We obtained peer provider and clinic leaders’ feedback on the screening and brief intervention program using ‘Dissemination and Implementation Measures’ – provider and organization tools respectively [30]. Each tool is comprised of the following modules: Adoption, Acceptability, Appropriateness, Feasibility, Reach/access, Organizational climate, and General leadership skills. The organization tool has an additional module on Sustainability. Each module comprises of 12–15 questions rated on a 4-point scale as follows: 1- Not at all; 2- a little bit; 3- a moderate amount; 4- a lot [30].
The Dissemination and Implementation Measures were developed for LMICs by researchers from John Hopkins University [30]. The tools were developed based on the implementation science outcomes of Adoption, Acceptability, Appropriateness, Feasibility, and Reach/Penetration by Proctor and colleagues [31]. We conducted surface adaptations (e.g., simplifying the language) on the tool as allowed by the developers to tailor it to our context.
Fidelity ratings of the audio-recorded session (all 100 of them) were conducted using a checklist of key elements of the screening and brief intervention program. The fidelity tool was divided into four sections corresponding to the different parts of the intervention: Screening (2 items with a maximum score of 4); positive reinforcement (6 items with a maximum score of 12); brief motivational interviewing (17 items with a maximum score 34); referral to treatment (1 item with a maximum score of 2). Each item was rated on a 3-point scale as follows: Not at all i.e., the peer did not do the step at all (0); Partially completed i.e., the peer tried but did not complete it or did not do it well) (1); The peer completed the step and did it well (2). Before rating, two members of the research team i.e., F.J. and J.B. each rated the first 10 sessions and achieved 95% agreement before moving on to independent rating.
Quantitative data analysis
Descriptive statistics were used to summarize the socio-demographic, substance use, and mental health characteristics of youth. Levels of the various implementation outcomes were obtained by calculating mean scores across all items of the fidelity, and dissemination, and implementation measures.
Qualitative data collection and analysis
Youth focus group discussions
Between 8th and 31st March 2022, we conducted five focus group discussions (FGD) with 25 youth to explore their perceptions on the acceptability of the screening and brief intervention program. The youth were purposively identified, and FGDs were conducted, based on age, gender, and substance use risk, to ensure that the youth were comfortable enough to express their opinions. Supplementary file 2 provides details on the composition of the FGDs. The FGDs were conducted in a meeting room at a tertiary-level health facility in Eldoret. The FGD guides explored areas such as youths’ perceptions of the session content, perceptions about their interaction with the peer providers, and recommendations for improvement. The FGD sessions were led by experienced moderators M.K., W.R., and J.B. and were conducted in English. The FGDs started with the consenting process followed by an introductory session to ensure a comfortable and relaxed atmosphere. The discussions lasted an average of one hour and twenty-two minutes and were audio-recorded. Participants were reimbursed USD 5.00 for their time.
Peer-provider and clinic leaders semi-structured interviews
In April 2022, we conducted six individual semi-structured interviews with two peer providers and four clinic leaders (psychologist, nurse, pediatrician, and clinical officer) to explore their perceptions on the feasibility and acceptability of the screening and brief intervention program. The interviews were conducted at a private place within the Rafiki clinic. The semi-structured interview guides were developed based on the Consolidated Framework for Implementation Research (CFIR) framework. The CFIR lists a comprehensive set of implementation determinants organized into 5 domains: intervention characteristics, outer setting, inner setting, characteristics of individuals, and executing [32] For the peer provider interview guides, we included questions that explored all five domains of the CFIR i.e., intervention characteristics, outer setting, inner setting, characteristics of individuals, and executing (Supplementary file 3). For the clinic leaders’ interview guide, we included questions that explored four CFIR domains including outer setting, inner setting, characteristics of individuals, and executing (Supplementary file 3).
The interviews were led by experienced moderators M.K., W.R., and J.B., and were conducted in English. The interviews started with the consenting process followed by an introductory session. The semi-structured interviews lasted an average of 42.3 min and were audio-recorded. Participants were reimbursed USD 5.00 for their time.
Qualitative data analysis
Youth FGDs
The audio-recorded interviews were transcribed verbatim by G.K. and then entered into NVivo for analysis. The transcripts were reviewed, and initial coding was done separately by F.J. and M.K. using an inductive approach. The two discussed the codes and sub-codes and resolved initial disagreements to develop a refined codebook. The final coding of the transcripts was done by F.J. and M.K. using the refined codebook.
F.J., M.K., and M.O. then performed thematic analysis to identify codes that fitted into themes that addressed the question of whether the screening and brief intervention program was acceptable to the youth. The themes were developed and defined through a process of discussion amongst the three authors (F.J., M.K., and M.O.) until a consensus was arrived at. The codes and sub-codes fit into five themes i.e., youths’ perceptions of the screening and brief intervention program content and delivery; youths’ perceptions of the peer providers; the impact of the intervention on youth behavior; youths’ perceptions of usefulness of the intervention; and recommendations for improving the screening and brief intervention program.
Peer-provider and clinic leadership semi-structured interviews
The audio-recorded interviews were transcribed verbatim by G.K. and then entered into NVivo for analysis. Initial coding was done by F.J. and M.K. guided by the CFIR domains and constructs. The codes and sub-codes were iteratively reviewed and discussed by F.J. and M.K. to ensure that the coding of the domains was being done as intended. A final codebook was developed, and final coding was done by F.J. and M.K. independently.
Figure 1 below provides a timeline of the study activities for this project.
Timeline of study activities
link