Pros/cons of conducting YRBS and YTS in the same year – 4/8/09

Q: Have any states worked with programs that conduct the YRBS to coordinate timing of the survey, do a split sample where half of the classes in the high schools are chosen to do the YRBS and the other half do the YTS, or in some other way coordinate?  We are being asked to do this in 2010/2011.  In the past the YRBS has been done in odd numbered years and the YTS has been done in the even numbered years.  Did combining them into one year make a difference to you? What do you see as the pros and cons of doing the surveys at the same time?

A:

1. Alabama: Alabama does not conduct the YTS and YRBS during the same year.
    
2. Hawaii:
   1. Have any states worked with programs that conduct the YRBS to coordinate timing of the survey, do a split sample where half of the classes in the high schools are chosen to do the YRBS and the other half do the YTS, or in some other way coordinate?
      • Hawaii has created a Youth Health Survey Committee with members from the Dept of Health and the Dept of Education to coordinate survey implementation for the YTS, the YRBS, and also an ATOD survey conducted by our Alcohol and Drug Abuse Division.  This has been very well received by the DOE as schools were protesting the frequent interruptions for yet another survey.  Our Tobacco Prevention & Education Program has had a major role even before we conducted the first YTS.  We have coordinated the last three surveys (2003, 2005, and 2007).
      • Yes, we do a split sample for YTS and YRBS.  CDC handles the selection of classes for both surveys.
         
   2. We are being asked to do this in 2010/2011.  In the past the YRBS has been done in odd numbered years and the YTS has been done in the even numbered years.
      • This may not be a viable option for Hawai‘i as the agreement with the DOE is that there will be a coordinated effort for surveys in the odd-numbered years.  They will not allow us to survey in the even years.
         
   3. Did combining them into one year make a difference to you?  What do you see as the pros and cons of doing the surveys at the same time?
      • Yes, we see this as very positive as this has made it possible to gain access to the DOE schools.  We contract with the same surveillance group (Curriculum Research Development Group [CRDG] for the administration of both surveys and have a great working relationship. This works well because the logistics are similar.
      • Cannot think of any cons.  However, an area of concern is the fact that we have not been able to administer the surveys in Hawai‘i’s private schools.  It is essentially a matter of funding, but also the needed coordination with the Hawai‘i Association of Independent Schools.
         
3. Massachusetts: In Massachusetts for the first time in 2007, we conducted two youth surveys in the same classrooms - YRBS and YHS (Youth Health Survey).  The YHS is similar to the YTS in that more tobacco-related questions are asked.  The UMass Center for Survey Research is the Massachusetts contractor for both surveys.
    
   The benefit of combining the surveys is that school response rates are higher, since we promised only to administer it every other year. Also, it creates a natural experiment for common questions - 30 day tobacco use questions in this case - to see if there is a bias.  For example, when the 30-day question about smokeless tobacco was included with the lifetime question, the current smokeless tobacco rate came out significantly lower.  The contractor believed that youth want to report using smokeless tobacco if they've used it fairly recently, even if it was more than 30 days ago.  The lifetime question gives them a place to indicate that they used smokeless tobacco.  If there is no lifetime question, then they tend to say that they used it within the past 30 days.
    
   The difficulty with administering two surveys in the same year is deciding about what value to use as your estimate.  In Massachusetts, the YRBS trumped the YHS.  That is, if the question was asked on the YRBS, then the YRBS figure was reported.  Otherwise, the YHS figure was reported.  Alternatively, the common survey questions could have been weighted so that both surveys are used in the estimate.  The sample size would have been greater, but there was concern that question ordering could have an impact on the response.  Ultimately, I believe that mostly because it was simpler, we reported the value from one survey - the YRBS or the YHS.
    
   In retrospect, I think it would have yielded more accurate information to combine the weights for common questions.  However, if the contractor is different for the two surveys, then determining the weights for common questions could be even trickier.
    
4. Nebraska: Nebraska is working towards a coordinated school health survey and there are still some details to work out but in a whole we are gearing up for 2010.
    
   Tobacco Free Nebraska program within the Department of Health and Human Services has interest in both YRBS and YTS and has taken a key role in both surveys. In Nebraska, Department of Health and Human Services work together with the Department of Education on the YRBS and YTS. YRBS has been conducted every odd years and YTS has been conducted every even years. However, since about 2005, another survey – the Risk and Protective Factor Survey (RPFS) by DHHS Behavior Health sponsored by SAMSHA was introduced. RPFS has been done as a census which means every school is asked to participate and targets Grades 8, 10 and 12. It has been a key point for evaluation for Local Coalitions that are funded through SAMSHA grants and is more of a requirement.
    
   The introduction of RPFS has created competition for YRBS and YTS as schools have felt inundated with surveys. Some schools have preferred to do the RPFS survey as it gives them local data including school specific reports which is not the case for YRBS and YTS (YRBS and YTS are random statewide surveys and give statewide estimates).
    
   In addition to competing for schools to participate in the surveys, other challenges have also been experienced in conducting the survey in the state. As a result we have not been able to get enough schools in the YRBS sample to participate in multiple years and our data could not be weighted.
    
   Since 2007, we brought together a team of people from DOE and DHHS Public health Division and DHHS Behavior Health to address this problem. We recognize the importance of each survey (local vs. statewide data) but decided we need to be able do all three surveys with limited convenience to schools and with full support for all. Here is our approach that was devised;
    
   1. Meeting
       
      1. Clean up our acts – basically an internal group from DHHS and DOE to iron out our differences and work as a team and not as competitors for school survey participation
      2. Survey with school Principals
      3. Meeting with stakeholders  – internal group (DHHS, DOE) meeting with School Principals or representatives, School Board members, Local Coalitions, Health promotion groups which came up with the plan for a Coordinated Survey
          
   2. Coordinated Surveys done once every two years
      Based on the meeting with stakeholders;
       
      1. We agreed that we will conduct all three surveys at the same time starting in Fall 2010
      2. All three surveys will be administered at the same time
      3. Sampling Procedure
         • A random sample of schools drawn by CDC will be selected and classrooms randomly selected from the schools
         • A random sample of schools drawn by CDC OSH for YTS and classrooms will be randomly selected from the schools. In cases where the school has been selected in the YRBS, a random sample of schools will be selected from the remaining classrooms excluding those in the YRBS i.e. sampling without replacement.
         • RPFS survey will use all remaining classrooms after drawing for the YRBS and YTS.
         • In case of small schools, we might have students take a couple surveys e.g. YRBS or YTS and RPFS
      4. Questionnaire
         • YRBS questionnaire will be used in its entirety with some state added questions
         • YTS questionnaire will be used in its entirety with some state added questions
         • RPFS questionnaire will be used in its entirety
            
   3. Marketing the Plan
       
   1. Involving stakeholders in the planning meeting
   2. Information Package
      • A brochure has been produced with a summary on each survey
      • All three survey questionnaires
         
   4. Combining them into one year
       
      1. since we will be doing this for the first time in 2010, we can not say if it will make a difference
          
   5. Pros and Cons of doing the surveys at the same time
       
   1. Pros
      • Cut down the cost of doing the surveys
   1. One contractor could be engaged to oversee all surveys
   2. Cut down on administrative costs
   3. One incentive for school participation taking two surveys (if you give incentives)
   • Less inconveniencing for schools since the surveys goes into schools once in two years
   • Less time spent in schools
   • Contact schools only once
      
   2. Cons
      • In smaller schools, students might have to take a couple surveys and that may increase the time needed to complete the survey
      • Doing all three surveys once in two years leaves room for other surveys to go into schools. There are other unknown surveys that may still be introduced to schools and thus take up the time – we hope to guard against this by sharing the three survey data extensively
         
5. New Hampshire: Over the past four years NH TPCP has had the same part-time survey coordinator which allows the tobacco program to field both surveys simultaneously. The reason for the change from past practice of five years ago is linked to No Child Left Behind. School sdministrators were challenged to reduce teaching/class time for surveys administration. (The DASH and OSH surveys are not the only surveys that are conducted.) It was at the behest of NH School Administrators that surveys be administered at the same time.  Administration of the two surveys requires a lot of relationship building and coordination to ensure enough data is collected but that the two surveys do not go to the same student (random class selection).

   Fielding two surveys simultaneously requires fiscal assistance from the NH Department of Education and the NH Bureau of Drug, Alcohol and Substance Abuse. OSH provides support through the cooperative agreement with NH TPCP which pays for 50% of the part time survey coordinator's time. This person has been fielding it for 4 years and has become very familiar with the nuances that occur every year. The survey coordinator sits within the NH TPCP. NH TPCP could not do this without her position and her experience. The TPCP Epidemiologist provides direct assistance to the PT Survey Coordinator and works directly with OSH Epi Staff.

   Because of NH's excellent response rate, NH YTS data was recently selected to be utilized for a national study on seasonaility of fielding the YTS.

   Other comments:

   • In NH an entire single classroom receives the YTS while another entire classroom receives the YRBS, we do not split a single classroom in half and field both surveys in the same class.
   • Although we would like to in the future, we do not field the YRBS to MS students.
      
6. New Jersey: In New Jersey, the Department of Health and Senior Services (DHSS) conducts the Youth Tobacco Survey and the Department of Education (DOE) conducts the YRBS.  In order to minimize school burden DHSS and DOE work closely to coordinate mutually exclusive samples for YTS and YRBS.  This ensures that no New Jersey schools are being asked to participate in both surveys and schools are more likely to participate.  The partnership between DHSS and DOE has worked well to get both surveys done.
    
7. North Dakota: The North Dakota Tobacco Prevention and Control Program has coordinated the administration of the YTS with the administration of the YRBS since 2003.  The surveys are conducted in the early spring of odd numbered years so we have been through three cycles now (2003, 2005, and 2007) and we are currently conducting our fourth.  We find that it really does help to reduce the “survey burden” on schools and we have achieved very high response rates each time.
    
   Schools are randomly selected to participate and the same schools that participate in the YRBS also participate in the YTS.  However, randomly selected classes within these schools that complete the YRBS will not also complete the YTS in the same year.  Also, schools in North Dakota have the option to voluntarily participate in the YRBS but there is currently no option to volunteer for the YTS.
    
   In addition, our Tobacco Program has begun issuing a joint RFP with the Department of Public Instruction for joint administration of the two surveys.  That way we have one contractor administering both surveys and we find that it streamlines things very well.  That is not to say that two contractors/programs could not coordinate between themselves to administer the YRBS and YTS but this joint RFP seems to work very well for us.
    
8. Oklahoma: Oklahoma coordinated the YRBS and the OYTS in 2005 and 2007 and the current administration is a coordinated sample.  Coordinating the two surveys is highly recommended since it relieves the burden on the schools.  Oklahoma coordinates by school rather than classroom but this year we are also coordinating with the Cherokee Nation for both YTS and YRBS; they are doing coordination by classroom and their samples were drawn independently.  We also did coordination with the Cherokee in 2007 and their schools were coordinated as part of our sample.  Survey coordination takes more upfront work but pays off when you are in the field.
    
   By administering both surveys in the same semester you can:
    
   
   • Split up recruiting of survey administrators
   • Send one letter to school district superintendents
   • Conduct one training
   • Share people resources - e.g., a school refuses to participate and you do not have any contacts but the staff for the YRBS do
   • Share learned wisdom
   • Reduce the burden on the schools
   • Tasks can be set up based on established knowledge and relationships
   • Able to tell the schools that you will not bother them next year [different parts of health department conduct the surveys so schools often do not distinguish the difference]
   • Able to reinforce the importance of the other survey
      
   General benefits:
    
   • Develop relationships that can carry over to other projects
   • Develop good will with the schools because you are trying to reduce the burden
   • Potential Barriers:
   • If staff change you have to redevelop all of the relationships and re-educate the new staff
   • Personalities may clash [this is not unique to survey coordination but can make it very difficult to carry out because of the interdependence]
    
   Survey coordination can vary from getting a coordinated sample and then each party going their merry way to coordinating all aspects of the survey.  It is best when there is full coordination but this can be more difficult to achieve.  The greatest determinant of success is how well you play together.  It has worked wonderfully in years where everyone played well together and not as well in years where one or more individuals did not want to work together.  However, it has always worked.
    
9. South Carolina: The South Carolina Division of Tobacco Prevention and Control has coordinated with the State Department of Education to conduct the YTS and YRBS in 2005, 2007, and 2009. Using random numbers generated by Westat, generally, about 2 classes from each school were selected for each survey.
    
   Pros:
    
   • School participation is still an issue in South Carolina. We have seen that schools are more likely to participate when the request is made by the Department of Education.
   • Can compare YTS and YRBS results.
   • Reduced impact on schools' instructional program.
   • Easier to coordinate and administrate the surveys, since the YTS and YRBS coordinators work together and share responsibilities.
   • Diminished cost (mailing, telephone, etc.).
      
   Cons:
    
   • Lack of continuous data (no data for even years) - this is relevant for several measurements which are included in both surveys (i.e. smoking prevalence).
   • No direct comparison between state and nationwide data for the same year (National YTS is conducted in even years).
      
10. South Dakota: The Tobacco Control Program in South Dakota supports continuing to provide the YTS on even years and YRBS on odd years. It is difficult enough to find administrative support to complete one tobacco survey per year; if combined we will have much resistance to overcome from school administration as they feel that they are surveyed far too often. There are a number of other surveys that schools need to complete each year in addition to the tobacco surveys we request, and we need to respect the significant staff time required to administer each survey. We would strongly urge keeping to the same survey schedule and NOT change to the proposed combined survey schedule!
     
11. Wisconsin: Wisconsin conducts the YTS on even years, the YRBS on odd years.  We struggle to reach our sample each year in order to have enough cases to analyze.  If we split the sample into half YTS and YRBS, there would be concern about the sample size and getting enough schools to participate.

Back to Table of Contents