Marseille 2007
| Abstract #401 - Using PDAs to Improve Data Quality in School-Based Surveys on Sensitive Topics | ||||
|
Session: 16.5: Methodology Matters (Parallel) on Monday @ 14.00-16.00 in 5 Chaired by Graham Hart, Dominique Costagliola Authors: Presenting Author: Dr Rick Zimmerman - PIRE, United States |
||||
| Additional Authors: Dr Arvin Bhana, Dr Pamela Cupp, Dr Matthew Courser, | ||||
| Aim: Our experience in collecting data in township schools near Pietermaritzburg has been that classrooms are very crowded; indeed, at baseline data collection, students often shared answers as many students sat at the same desk. At Time 2, we implemented a technique that our research group commonly used; multiple versions of the survey were created with questions in different orders. Prior to survey administration we shared with students the fact that there were various survey orderings. While that seemed to improve the quality of the data some, we felt there was still room for improvement. It was at that time that we began to consider the APDA audio-personal digital assistant) Our experience in collecting data in township schools near Pietermaritzburg has been that classrooms are very crowded; indeed, at baseline data collection, students often shared answers as many students sat at the same desk. At Time 2, we implemented a technique that our research group commonly used; multiple versions of the survey were created with questions in different orders. Prior to survey administration we shared with students the fact that there were various survey orderings. While that seemed to improve the quality of the data some, we felt there was still room for improvement. It was at that time that we began to consider the APDA method of data collection (audio-personal digital assistant), as Flisher and colleagues had been successfully doing with PDAs (without audio) in the Cape Town area in school-based data collections (Mukoma et al., 2004). We also spoke to Elaine Borawski, Case Western Reserve University, who received funding from NICHD to test audio-PDA technology. She experienced great success in terms of dealing with low literacy and in improving data quality (Trapl et al, 2005). Using audio capabilities within the Palm PDA platform, our programming team developed a survey that allowed participants to choose between two languages (Zulu or English) in both the screen text and sound components. | ||||
| Method / Issue: Our paper reports the results of a natural experiment to improve the data quality in student surveys in an ongoing study in township schools in Pietermaritzburg, South Africa. As part of our research, a survey was administered to the same group of students at three time points. At time 1, all students received an identical paper and pencil version of the survey instrument. To reduce sharing of answers by students, at Time 2, we administered multiple versions of the survey with different question orders. Prior to survey administration we shared with students the fact that there were various survey orderings. At Time 3 we administered the survey using audio-personal digital assistants (APDAs) that allowed participants to choose between two languages (Zulu or English) in both the screen text and sound components. | ||||
| Results / Comments: Results indicated significant improvement in both perceptions of the test-taking environment as well as reductions in missing data overall and for reports of sensitive information specifically. APDA use resulted in statistically significant increases in reported privacy and self-reported honesty on surveys over paper and pencil versions of the surveys. In addition, the APDA version of the survey was associated with less than 2% missing data for sensitive questions as compared to 7.5% when paper and pencil surveys were used. Finally, only 1% of respondents didn't get to the equivalent of the last page of the survey with the APDA method as compared to more than 3% with the paper and pencil method. | ||||
| Discussion: As found by a small number of other researchers to date, audio-assisted personal digital assistant data collection methods improve data quality when surveys are administered in crowded classrooms in a developing world context. Perceived privacy and reported honesty increase, overall missing data is reduced (as students can move more quickly through the survey as a result of some skip patterns as well as greater ease with audio enhancement) and missing data for sensitive questions is dramatically reduced. APDAs appear to be an outstanding alternative to paper and pencil surveys in developing world classroom situations, if some start up funding is available. | ||||
| Go Back | ||||
