McMaster Child Health Conference

Authors

Abstract

Background: There is a recent and dangerous threat to online research and data collection: bots and bad actors. Bots (artificial intelligence (AI)) and bad actors (human participants who do not truthfully complete surveys) are growing in their strength and numbers when it comes to infiltrating studies. In the field of autism research, these present additional barriers to an already underrepresented population. As these fraudulent participants continue to evolve past current methods which aim to combat them, it is imperative researchers consider all options of prevention, detection, and elimination, without compromising their survey’s integrity or increasing participation barriers for valid participants. Objective: To synthesize current reviews of bot and bad actor prevention, detection, and elimination from online research surveys. Methods: A search of major databases was conducted for independent studies and reviews, resulting in 19 papers found to be most applicable. The literature was then summarized by what methods of bot and bad actor prevention, detection, and elimination were explored, how authors used each method, and the effectiveness of these. Results: The search resulted in 19 articles, 12 of which were independent studies which explained authors firsthand experiences dealing with bots and bad actors [1-12]. The remaining 7 were reviews which assessed common strategies for bot/bad actor prevention, detection, and elimination [13-19]. Of the independent studies, 2 focused on dealing with bad actors, 1 focused on bots, and 9 focused on both. For the reviews, none focused solely on bad actors, 1 focused on bots, and 6 discussed dealing with both. Conclusion: Current strategies employed to tackle bots and bad actors in online autism research is a complex and nuanced landscape. A synthesis of 19 relevant studies revealed several distinct approaches. However, none of these methods are completely effective in isolation. This emphasizes the necessity to combine multiple strategies to enhance their overall efficacy. Another recurring concern surfaces throughout the discussion: the imminent obsolescence of current strategies in the face of rapidly evolving AI capabilities. While these methods can have effective outcomes, the relentless progress of AI technologies poses a formidable challenge to their sustainability. Thus, it becomes evident that a dynamic and adaptable approach is needed. Researchers across disciplines must collaborate to find novel method combinations and novel strategies. As the use of online questionnaires in all research—especially studies on autism and other underrepresented populations—continues to grow, it is increasingly important to keep participation barriers low while collecting valid data.

1. Ryan Kennedy , Scott Clifford , Tyler Burleigh , Philip D. Waggoner , Ryan Jewell , Nicholas J. G. Winter (2020). The shape of and solutions to the MTurk quality crisis . Political Science Research and Methods, 8(4), 614-629. Cambridge University Press (CUP).

   Type: Article DOI: 10.1017/psrm.2020.6

   OpenCitations