Publication Date
1-1-2023
Document Type
Article
Publication Title
International Journal of Law and Information Technology
Volume
31
Issue
3
DOI
10.1093/ijlit/eaad023
First Page
186
Last Page
212
Abstract
The Internet has become an increasingly attractive location for collecting data about cyber threats, driven by the abundance of quality data available and accessible online. As such, researchers and practitioners have turned to automated data collection technologies (ADCT), including ‘web crawlers’ and ‘web scrapers’, to study these threats. The rapid proliferation of ADCT has meant directions for their ethical and legal operation have been slow to adapt, with no clear guidelines regulating their use for research. This article identifies the relevant ethical and legal frameworks guiding the deployment of ADCT in Australia for cybersecurity research. This is accomplished through a systematic review of research within this context, coupled with ethical and jurisprudential analysis. We argue that the use of ADCT can be both ethical and legal, but only where mitigating measures are implemented. We provide a series of practical directions to guide researchers and practitioners when navigating this novel terrain.
Keywords
Cybercrime, Cybersecurity, Ethics, Web crawl, Web scrape
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Department
Justice Studies
Recommended Citation
Katie Logos, Russell Brewer, Colette Langos, and Bryce Westlake. "Establishing a framework for the ethical and legal use of web scrapers by cybercrime and cybersecurity researchers: learnings from a systematic review of Australian research" International Journal of Law and Information Technology (2023): 186-212. https://doi.org/10.1093/ijlit/eaad023
