Open Data + Robust Workflow: Towards Reproducible Empirical Research on Organic Data

  • No photo
    Heng Xu

    Co-Director of Privacy + Analytics Lab (PAL)
    Associate Professor of Information Sciences and Technology
    College of Information Sciences and Technology
    The Pennsylvania State University
    University Park, PA 16802
    Phone: (814) 867-0469
    Email
    Website

Bio

Dr. Heng Xu is an associate professor of Information Sciences and Technology at Penn State. She is the co-director of Privacy + Analytics Lab (PAL), an interdisciplinary research group focusing on the interplay between social and technological issues associated with information privacy. Her research group aims to understand privacy dynamics in the real-world population and integrate this understanding into technological design and system development. Her work has been published across different fields such as Business, Law, and Human-Computer Interaction. She was a recipient of the NSF CAREER award (2010) and the endowed PNC Technologies Career Development Professorship (2010-2013).

During 2013-2016, Dr. Xu served as a program director for several interdisciplinary research programs at the National Science Foundation (NSF). Much of her work at NSF focused on bringing the social, behavioral and economic sciences to address major challenges in Cybersecurity & Privacy. She has also served on a broad spectrum of national leadership committees including the National Privacy Research Strategy Forum (2014-2016), the Federal Cybersecurity R&D Strategic Plan (2016), and the National Academies Committee on Open Science (2017).

  • No photo
    Nan Zhang

    Co-Director of Privacy + Analytics Lab (PAL)
    Professor of Information Sciences and Technology
    College of Information Sciences and Technology
    The Pennsylvania State University
    University Park, PA 16802
    Phone: (814) 863-9463
    Email: nan@ist.psu.edu
    Website: https://faculty.ist.psu.edu/nan/

Bio

Dr. Nan Zhang is a Professor of Information Sciences and Technology at the Pennsylvania State University. Before joining Penn State, he was a Professor at the George Washington University and a Program Director in the Division of Information and Intelligent Systems (IIS) at the National Science Foundation (NSF). The focus of Dr. Zhang’s research is on databases, data analytics, and taking the data-science methodology to examine information security and privacy issues. His research is supported by NSF and the Army Research Office, and has received several awards, including the NSF CAREER award in 2008, Best Paper Awards from IEEE ICC 2013 and IEEE NAS 2010, the Best Student Paper Award from ACM CIKM 2013, and Best Paper Nominations from IEEE ISI 2015 and HICSS 2018. His work on technology transfer was also recognized by the GW Technology Transfer Innovation Price and the first place finish at the GW Business Plan Competition, both in 2012.

Abstract

Reproducibility is obtaining the same results when re-analyzing the same data. It can be used to confirm the findings reported in a focal study and serve as a preliminary step in the replication process. This presentation will discuss why reproducibility has become an important part of the research process, describe how to test it, report on findings from applying those tests to published work, relate their relevance to replication, and offer recommendations for the publication process.

DIGITAL READER: RESOURCES RECOMMENDED BY THE SPEAKER

Groves, M. R. (2011). Three eras of survey research. Public Opinions Quarterly, 75(5), 861-871.

King, G., & Zeng, L. (2005). The dangers of extreme counterfactuals. Political Analysis,14(2), 131-159.

González-Bailón, S., Wang, N., Rivero, A., Borge-Holthoefer, J., & Moreno, Y. (2014). Assessing the bias in samples of large online networks. Social Networks,38, 16-27.