Time
Start:
End:
Address
Room 464, UNSW Business School, UNSW
Description
January 20, 2020: Gilad Feldman
Topic: Mass Mobilizing For Collaborative “Credibility Revolution” Open-Science: Insights, Findings, And Opportunities From A Project Of 56 Pre-Registered Replications And Extensions In Judgement And Decision Making
Abstract:
Reproducibility and replicability are at heart of science, yet increasing evidence from recent years suggests that many of the findings in psychological science are irreproducible and non-replicable in what some termed as a “replication/reproducibility crisis” and a new “credibility revolution” calling for significant changes in the way we do science, to embrace open-science practices. Even if we adopt these changes, it will take us decades to address the crisis. I will argue that to successfully address the situation we must mass mobilize students and leverage our courses to implement collaborative open science with real impact. I have been implementing such a change in my taught social-psychology courses at the University of Hong Kong (HKU). We have concluded the 3rd semester of running a mass replication effort of impactful Judgment and Decision Making (JDM) literature classics (summary). Last year we completed 45 completed pre-registered replications (examples), and this year we pushed things further with 22 teams conducting replications of 11 new target studies, with 2 extensions for each article. We implemented a Registered Reports format (manuscript template, and supplementary template), with pre-registrations of code analyzing simulated datasets, advanced templates for all outputs, comprehensive updated guides, Slack open communications, and open peer-review with external reviewers from around the world, in collaboration with large-scale paid prediction markets. This semester, we also went beyond replications and extensions, to try out new directions: 1) Students conducted open-science assessments of impactful Registered Replication Reports articles and our students’ projects from previous years, testing, coding, and learning from others’ pre-registrations, open-science code/data sharing, and reports, 2) Students collaboratively wrote a book detailing the development of the credibility revolution, 3) Students designed and ran in-class experiment activities on their class-mates to gain hands-on experience with adjustment and implementation of classics for engagement in real-life. I will briefly present the process, findings, main insights, and my own take-aways, and will review open resources we developed such as course materials, collaborative guides, and tools that can be used by all in their research labs and teaching.
About the speaker:
Gilad is an assistant professor with the psychology department at the University of Hong Kong. Gilad’s work is in the areas of social-cognitive psychology and judgment and decision-making. Research focuses on the topics of biases and heuristics, especially regarding agency and action. In recent years, with the emergence of the credibility revolution (so-called “replication/reproducibility crisis”), Gilad has taken a special interest in the movement for advancing psychological science by giving workshops and seminars and guiding researchers and students about open-science, statistical tools and simulations using open-source software (R), meta-analyses, pre-registrations, replications, and reproducibility.
Open workshop materials
Open teaching materials
More details on Gilad can be found at http://giladfeldman.org
Additional information regarding this event will be updated here as it becomes available.
Topic: Mass Mobilizing For Collaborative “Credibility Revolution” Open-Science: Insights, Findings, And Opportunities From A Project Of 56 Pre-Registered Replications And Extensions In Judgement And Decision Making
Abstract:
Reproducibility and replicability are at heart of science, yet increasing evidence from recent years suggests that many of the findings in psychological science are irreproducible and non-replicable in what some termed as a “replication/reproducibility crisis” and a new “credibility revolution” calling for significant changes in the way we do science, to embrace open-science practices. Even if we adopt these changes, it will take us decades to address the crisis. I will argue that to successfully address the situation we must mass mobilize students and leverage our courses to implement collaborative open science with real impact. I have been implementing such a change in my taught social-psychology courses at the University of Hong Kong (HKU). We have concluded the 3rd semester of running a mass replication effort of impactful Judgment and Decision Making (JDM) literature classics (summary). Last year we completed 45 completed pre-registered replications (examples), and this year we pushed things further with 22 teams conducting replications of 11 new target studies, with 2 extensions for each article. We implemented a Registered Reports format (manuscript template, and supplementary template), with pre-registrations of code analyzing simulated datasets, advanced templates for all outputs, comprehensive updated guides, Slack open communications, and open peer-review with external reviewers from around the world, in collaboration with large-scale paid prediction markets. This semester, we also went beyond replications and extensions, to try out new directions: 1) Students conducted open-science assessments of impactful Registered Replication Reports articles and our students’ projects from previous years, testing, coding, and learning from others’ pre-registrations, open-science code/data sharing, and reports, 2) Students collaboratively wrote a book detailing the development of the credibility revolution, 3) Students designed and ran in-class experiment activities on their class-mates to gain hands-on experience with adjustment and implementation of classics for engagement in real-life. I will briefly present the process, findings, main insights, and my own take-aways, and will review open resources we developed such as course materials, collaborative guides, and tools that can be used by all in their research labs and teaching.
About the speaker:
Gilad is an assistant professor with the psychology department at the University of Hong Kong. Gilad’s work is in the areas of social-cognitive psychology and judgment and decision-making. Research focuses on the topics of biases and heuristics, especially regarding agency and action. In recent years, with the emergence of the credibility revolution (so-called “replication/reproducibility crisis”), Gilad has taken a special interest in the movement for advancing psychological science by giving workshops and seminars and guiding researchers and students about open-science, statistical tools and simulations using open-source software (R), meta-analyses, pre-registrations, replications, and reproducibility.
Open workshop materials
Open teaching materials
More details on Gilad can be found at http://giladfeldman.org
Additional information regarding this event will be updated here as it becomes available.