Helping Give Away Psychological Science/Creating an Open Teaching Toolkit

Creating an Open Teaching Toolkit to Level the Playing Field for Access to Research Skills for Analysis and Presentation at Conferences

This is a project that was funded by the Association for Psychological Science and currently is getting implemented.

Background

Only one in 20 people alive today live in the United States. Of those, <1 in 50 pursues a graduate degree, and only a fraction of those are able to attend a “Research I” university. Those “fortunate few” have a tremendous adventure in terms of institutional resources, professional networks, research mentorship, and teaching support. For those on the outside, barriers to entry include expensive software licenses, firewalled networks, and difficulty accessing data.

Open Science and open-access have been striving to level the playing field and democratize access to materials and resources. Helping Give Away Psychological Science (HGAPS.org) is a 501c3 dedicated to bringing the best information about psychological science directly to the people who would benefit. Over the five years since incorporation, the organization has grown to more than 300 members and alumni (dashboard here), with an international Board of Directors. It also has an affiliated Wiki User Group. This all grew from prior grants of a few thousand dollars (listed here), including a prior APS Teaching award to UNC Chapel Hill, with prizes and pizza to engage students.

COVID-19 created both a crisis and an opportunity. Disrupted teaching, disrupted meetings…forced us to flip classes and convert materials to online formats. We have been experimenting with combining Wikiversity and OSF.io to replace Sakai/Blackboard/Canvas as a more open way of teaching. Our class and HGAPS meetings have been on a steep learning curve, exploring videoconferencing for synchronous activities, and how other free platforms can augment and improve the pedagogy in high-access ways.

The proposed project will gather teaching materials, convert them to open access formats, deploy them in an organized way that is freely accessible, and share them with a series of state and national conferences. The project uses objective metrics to track usage and success, and will share the results (many in near real time).

Project Goals

The APS funding will convert teaching materials from an Archival Data Science class, and the Evidence-Based Assessment (EBA) resources that HGAPS has gathered over several years into a suite of teaching resources to build a hybrid “Conference 2.0” Toolkit. Activities and pedagogical materials will use open-access online formats to create an enriched conference experience. The Conference 2.0 Toolkit will meet the needs of three audiences: students, teachers, and conference organizers. Engaging these three audiences will make high quality learning experiences more accessible and engaging, helping students to learn how to conduct statistical analyses and visualizations and present them at scientific meetings. Teachers will enjoy the activities and resources, as well. The conference-oriented toolkit components include innovative conference programming options, open source ways of packaging and sharing meeting products, and a suite of objective utilization metrics that are easy to share with membership.

The project’s methods, justification, and how they address an issue relevant to the field

The project will adapt materials from traditional classes and pilot projects, with the specific goal of creating a Conference 2.0 format that transforms access to the skills and resources, promotes experiential learning of open science principles and practices, and enhances the dissemination of conference materials. The toolkit will be designed to be crowdsourced and updateable, using open platforms (e.g., R, OSF.io, Zotero, Wikiversity) instead of commercial ones (e.g., SPSS, Dropbox, Endnote, Blackboard). As conferences adopt the methods, their events will further expand the library of examples and resources, also raising visibility and diffusing the innovation.

Feasibility is high because many of the raw materials have been developed and piloted in existing courses. The APS grant is investing in converting this from “walled-garden,” UNC and Lancaster University specific offerings into open source versions, with clear organization and access as Wikiversity teaching resources, a YouTube subchannel, and templates for conference-specific versions. Students are empowered to make, record, and upload their own presentations (with no added work for the instructors or conference organizers), using forms, templates, and automated processes to provide scaffolding.

	Stakeholder Audience
Product	Conference Organizers	Teachers	Students
R Notebooks	--	Teaching examples Teaching datasets (Turnkey ready)	Templates - code in speaker notes turns past posters into recipe book of examples
CC BY 4.0 Slides	--	Slide decks can be borrowed, remixed; shared back (with credit!)	Can take notes directly into personal copy of slides; can comment on master
Templates	Presentation examples	Teaching examples and exercises	Examples of prior student work; rubrics
YouTube How to Videos	How to submit to conference; What to expect with innovative formats	Flipped teaching resources (pre-recorded, modular)	Video reference library APS grant will make more modular, better indexed
YouTube Recording of Talks	New product for the conference (video library of poster session)	Gallery for course portfolio, rating exercises	Durable product with personal metrics for views
OSF.io Platform	OSF Meeting	Archives of past meetings provide more teaching examples, source code	Personal OSF project becomes “portfolio,” with unique DOI, R code and sufficient data, as well as copy of abstract, poster
Open-Access Data	--	Access to teaching data Free measures as PDFs, Qualtrics or REDCap import files R scoring, visualizations, psychometrics	Free open-access data for learning projects, secondary analyses
Pro Tips	Value add for membership	CC BY, CC0, R notebooks, code in speaker notes -- all easy ways of sharing	Learning good habits, working efficiently
Dissemination	New suite of conference metrics -- pages and videos added; views, downloads all easily available	Metrics for resources added (could add to “evaluation of teaching” in performance reviews)	Personal metrics for posters, videos, downloads

How the project advances high-quality, innovative teaching of psychological science

We repurpose tools not traditionally applied to teaching, such as the OSF Meetings platform, as well as open-access data and R notebooks to create teaching resources versus project-specific replications. We also combine the HGAPS EBA materials, repurposing them as teaching support materials. Funded by a series of small grants, including an APS teaching award, the EBA project now has 210+ web pages that would fill a 640 page printed book, with 300 PDFs for free measures and Qualtrics modules for more than 70 that can be imported by the student or instructor. The EBA pages have been viewed 300,000 times (metrics here), and are already on Wikiversity -- the Wikimedia Foundation’s core tool for supporting open teaching.

The project further innovates by using the same platforms to develop conference-specific resources that offer free options for online and hybrid programming. More subtly, because these are themselves open-access, adoption by conferences is a win-win that gives them more content and also enriches the open “library” of examples and templates for future students and teachers to use.

Impact the project will have on its target audience

The project will share new conference program formats and tools that augment traditional offerings, make them more accessible, and expand the open-access material generated by conferences.

The project improves access to more powerful statistical methods (e.g., worked examples for mediation, moderation, psychometric analyses), as well as making larger curated open-access data available. Students and teachers will have more and clearer examples and more scaffolding as they pursue secondary analyses.

Reliance on open source software promotes skill transportability and generalization. Learners will always have access to R, Zotero, OSF, and Wiki, whereas they will lose their institutional student licenses to commercial products when they graduate.

Finally, the focus on “Conference 2.0” as the teaching goal, and the adoption of these formats by multiple organizations, creates tangible products from learning activities. These give a focus to experiential learning activities, and level the playing field in terms of creating CV-ready research products and open science skills.

How the project speaks to unmet needs in the teaching and learning community and possibly assists international or other under-served populations

The potential benefits of adoption of the processes and products of the Conference 2.0 format include much greater access and engagement for traditionally underserved populations and international students. The curated open-access data has prioritized a focus on more representative samples (e.g., low income, community mental health, or international samples, inclusion of nonbinary gender). Similarly, the R code and teaching examples put methods for evaluating potential bias in research tools front-and-center as teaching and research topics (e.g., tests of statistical moderation--are the effects the same for different groups; multi-group factor analyses, and differential item functioning). Sharing these materials offers knowledge that has previously been siloed within labs or institutions and makes it globally, freely available.

The open-access platforms are internationally accessible, eliminate cost barriers, and the asynchronous projects (e.g., recorded videos, uploaded materials, individual “homework” assignments) avoid time-zone challenges to scheduling. The pilot activities have resulted in more than 160 student presentations at peer-reviewed national and international meetings to date, with participation from more than 13 different countries.

Drawing on the affiliations of members of the Board of Directors for HGAPS creates natural partnerships for several professional society meetings, many of which piloted aspects of the Conference 2.0 Toolkit at their own conferences. The partnership with the North Carolina Psychological Association in particular offers opportunities to engage smaller teaching colleges and different student demographics.

Similar resources or projects and how the current proposal adds to or differs from them

During the pandemic, conferences either cancelled or flipped to online. All of the teaching, HGAPS EBA, and Wikiversity conference pages were built and used in face-to-face traditional formats. Their online components were adjuncts--not the primary delivery format. The HGAPS YouTube Channel and recording lectures are new initiatives.

Other meetings have used some of the platforms, but none with an explicit connection to Open Science or Wikiversity. For example, the STEM Village organized a virtual symposium, where some presenters recorded their posters onto YouTube (link to example). Similarly, #LIBRE2020 had poster snapshots presented publicly on Twitter. We also differ via incorporating the OSF.io and Wikiversity elements.

Summary

The proposed project uses funding to focus activity on taking prior materials and pilots and building an organized, open-source platform that can augment existing conferences while creating shareable teaching materials that improve access to high quality data, analytic code and examples, and templates and support to transform the access of traditionally marginalized learners. The result will be a “Conference 2.0” toolkit that teaches open science practices through experiential learning in a way that credentials learners with peer-reviewed presentations and a portfolio of work that prepares them to engage with their personal and professional development.

External links for related resources

Here is the link to suggest videos to be uploaded to the HGAPS YouTube page.