Quizbank/Creating a bank so students won't ''break the bank''
A proposal by Guy vandegrift and some of his students to create an Open Source Exam Bank: If my experience in 1976 as a TA at U.C. Berkeley is included, I have been teaching college for 43 years. These days I routinely encounter two opinions about higher education that disturb me greatly:
- It has limited value.
- It needs to be expensive.
Although this effort is focused on Physics and Astronomy, I believe this and other methods[1] can be used to significantly reduce the cost of higher education.
An open source bank of exam and quiz questions
Share article
Email
| 
Facebook
| 
Twitter
| 
LinkedIn
| 
Mendeley
| 
ResearchGate
Presented at the Wright State University Lake Campus Symposium, April 18,2019
Kaitlin Break, Benjamin Davis, Shiane Deal, Skye Grube, Gregory Luchnenko, Aegis Oswalt, Britney Slater, Guy Vandegrift
In an era where the answer to almost any question is available on the internet, the testing of competencies across various subjects has become increasingly problematic. Additionally, tuition is tens of thousands of dollars when an online textbook barely costs fifty. This diminishes the effort to use technology to significantly reduce the cost of higher education by forcing students to pay to establish their competency in the subject. A partial solution is the creation of an bank of sample questions open to the students and large enough to prevent memorization. This bank could be used as a preliminary exam for entrance to a course; a grading system with a pass fail basis; provide a means for other students to tutor their peers; or even enable students to receive credit for writing the sample questions. In addition to the open bank of questions, each professor could maintain a private bank of questions not available to students.
More information by Guy Vandegrift
A user-friendly prototype exam-generating code has already been written: Links to an 8-minute video and accompanying pdf are found in the links to the right. A higher quality video is available at Youtube. Please like and share it:
Why we need this bank
Higher education is too labor intensive
When my mother studied math and then architecture at U.C. Berkeley in the late 1930s, during the Great Depression. Her family had no money, but she could borrow the tuition each semester, but could not renew the loan for the next semester until the first loan was paid off. She managed this by working part time as an usher in a movie theater. That was her student loan problem. What went wrong? The answer is simple:
- As technology reduces the costs of goods and services, the relative cost of skilled labor rises.
The solution is equally simple:
- Use technology to reduce the labor required to educate.
Not all problems should be "Google-solved"
In 1970 I was on the same campus that my mother attended, in the library , looking at a homework problem and wondering how I was going to calculate the length of the spiral track on a LP vinyl record. I decided to go home for the day, and at some point during that 15-minute walk, the answer hit me. I recently searched for this same problem, and with a proper choice of keywords, the following link appears near the top of the page:
Today, physics students are denied such an experience for virtually every homework problem one can imagine. The fact that I remember this from almost 50 years ago suggests that it was a valuable experience that taught me something: Sometimes a mathematical problem is best solved, not by calculation or careful analysis, but simply by pondering the question. Fortunately, it is possible to allow such experiences in the Google era: The same lockdown browser used to provide cost-effective testing can be used for questions like this. Just as traditional homework from the pre-Google era had a low impact on the grade, low-impact "group" quizzes would ensure that students are not combing the bank in order to guess what might be on today's group exercise. This way of flipping the classroom also minimizes the skilled labor costs that plague higher education today.
The cat is already out of the bag
Websites are now attempting to collect and post homework solutions to every problem in every textbook. Ideally, exam questions would be kept "secret" from students, but these websites are letting the cat out of the bag. Educators need to face this fact and respond by creating a bank that brings the cost of higher education and its accreditation in certain courses down to almost zero.
- External link: https://www.slader.com/
Not only homework solutions, but even exam questions are available online, and for a modest price:
- External link: https://testbankscafe.eu/
Evidence that this effort might succeed
A superficial look at the vast library of information freely available on Khan Academy and other OER efforts strongly suggests that the next logical step is an open source exam bank. It is less certain that this Quizbank effort will be that first step (I am working alone while teaching a typical professor's "full" load.) But there are two reasons to hope that things are beginning to happen:
- This Tophat pagelink Apparently TopHat found 140 astronomy questions on QB and used the Creative Commons license to offer these questions to customers. It should be noted that while TopHat does not charge students or instructors for using these questions, they do charge for the use of the CMS that supports the exams.
- This Pageviews Analysislink This page documents 3,552 visits to Quizbank/All questions in the past 6 months. Unfortunately I have no idea of the source of those pageviews, since the WMF is careful to protect the identity of its readers. I am certain that the page views did not originate from Lake Campus students because these students are directed to much more useful pages that list only the questions that might be in a given course: See for example this Phy2400 Studyguide.
- v:Calculus_II Pageviews gets even more pageviews. Although the questions are not in multiple choice format, I believe that standardized tests can be developed that permit automatic grading in a way that does not violate the intent of this course. Implementation of this "reform" in how we assess college calculus will take considerable effort, IMHO.
Promoting scientific literacy with "cluster questions"
Any effort to create an exam bank that is a useful OER must confront the following dilemma regarding introductory college courses:
- One cannot learn a subject without learning facts and vocabulary.
- Learning facts and vocabulary in order to pass an exam is worse than useless.
The traditional way to resolve this dilemma is for the instructor to play a cat and mouse game: The instructor introduces facts and vocabulary in order to induce higher-order thinking by the student. To probe whether this has actually been achieved, the student is then tasked with answering a question, solving a problem, or writing an essay. If all goes well, the student succeeds with the task, even though information needed for the exam was deliberately concealed in the lesson that preceded the exam. This forces the student to synthesize new facts, perhaps embellishing the vocabulary with subtle new meanings. This deliberate concealment of what will be on the exam, combined with the fact that students are usually kept too busy with other courses to independently learn what might be on the exam is essential to higher education.
I must confess ignorance as to how an Open Exam Bank could resolve all such dilemmas. But for the conceptual sciences courses (typically taken by non-science major), "cluster questions" based on expository prose might help. This is a common type of question for standardized exams (although others may attach a different name to this concept). The question begins with a short passage of expository prose, followed by a "cluster" of questions based on that prose. This might give us more flexibility regarding what students are expected to remember as they review the material. Also, keep in mind that while the bank is "open" in that students will soon deduce what excerpts of expository prose is likely to be on the exam, perhaps we will discover that creation of new fact-based questions based on this prose is not too difficult. Such questions will remain in the instructor's "secret cache" of questions that are not made publicly available. And ... a large bank of publicly available questions based on the same passage might keep students so busy that efforts to find or obtain the "secret" cache of questions will be inhibited.
Under ideal conditions, the student should be reading the passage for the first time as they take the exam. This will not be feasible in the early stages of an open bank's introduction. But as discussed at the end of the aforementioned "Quizbank Proposal Video", we can task students with finding articles licensed under Creative Commons license and creating fact-based questions based on that prose. It is my experience that students are not very adept at developing exam questions. However such activities might be suitable for advanced students of the subject, especially if they hope to someday teach it. A low-cost technology for assessing efforts by advanced students attempting to develop such "cluster questions" already exists.[2]
Students at Wright State University and I will be experimenting with cluster questions using one (or both) of the following collections of scientific prose intended for the non-expert:
- WikiJournal of Science/Radiocarbon dating (Mike Christie, et al.)
- Physics for beginners (Matthew Raspanti)
See also
- Quizbank/News
- Pageview Analysis (documents the monthly visits to Quizbank/All questions.)
- Quizbank/Flipped semester
- Quizbank/Creating a bank so students won't break the bank
- Quizbank/Cost-benefit analysis
- MyOpenMath/Solutions
- Calibrated Peer Review
Share article
Email
| Facebook
| Twitter
| LinkedIn
| Mendeley
| ResearchGate