Video-based test questions: a novel means of evaluation.

Video-based test questions: a novel means of evaluation. Many psychology instructors present videotaped examples of behaviorat least occasionally during their courses. However, few include videoclips A short video presentation. during examinations. We provide examples of video-based questions,offer guidelines guidelines,n.pl a set of standards, criteria, or specifications to be used or followed in the performance of certain tasks. for their use, and discuss their benefits anddrawbacks. In addition, we provide empirical evidence to support the useof video-based test questions. The data indicate that students preferredvideo-based questions to multiple-choice mul��ti��ple-choiceadj.1. Offering several answers from which the correct one is to be chosen: a multiple-choice question.2. questions on a variety ofoutcome variables, and data suggest that student learning may beenhanced. The use of video-based test questions that is discussed inthis paper can be applied to a variety of educational disciplines andlevels. ********** Most instructors agree that high-quality examinations in coursesshould serve the dual goals of instruction and evaluation (Renner Ren´nern. 1. A runner. &Renner, 1999). Traditionally, these goals have been served by six typesof questions: multiple-choice, true-false, matching, fill-in-the-blank,computation-based, and essay. We propose the inclusion of questionsbased on video clips in which students respond to behavioral behavioralpertaining to behavior.behavioral disorderssee vice.behavioral seizuresee psychomotor seizure. data duringexams. In this article we provide examples of video-based questions inthe field of psychology, offer guidelines for their use, discuss theirbenefits and drawbacks, and provide data supporting the utility ofvideo-based questions. We have opted to present the idea of video-basedassessment through the lens of psychology. However, video-based testquestions could be utilized in any discipline relating to relating torelate prep → concernantrelating torelate prep → bez��glich +gen, mit Bezug auf +accbehavior(e.g., ethological sections of biology courses, education courses) andbe adapted to any educational level. Given the fundamental basis of behavior in psychology, it isinteresting that instructors show videos during non-exam class sessionsto demonstrate psychological principles, but do not employ video-basedquestions on exams. There is no mention of video-based questions inmajor pedagogical ped��a��gog��ic? also ped��a��gog��i��caladj.1. Of, relating to, or characteristic of pedagogy.2. Characterized by pedantic formality: a haughty, pedagogic manner. books (e.g., Lowman, 1995; McKeachie, 2002), as wellas in articles that promote the use of videos for psychology courses(e.g., Boyatzis, 1994; Green, 2003; Hollander, 2000). Video-basedquestions might fall under the umbrella of performance tests (Davis,1993), although a more typical member of this category would belaboratory tasks. We have successfully used video clips on exams that (a) werealready viewed by students in class when originally learning material,(b) were based on lectures and discussion but had not been seen bystudents, and (c) were similar but not identical to video clips whenoriginally learning material. In a developmental psychology developmental psychologyBranch of psychology concerned with changes in cognitive, motivational, psychophysiological, and social functioning that occur throughout the human life span. course, forexample, the first author presented video clips of the same infantengaging in classic Piagetian Adj. 1. Piagetian - of or relating to or like or in the manner of Jean Piaget tasks over the first two years of lifeduring non-exam class sessions. For the exam, students viewed one ofthese clips and applied their knowledge of assimilation AssimilationThe absorption of stock by the public from a new issue.Notes:Underwriters hope to sell all of a new issue to the public.See also: Issuer, UnderwritingAssimilation andaccommodation via essay to the presented behavioral observations, aswell as answered a multiple-choice question inquiring inquiring,v to draw information from a client—whether by verbal questioning or physical examination—to assess the person's state of health. about theinfant's specific stage of sensorimotor sensorimotor/sen��so��ri��mo��tor/ (sen?sor-e-mo��ter) both sensory and motor. sen��so��ri��mo��toradj.Of, relating to, or combining the functions of the sensory and motor activities. development. In anintroductory psychology exam, the second author presented a clip thatstudents had never viewed of a patient displaying symptoms ofschizophrenia schizophrenia(skĭt'səfrē`nēə), group of severe mental disorders characterized by reality distortions resulting in unusual thought patterns and behaviors. and asked students to list the symptoms of the patientalong with examples of his behavior to substantiate To establish the existence or truth of a particular fact through the use of competent evidence; to verify.For example, an Eyewitness might be called by a party to a lawsuit to substantiate that party's testimony. the symptoms theylisted. In a different introductory class session, the second authorpresented a video of a split brain patient named "Joe"answering questions about visual images presented to either his left orright visual fields. During the exam, the instructor showed a similarvideo of a split brain patient named "Vicki" and askedstudents to explain the results of her test. Video-Based Questions: Guidelines, Benefits, and Drawbacks Instructors should consider several factors when employingvideo-based questions. First, to conserve class time, clips ofbehavioral data should be no longer than 2 or 3 minutes. Second,instructors should choose clips for which students must attend to thewhole rather than the details because clips that require carefulscrutiny generally require multiple viewings (an often distracting dis��tract?tr.v. dis��tract��ed, dis��tract��ing, dis��tracts1. To cause to turn away from the original focus of attention or interest; divert.2. To pull in conflicting emotional directions; unsettle. circumstance Circumstance or circumstances can refer to: Legal terms: Aggravating circumstances Attendant circumstance when some students are trying to write answers to theexam). Finally, instructors should tailor A tailor is a person whose occupation is to sew menswear style jackets and the skirts or trousers that go with them.Although the term dates to the thirteenth century, tailor the format of the questionsthey use about the video clips to the type of information they wish toask students. Although we have used multiple-choice questionssuccessfully, we find that short-essay questions are often mostappropriate as they require students to more fully explain and applypsychological concepts to given behavioral clips. Video-based questions may offer several advantages. First, we havenoticed that students attend to video clips during class much more whenthey know that they will see clips during an examination and will haveto evaluate them in relation to a psychological principle. Thus,students may perceive video clips as important and increase theirmotivation to understand the behavioral data. Second, some students perceive themselves to be "visuallearners." Students often report that they appreciate the use ofvideo-based questions because such questions allow them to demonstratetheir knowledge by means other than traditional questions. Third, video-based items arm instructors with an additional meansto evaluate students' understanding of material in a novel context.For example, when discussing behavioral neuroscience neu��ro��sci��encen.Any of the sciences, such as neuroanatomy and neurobiology, that deal with the nervous system.neurosciencethe embryology, anatomy, physiology, biochemistry and pharmacology of the nervous system. , an instructor inclass may discuss how strokes affect human behavior. For an exam, theinstructor may play a clip of an individual with slurred slur?tr.v. slurred, slur��ring, slurs1. To pronounce indistinctly.2. To talk about disparagingly or insultingly.3. To pass over lightly or carelessly; treat without due consideration. speech andweakness on the right side of the body and ask students to explain whythe stroke affected the patient in this manner. This type of questionrequires students to apply their knowledge of behavioral neuroscienceoutside the context in which they originally learned it. Armed with theability to analyze behavior that they have never before encountered,they may have increased confidence to apply psychological concepts totheir observations outside the classroom. In contrast, using video-based items involves some drawbacks.First, although instructors can begin using video-based items with asmall library of clips, they should optimally have a large video-clipcollection as they may wish to use one video clip during a class whenstudents learn material and a different clip on the same topic for avideo-based test item. This problem becomes easier to overcome each yearas more publishers provide short clips with their textbooks and severalarticles provide video sources for psychology courses (e.g., Boyatzis,1994; Fleming Flem��ing, Sir Alexander 1881-1955.British bacteriologist who discovered penicillin in 1928. He shared a 1945 Nobel Prize for this achievement. , Piedmont Piedmont, region, ItalyPiedmont(pēd`mŏnt), Ital. Piemonte, region (1991 pop. 4,302,565), 9,807 sq mi (25,400 sq km), NW Italy, bordering on France in the west and on Switzerland in the north. , & Hiam, 1990; Green, 2003). Second,video-based items can detract from detract fromverb 1. lessen, reduce, diminish, lower, take away from, derogate, devaluate verb 2. the self-paced nature of exams anddisallow To exclude; reject; deny the force or validity of.The term disallow is applied to such things as an insurance company's refusal to pay a claim. students to review the clip once the instructor is finishedshowing the clip. We typically show the videos 10 minutes into the examto avoid problems with late-arriving students. Finally, perhaps the mostserious drawback DRAWBACK, com. law. An allowance made by the government to merchants on the reexportation of certain imported goods liable to duties, which, in some cases, consists of the whole; in others, of a part of the duties which had been paid upon the importation. is that it is very difficult to present the video-baseditems during "makeup makeupIn the performing arts, material used by actors for cosmetic purposes and to help create the characters they play. Not needed in Greek and Roman theatre because of the use of masks, makeup was used in the religious plays of medieval Europe, in which the angels' faces " exams. We tend to use different videoclips for makeup exams along with a different version of the entireexam, but this practice will differ, of course, depending oninstructors' policies. Student Data Outcome data support the use of video-based test items. First,students in an Introductory Psychology course (n = 38) and aDevelopmental Psychology course (n = 23), taught by two differentinstructors, anonymously completed a questionnaire to assess theirperceptions of video-based questions. Students indicated their agreementwith four statements each for video format questions and multiple-choicequestions, using a Likert scale Likert scaleA subjective scoring system that allows a person being surveyed to quantify likes and preferences on a 5-point scale, with 1 being the least important, relevant, interesting, most ho-hum, or other, and 5 being most excellent, yeehah important, etc (1 = strongly agree, 3 = neither agreenor disagree, 5 = strongly disagree). Student responses did notsignificantly differ between the courses. With question format(video-based vs. multiple choice) as the independent variable andstudents' ratings as the dependent variable for each question,repeated measures ANOVAs indicated that students thought the video-basedquestions deepened their level of understanding of the material,assessed their understanding of "real-life" behavior, were afair assessment of their knowledge compared to the multiple-choiceitems, and recommended that video-based questions be included on futureexams (see Table 1). The second piece of evidence supporting the use of video-based testitems comes from a quasi-experiment in which students from oneIntroductory Psychology course (n = 39) were instructed at the beginningof the last unit of a semester-long course that there would bevideo-based questions on the exam, whereas students in anotherIntroductory Psychology course (n = 34) did not receive theseinstructions (although these students were told that they wereresponsible for all information presented during class). The sameinstructor taught both courses presenting the same material and theclasses did not differentially perform on examinations earlier in thesemester se��mes��ter?n.One of two divisions of 15 to 18 weeks each of an academic year.[German, from Latin (cursus) s (all ps > .05). Participants in both classes received thesame exam, a portion of which included 12 video-based test items asextra credit (essay, short-answer, and multiple-choice) derived fromthree different video clips that the students had never seen in class(thus requiring students to extend their analysis to novel clips). Allof the students in this study attempted to answer all of the video-basedquestions. Students told at the beginning of the unit that their examwould include video-based items correctly answered more (M= 9.95, SD =1.49) video-based items than did the control group (M = 8.47, SD =2.78), F (1,71) = 8.33, p < .01, [[eta].sub.2] = .11. Although thisstudy must be interpreted with caution, the study provides initialsupport for including video-based questions on exams. The above data constitute a first step in understanding the utilityof video-based questions and raise several important issues. Forexample, it is uncertain whether video-based questions or traditionallyformatted questions better distinguish between students who cansuccessfully apply psychological concepts to real-life behavior. Inaddition, it is unclear whether improved performance on video-based examquestions translates into more sophisticated analysis of behavioroutside the classroom. These and other important issues related tovideo-based questions deserve future empirical attention. Conclusion Instructors can readily employ video-based questions in a varietyof disciplines including education, biology, social work, and, perhapseven physics and chemistry. Although there are drawbacks, we believe thebenefits outweigh out��weigh?tr.v. out��weighed, out��weigh��ing, out��weighs1. To weigh more than.2. To be more significant than; exceed in value or importance: The benefits outweigh the risks. the problems. Instructors who use video-basedquestions have a unique opportunity to write creative and validquestions to assess their students' understanding of importantconcepts in relation to real-life behavior. References Boyatzis, C. J. (1994). Using feature films to teach socialdevelopment. Teaching of Psychology, 21, 99-101. Davis, B. G. (1993). Tools for teaching. San Francisco San Francisco(săn frănsĭs`kō), city (1990 pop. 723,959), coextensive with San Francisco co., W Calif., on the tip of a peninsula between the Pacific Ocean and San Francisco Bay, which are connected by the strait known as the Golden :Jossey-Bass. Fleming, M. Z., Piedmont, R. L., & Hiam, C. M. (1990). Imagesof madness MadnessAlcithoedriven mad by Dionysus. [Gk. Myth.: Kravitz, 16]Alcmeondriven mad by the Furies. [Gk. Myth. : Feature films in teaching psychology. Teaching ofPsychology, 17, 185-187. Green, R. J. (2003, July). Teaching psychology through film, video.APS Observer, 16, 23-25, 28. Hollander, S. A. (2000, May/June). Hot off the press: Using popularmedia in instruction. APS Observer, 13, 22-23, 28. Lowman, J. (1995). Mastering the techniques of teaching (2nd ed.).San Francisco: Jossey-Bass. McKeachie, W. J. (2002). McKeachie's teaching tips:Strategies, research, and theory for college and university teachers(11th ed.). Boston: Houghton-Mifflin. Renner, C. H., & Renner, M. J. (1999). How to create a goodexam. In B. Perlman, L. I. McCann, & S. H. McFadden (Eds.), Lessonslearned: Practical advice for the teaching of psychology (pp. 43-47).Washington, DC: American Psychological Society. Matthew J. Hertenstein and Joseph F. Way and, Department ofPsychology, DePauw University DePauw University,at Greencastle, Ind.; coeducational; United Methodist; est. 1832, chartered 1837. The school opened in 1838 as Indiana Asbury College, and in 1884 the present name was adopted. . Correspondence concerning this article should be addressed toMatthew J. Hertenstein at mhertenstein@depauw.edu.Table 1Student Evaluations of Video-Based and Multiple-choice Test Formats Video-based Multiple- Format choice Format M SD M SDStudying for the video format(multiple-choice) questions on 2.38 1.10 2.80 1.17the exam deepened my level ofunderstanding of the material.Studying for the video format(multiple-choice) test questionsassessed my understanding of 2.36 1.18 3.44 .92"real-life" behavior better thantraditional testing methods(video-based questions).The video-based format(multiple-choice) exam questions 2.48 1.22 3.33 1.13are a fair assessment of myknowledge of the class material.I would recommend that the useof video-based (multiple-choice) 2.39 1.23 3.13 1.26exam questions be continued onfuture Eta- F(1, 60) squaredStudying for the video format(multiple-choice) questions on 4.86 * .08the exam deepened my level ofunderstanding of the material.Studying for the video format(multiple-choice) test questionsassessed my understanding of 24.54 ** .29"real-life" behavior better thantraditional testing methods(video-based questions).The video-based format(multiple-choice) exam questions 13.32 ** .18are a fair assessment of myknowledge of the class material.I would recommend that the useof video-based (multiple-choice) 9.40 ** .14exam questions be continued onfuture* p < .05 ** p < .01