My response to Thought Questions vs. "Spit Back"

Rabbi Yaki Blau, a colleague of mine at The Frisch School has raised an interesting discussion on Lookjed about Thought questions vs. "spit back". You can read his query/comments and the ensuing discussion here. Rabbi Blau recommends exclusively using "spit back" questions on formal tests for various reasons. I just wrote a response that I am including in this blog for the benefit of my readers since it touches on an important educational topic that has some applications to technology that I discuss below. I would also recommend my posting on IBM's computer Watson, What Watson Can Teach Us, which touches on the broader issue of the place of knowledge vs. skills in this new age of computer databases, Watson, and Google.

Dear Shalom and List:
Regarding the query by my colleague at The Frisch School, Rabbi Yaakov Blau on Thought questions vs "spit back", I believe that the best answer, as is often the case, is there is a place for both of these types of questions on well written assessments.

My philosophy towards testing is greatly influenced by a seminal class that I took a number of years ago with Dr. Scott Goldberg in the Azrieli Graduate School for Jewish Education and Administration. We used the textbook, Classroom Assessment: Principles and Practice for Effective Standards-Based Instruction by JH McMillan which I would highly recommend. In the course, Dr. Goldberg advised that the teacher first identify Learning Targets for their course and then make sure that all assessments address at least some of these goals. This is similar to the Understanding By Design approach where you "Begin With the End in Mind", as Stephen Covey would say, or put another way, סוף מעשה במחשבה תחילה, as we describe the Shabbat in לכה דודי. The learning targets should include the areas of Knowledge and Simple Understanding, Deep Understanding and Reasoning, Skills, Products, and Affective Targets.

Obviously some of these targets would require "spit back" types of questions while others would call for more "thought" based questions. Even in reference to "spit back" questions, it is important to recognize that there are different types of "spit back". One can ask questions for simple knowledge like basic translation and information and one can ask more sophisticated deeper understanding "spit back" that requires the learner to follow the various stages in a logical progression like the back and forth arguments in a long Tosfot. This is the type of high-level "spit back" that I believe Rabbi Blau recommends for formal exams. However, if one's targets include skills like using keywords to read a Talmudic sugya or applying knowledge to new situations then "thought" type questions are also warranted whether in a formal test or using alternative forms of assessments as Rabbi Aaron Ross advocated.

Technology can greatly assist in constructing these alternative skills-based assessments for Talmud. For example, students can indicate their mastery of the Shakla Vetarya of a sugya by breaking down the stages of the Gemara and classifying them using a computer program like Gemara Berura. They can also read the sugya for the teacher to listen to later using Voicethread, a web-based app that allows students to record their voices using a computer's microphone . This is much more practical than the oral testing that Rabbi Blau advances but points out cannot easily be done in a Yeshiva Day School schedule. Since the teacher can listen to the Voicethreads at his/her leisure, these types of assignments can be assigned regularly. They can also include any Hebrew text including Talmud, Tanach, Rishonim, or Acharonim so they can be easly adapted for many levels of learners.

At the same time, when I have taught Gemara in the past, I also included skills-based "thought" questions on my formal tests as well by giving my students "unseen" Gemara texts containing the same keyword structures studied in class. My students were naturally worried about these "unseen" texts but soon realized that they were eminently doable since I only asked them to replicate exactly the skill learned in class; to use the keywords to explain what a new Gemara is doing in terms of unlocking the Shakla Vetarya, the back and forth of the debate, rather than decipher what an unfamiliar Gemara is saying in terms of the content of the "unseen" Gemara. This is an important skill for students to master so they can learn to "make a laining" on a new Gemara as we would say in Yeshiva, to gain the ability to independently read an unfamiliar Gemara. Since this is an important learning target, naturally I included this on my exams. I also included application questions on my tests as well, although I usually limited them to one or two questions at the end for many of the same reasons Rabbi Blau mentioned.

The bottom line is that one should test to the learning targets that one teaches. Therefore, I believe that a good assessment should include simple knowledge and deeper understanding "spit back" questions, skills based questions, and some application type questions as well. If a test is balanced and closely aligned to the clearly communicated learning targets then students will adapt to them and use these assessments to show their knowledge and grow in their understanding.

I welcome continued feedback on this most fruitful discussion.
Kol Tuv,
Rabbi Tzvi Pittinsky
Director of Educational Technology
The Frisch School