Pokemon Go, Augmented Reality, and the Jewish Experience

My heart is in the east, and I in the uttermost west
-- Rabbi Yehuda Halevi, "My Heart Is In the East" 

The seventh day is like a palace in time with a kingdom for all. It is not a date but an atmosphere. It is not a different state of consciousness but a different climate; it is as if the appearance of all things somehow changed. The primary awareness is one of our being within the Sabbath rather than of the Sabbath being within us... The air of the day surrounds us like spring which spreads over the land without our aid or notice.
-- Rabbi Abraham Joshua Heschel, The Sabbath, Pg 21 

I remember how once, on the Day of Atonement, I went outside into the synagogue courtyard with my father [R. Moses Soloveitchik], just before the Ne'ilah service. It had been a fresh, clear day, one of the fine, almost delicate days of summer's end, filled with sunshine and light. Evening was fast approaching, and an exquisite autumn sun was sinking in the west, beyond the trees of the cemetery, into a sea of purple and gold. R. Moses, a halakhic man par excellence, turned to me and said: "This sunset differs from ordinary sunsets for with it forgiveness is bestowed upon us for our sins" (the end of the days atones). The Day of Atonement and the forgiveness of sins merged and blended here with the splendor and beauty of the world and with the hidden lawfulness of the order of creation and the whole was transformed into one living, holy, cosmic phenomenon.
-- Rabbi Joseph B. Soloveitchik, Halakhic Man, Pg 38

Augmented reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is modified (possibly even diminished rather than augmented) by a computer. As a result, the technology functions by enhancing one’s current perception of reality.
-- Wikipedia

It has been hard to miss the Pokemon Go craze this past month. The game has rocketed to the top of the app store and caused numerous reports of players getting into accidents due to the immersive nature of the game which is played outside on busy city streets. Players have invaded sacred spaces playing Pokemon Go like the US Holocaust Museum, Arlington National Cemetery, and Auschwitz which has sparked much consternation and debate. Numerous guides have been published to Pokemon Go both for the novice and advanced gamers. The Jewish blogging world has taken notice with posts about Pokemon Go and Halacha, ideas for using the game in Jewish education and musings on Pokemon Go and Torah.

The Magic of Augmented Reality

What makes this app so magical is its use of augmented reality to connect the real world with an enhanced virtual one. One can be walking down the street and a Pokemon pops up in the viewfinder of one's phone that one needs to capture using the Poke Ball. Local buildings double dip as Poke Stops where one can pick up extra Poke Balls. More advanced users can pit their Pokemon against each other in gyms which are usually more prominent local landmarks. For example, both my local synagogues are gyms which explains the random people who seem to hang out outside them glued to their cell phones.

A Pokemon Go Field Trip

I am hardly a hard-core gamer, I think I am on Level Two right now, but I have taken a few Pokemon Go field trips in my neighborhood. You can view pictures from one of my excursions below.



Pokemon Go can make any routine chore into an adventure. For example, on a recent trip with my daughter to the doctor, I whipped out my cell phone and opened the app. In the waiting room, my daughter caught a Pokemon in the waiting room. Then as we waited for the doctor to come to the examination room for what seemed like an endless time, she posed with a different Pokemon.

Look what my daughter caught at the doctor! #PokemonGo

A photo posted by Tzvi Pittinsky (@techrav) on Jul 26, 2016 at 9:27am PDT

Pokemon Go is likely a fad which might be supplanted by other apps even before the new school year. However, it illustrates the power of augmented reality. Augmented reality together with its even more immersive cousin virtual reality, made popular by the inexpensive Google cardboard headsets, has been all the rage in educational settings. I have blogged about using the Aurasma augmented reality app at our Evening of the Arts here and here.

AR and VR at ISTE

AR and VR as they are known were very popular subjects at the ISTE conference in the end of June. The line for Google's VR sessions were down the hall throughout the conference. I attended an excellent session on Google Cardboard and Virtual Reality Apps and an Augmented Reality tour entitled Walking the Augmented Reality Line. The tour gave participants the opportunity to leave the convention center and walk through Denver's beautiful downtown. Each of us practiced making Auras and we even made a 360 image on our smartphones using Google street view. Google has created guide to creating street view images and sharing them with others on Google Maps. My street view is embedded below. Note that since this was my first attempt at making a 360 image and I only used my phone, not a specialized camera like the Ricoh Theta, the image has some gaps and many people moved while I was taking the image which also created some gaffes but one gets the idea. I am sharing it, warts and all, for your benefit (and a few laughs).

AR, VR, and the Jewish Experience

But what does any of this have to do with Jewish education? As the people of the book, many of us prefer the old school approach of pen and paper to these flashy new apps. And as scions of a tradition dating back over three thousand years, new fads do not necessarily excite us. However, I believe that augmented and virtual reality are not just modern gimmicks but cut to the core of the experience of Judaism.

The Jewish experience has always fundamentally been one of augmented reality. How else can one explain the fact that in nine days from now we will be sitting on the floor, fasting and crying over a temple destroyed almost two thousand years ago. Tisha B'Av is not merely a commemoration of events from long ago. We re-experience those events as if they are occurring right now. The actions of fasting and mourning which begin three weeks prior to the fast and reach its crescendo on that sorrowful day, transform the reality in front of us. We are no longer enjoying the fun and sun of a lazy summer day but the sorrow of a people who throughout history have faced tragic events on these days.

What is true about the sorrow of the month of Av is true about our times of joy as well. Purim and the month of Adar in which it takes place is not just a joyous event from the past but through the cycle of the Jewish calendar, a continuously joyous time in the present as well. Rabbi Abraham Joshua Heche observes in his classic work on The Sabbath that Shabbat is not a just a day of the week but an atmosphere. When Rabbi Joseph Soloveitchik viewed the setting sun on Yom Kippur afternoon, he did not merely see a beautiful natural event but experienced an augmented reality, through the setting sun on Yom Kippur one is forgiven of all of their sin. The very day is imbued as one of forgiveness.

This I believe this is the promise of augmented reality in Jewish education. Rabbi Soloveitchik often bemoaned the fact that it is much easier for a teacher to transmit the knowledge of Judaism than to communicate the the emotional connection to Jewish practice, the Jewish experience. (One can read more about this in an article that I wrote a few years back in the journal Ten Da'at on The Role Of Teacher And Student In Jewish Education According To Rabbi Joseph B. Soloveitchik.)

Perhaps these new technologies can be a part of a solution to this. Immersive technology-enhanced Jewish experiences, either created by students or by Jewish educators, could become a part of the Jewish experience.

I recently had a phone conversation with someone who worked for a Jewish foundation. She wanted advice on the next big Jewish  educational technology initiative. After some words of caution about chasing the newest exciting technological device rather than sound pedagogy, I mused how great it might be if professional Jewish educators, tour guides, photographers, and Israeli students could created immersive virtual reality tours of all of the major sites in Israel for the benefit of students in the Diaspora. In the United States, when learning the Book of Joshua for example, I cannot take my students on a field trip to see the walls of Jericho. We cannot go see Jeremiah's scribe's autograph on a signet ring discovered at the City of David. But through AR, VR, and other technologies perhaps we can help fashion these experiences for our students.

This is one of the reasons that I am so passionate about Jewish EdTech. The people of the book also taught the nations how to experience the world, how to transform the reality they lived in. For thousands of years from all the corners of the world they pined for a land they could not see; they experienced the texture of the Sabbath and festivals, days that outwardly seemed like any other. This ability to augment the Jewish reality is perhaps the secret of Jewish survival.