Immersive Storytelling

Introduction to Immersive Storytelling – VR for the senses?


My interest in Immersive Storytelling and Immersive Technologies began just over a year ago, when I explored Isadora as a tool for creating an immersive experience. This was my first exploration into an experience that was almost fully dictated by the user. Despite being engaging and technologically interesting (and often enjoyable), it was purely a demonstration of the immersive technology, rather than any sort of narrative. My next exploration into interactivity was a web-based interactive documentary covering the topic of Synaesthesia. This became a more polished and engaging experience that had a clear story and defined narrative, yet retained interactivity and user choice. Whilst these two practices shared user engagement and choice, they were completely different interactions. My Isadora prototype was designed to be housed in a physical space, whereas my documentary was designed to be accessed from any internet enabled device.

According to Miller (p. 424, 2014), the types of immersive environment are as follows:

  • virtual reality (VR)
  • mixed reality
  • augmented reality (AR)
  • large-screen immersive experiences for audiences (also known as ride films and 4D dark rides)
  • large-screen immersive experiences for single participants
  • immersive multiplayer motion-based games
  • interactive theme park rides
  • full dome productions

I am thinking about what my next project will entail (likely Synaesthesia and/or sensory based again, due to the extensive amounts of narrative and research material I have conducted already). Additionally, I have begun to explore the use of ‘Mixed Reality’ (MR) environments as a medium to facilitate my narrative, whereby ‘digital technology is used in conjunction with physical props or in real-life settings to create a variety of unique experiences’ (ibid, p.432). Using our already sensory-enriched world could allow me to construct a strong narrative. However, Virtual Reality (VR), whereby ‘a 3D artificial world [is] generated by computers [to create] a world that seems believably real’ (ibid, p.525), is another medium that could facilitate a multi-sensory environment, as it may allow more control over the sensory responses I could elicit.

This blog post will explore two VR experiences, briefly covering the key areas of research and contextual debates around using VR environments to encompass a sensory narrative. The blog will discuss why VR will or will not become the storytelling medium for sensory-based narratives.

The first VR experience I am exploring is ‘Racing the King Ride’ (The Hatch, 2021). This VR documentary is a 5 minute narrative about some small islands off the province of Bohol, Philippines, which were impacted by a 7.2 magnitude earthquake, making the islands subside by 2 metres. This is a really effective use of VR, as there is not really any other medium that could elicit the emotion and presence of the story. Emotion is a wide spectrum, but in this case, it’s the elements of sympathy and empathy, which were used as a tool for the documentary. Looking into the children’s eyes, as they wave back at you, is unexpected; but this empathetic response could not be simulated in any other medium.

Child at COP26 waving to the children in the documentary

This shows the importance of using VR to elicit emotion. VR is often referred to as the ’empathy machine’ (Constine, 2015), as it is ‘a giant leap forward in mankind’s propensity for compassion. You don’t just walk in someone’s shoes, but see the world through their eyes. In essence, a virtual reality headset is an empathy machine’. By using VR , the gap between the viewer and the subject is getting closer, and we are ‘transform[ed] into a subject with whom [we] can relate’ and our attitudes towards the ‘presented individual’ (in this case the Children within the documentary) and the ‘respective group’ (the audience) are affected.

Reaching under the water whilst on a VR headset

Whilst the feeling of presence is ‘present’ in this documentary, I do feel this is an element where the experience falls short. I was lucky enough to help exhibit Racing the King Ride at COP26 in Glasgow, and I made an unexpected observation – people were trying to look under the water! This film was shot with an Insta360 camera on a fixed tripod. Whilst the experience was incredibly immersive, and the narrative, in my opinion, is perfected, to the point people were brought to tears, I do feel that this medium is missing any form of kinetic movement that could further enhance the story. This can be done, yet I am aware of the complexities surrounding this. However, considering the fact people thought you could go underwater really speaks for itself; you do feel like you are present – it’s just missing that final finesse.

The second experience I am exploring is ‘Notes on Blindness’.

“In 1983, after decades of steady deterioration, John Hull became totally blind. To help him make sense of the upheaval in his life, he began documenting his experiences on audio cassettes. These original diary recordings create the basis of this interactive non-fictional narrative which is a cognitive and emotional experience of blindness. Storytelling, art direction and graphical universe form a unique and singular immersion, completed by movement tracking, spacialized sound and controller interactions”. (Oculus, 2019)

Much like Racing the King Tide, this is an experience that could not be replicated using any other form of media. The experience attempts to simulate being blind to further contextualise the voice narration, and by using VR, our sense of sight is completely taken over, and – to a certain extent – removed. As McLuhan (2008) explained, ‘the medium is the message’ and in this case, the message that the medium of VR provides, is that all external senses and visions no longer matter outside of the headset. This shows that VR may be the best suited medium as a way of controlling and removing senses, if this was something I wanted to replicate in my own work. However, when trying to facilitate our existing senses, (in the sense of Synaesthesia, perhaps associating our current visual identifications with virtual and augmented audible representations), using VR may not be the best medium, as we would need to re-create our current vision virtually, which defeats the point anyway. Therefore, a mixed reality environment might suit any experience where all senses need to be present.

When trying Notes on Blindess for myself, I found myself having to re-watch the same note twice, because I wasn’t paying attention to anything John Hull said the first time. This was because I was so facinated by the visuals, exploring my environment and immersed in the surround sound scape. This is positioned as a VR mediation of a documentary, and perhaps as a relatively new VR user, the spectacle of the medium itself is distracting. After a study based on concentration in VR, Barreda-Ángeles, et al. (2020) concluded that ‘the immersive presentation elicits higher arousal and presence, but also lower focused attention, recognition, and cued recall of information’. I did feel present, emotionally aware and fully engaged, but I would struggle to recall anything John said. Maybe we need to adapt to this form of storytelling before it can be used as a way of teaching.

References

Constine, J., 2015. Virtual Reality, The Empathy Machine. TechCrunch, [online] Available at: <https://techcrunch.com/2015/02/01/what-it-feels-like/> [Accessed 13 November 2021].

Bujić, M., Salminen, M., Macey, J. and Hamari, J., 2020. “Empathy machine”: how virtual reality affects human rights attitudes. Internet Research, [online] 30(5), pp.1407-1425. Available at: <https://www.emerald.com/insight/1066-2243.htm>.

Oculus.com. 2019. NOTES ON BLINDNESS on Oculus Quest | Oculus. [online] Available at: <https://www.oculus.com/experiences/quest/1946326588770583/?locale=en_GB> [Accessed 13 November 2021].

McLuhan, M. and Fiore, Q., 2008. The medium is the massage. London: Penguin.

Barreda-Ángeles, M., Aleix-Guillaume, S. and Pereda-Baños, A., 2020. Virtual reality storytelling as a double-edged sword: Immersive presentation of nonfiction 360°-video is associated with impaired cognitive information processing. Communication Monographs, 88(2), pp.154-173.

Miller, C., 2014. Digital Storytelling. 3rd ed. Burlington: Focal Press.