Mapping the Unknown: Café Glaser-Müller An exploratory creative process for an interactive volumetric video experience Master’s thesis in Computer Science and Engineering BUDI SUGIANTO JAP Department of Computer Science and Engineering CHALMERS UNIVERSITY OF TECHNOLOGY UNIVERSITY OF GOTHENBURG Gothenburg, Sweden 2020 Master’s thesis 2020 Mapping the Unknown: Café Glaser-Müller An exploratory creative process for an experimental interactive experience BUDI SUGIANTO JAP Department of Computer Science and Engineering Division of Interaction Design Chalmers University of Technology University of Gothenburg Gothenburg, Sweden 2020 Mapping the Unknown: Café Glaser-Müller An exploratory creative process for an interactive volumetric video experience Budi Sugianto Jap © Budi Sugianto Jap, 2020. Supervisor: Thommy Eriksson, Department of Computer Science and Engineering Advisor: Gorki Glaser-Müller, GGM Films Examiner: Staffan Björk, Department of Computer Science and Engineering Master’s Thesis 2020 Department of Computer Science and Engineering Division of Interaction Design Chalmers University of Technology and University of Gothenburg SE-412 96 Gothenburg Telephone +46 31 772 1000 Typeset in LATEX Gothenburg, Sweden 2020 iv Mapping the Unknown: Café Glaser-Müller An exploratory creative process for an interactive volumetric experience BUDI SUGIANTO JAP Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg Abstract Café Glaser-Müller (working title) is an art project by GGM Film to present an interactive dance performance with three-dimensional volumetric video and virtual reality technology. This Master Thesis aimed to investigate the creative process that goes behind projects that use these technologies through embedded research, and analyse the experience to provide design recommendations. In the process of this thesis project, design theory and methods were employed and evaluated by collaborators at GGM Film. This resulted in a few playable VR prototypes of various fidelity levels, ending in a short experiential story prototype that became a proof-of-concept for the project going forward. Interviews were also taken in the beginning and at the end of the thesis to gain an insight into the creative processes used in film and theatre. From this experience, six design process recommendations were formed to advise interaction designers in similar spaces. Keywords: interdisciplinary integration, interaction design, design methodology, arts, virtual reality, volumetric video. v Acknowledgements Many thanks to Thommy Eriksson and his continuing patience and enthusiasm in supervising this thesis, to the folks at GGM Film for the opportunity to work with them, and to the friends who supported me throughout the project. Budi Sugianto Jap, Gothenburg, December 2020 vii Contents List of Figures xiii 1 Introduction 1 1.1 Project introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Project Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 About Café Glaser-Müller . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4.1 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4.2 GGM Film . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4.3 Kulturbryggan . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4.4 VR headset manufacturers . . . . . . . . . . . . . . . . . . . . 4 1.4.5 Volumetric recording technology providers . . . . . . . . . . . 4 1.4.6 VR Designers . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4.7 Thesis student (the author) . . . . . . . . . . . . . . . . . . . 4 1.5 Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Background 7 2.1 Term definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Technology background . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Background on GGM Film . . . . . . . . . . . . . . . . . . . . . . . . 9 3 Theory 11 3.1 Wicked Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2 Iteration Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4 Methodology 15 4.1 Theory on methods and frameworks . . . . . . . . . . . . . . . . . . . 15 4.2 Embedded research methodology . . . . . . . . . . . . . . . . . . . . 15 4.3 Project phases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.3.1 Early Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.3.1.1 Literature review . . . . . . . . . . . . . . . . . . . . 16 4.3.1.2 Opening interview . . . . . . . . . . . . . . . . . . . 16 4.3.1.3 Early prototyping . . . . . . . . . . . . . . . . . . . . 17 4.3.1.4 VR experience review . . . . . . . . . . . . . . . . . 17 4.3.2 Design Formation . . . . . . . . . . . . . . . . . . . . . . . . . 18 4.3.2.1 Design frameworks . . . . . . . . . . . . . . . . . . . 18 4.3.2.2 Sketching . . . . . . . . . . . . . . . . . . . . . . . . 19 ix Contents 4.3.3 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.3.3.1 Scene prototypes . . . . . . . . . . . . . . . . . . . . 19 4.3.3.2 Story prototype . . . . . . . . . . . . . . . . . . . . . 20 4.3.4 Process Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 21 5 Project Timeline 23 6 Execution Process 25 6.1 Early Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 6.1.1 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . 25 6.1.2 Opening interview . . . . . . . . . . . . . . . . . . . . . . . . 25 6.1.3 Early prototyping . . . . . . . . . . . . . . . . . . . . . . . . . 26 6.1.4 VR experience review . . . . . . . . . . . . . . . . . . . . . . . 27 6.2 Design Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 6.2.1 Design frameworks . . . . . . . . . . . . . . . . . . . . . . . . 28 6.2.2 Sketching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 6.3 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 6.3.1 Scene prototypes . . . . . . . . . . . . . . . . . . . . . . . . . 29 6.3.2 Story prototype . . . . . . . . . . . . . . . . . . . . . . . . . . 30 6.4 Process Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 7 Results 33 7.1 Early prototypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 7.2 Design workshops and sketches . . . . . . . . . . . . . . . . . . . . . 35 7.3 Scene prototypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 7.4 Final story prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 7.5 Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 7.5.1 Opening interview . . . . . . . . . . . . . . . . . . . . . . . . 39 7.5.2 Final interviews . . . . . . . . . . . . . . . . . . . . . . . . . . 40 7.6 Lessons learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 7.6.1 An interaction designer’s place in an art project . . . . . . . . 42 7.6.2 Methodology as an interdisciplinary communication tool . . . 42 7.6.3 The need for constant ideation in an art project . . . . . . . . 43 7.6.4 Risks in design goal formulation . . . . . . . . . . . . . . . . . 43 7.6.5 Potential design methods for future projects . . . . . . . . . . 44 7.6.6 Design potential of volumetric video in VR . . . . . . . . . . . 44 8 Discussion 47 8.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 8.1.1 Data gathering and analysis limitations . . . . . . . . . . . . . 47 8.1.2 Generalisability . . . . . . . . . . . . . . . . . . . . . . . . . . 47 8.1.3 Effects of the global pandemic . . . . . . . . . . . . . . . . . . 47 8.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 9 Conclusion 49 Bibliography 53 x Contents A Appendix 1: Prototype Screenshots I A.1 Early prototypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I A.2 Cafe scene prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . IV A.3 Chair scene prototype . . . . . . . . . . . . . . . . . . . . . . . . . . VI A.4 Story prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VIII B Appendix 2: Pitch Document XIII xi Contents xii List of Figures 3.1 "The creative process as an iteration between idea and material." (Dahlstedt, 2012) [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.2 The Wheel design lifecycle template from The UX Book. [2] . . . . . 13 4.1 Photo of the volumetric video recording set, mid-recording. . . . . . . 19 4.2 Photo of the maps used for the story prototype recording. . . . . . . 20 6.1 Screenshot of Depthkit, a volumetric recording solution, in use. . . . . 26 6.2 A photo of a bodystorming session. . . . . . . . . . . . . . . . . . . . 27 6.3 A photo of the sketching workshop while it was in session. . . . . . . 29 6.4 Photos of recording set for the story prototype, with updated setup. . 30 7.1 A screenshot of the first early prototype. . . . . . . . . . . . . . . . . 33 7.2 Photo of the final MDA diagram. . . . . . . . . . . . . . . . . . . . . 35 7.3 Photo of the sketches drawn in the sketching workshop. . . . . . . . . 36 7.4 Screenshots of the scene prototypes. . . . . . . . . . . . . . . . . . . . 37 7.5 A screenshot of the story prototype. . . . . . . . . . . . . . . . . . . . 39 xiii List of Figures xiv 1 Introduction 1.1 Project introduction Virtual Reality (VR) is technology that can "place" a user in virtual space and potentially let them interact with virtual objects and environments. With the intro- duction of significantly more affordable and effective VR equipment in recent years, this technology has lately been in a renaissance. New ways to use VR in differ- ent contexts and new design solutions to its limitations have been sought after and tested by designers in both research and commercial ventures over these past years. This thesis project aims to take part in this by investigating into the methodologies of designing VR experiences in an artistic context. This thesis project was a collaboration with the GGM Film project Café Glaser- Müller (CGM for short, working title), which is planned to be an interactive dance performance set in VR. The project is to make use of another techonology that is recently growing in popularity: volumetric video recording. The project aims to deliver an experience that features a dancer recorded in volumetric video and presented within a virtual space, to give a unique sense of presence and human connection to the subject. The director of this project is Gorki Glaser-Müller, who is experienced in creat- ing films, but not so much in the field of interactive digital experiences. This lack of experience proves to be an issue in forming a concrete idea on what should be in the experience, and how to create this experience to begin with. This issue is further ex- acerbated by the relative lack of widespread use and knowledge for the technologies in use, in general. The medium chosen for this project so results in an exceptionally vaguely defined possibility space, and a difficulty in finding a clear inspiration or direction for the project. Therefore, this thesis project aimed to assist the creative process for this experience by proposing interactive design methodologies that can inspire, prototype, and validate the artistic ideas behind the experience. 1 1. Introduction The research question proposed was: What design process recommendations can be made to interaction designers in an artistic volumetric video pro- duction? The sub-questions to this were: • What interaction design methodologies can be effective to use when working with artists from interdisciplinary fields? • What are the requirements and trade-offs for the use of currently available volumetric video solutions in a VR experience? 1.2 Project Outcomes This project took an ethnographic approach to design research, meaning that I was embedded into the team to assist in designing the VR experience as part of the research. The research outcome of this approach is the identification of lessons learned from hands-on experimentation with interactive design methodologies in an exploratory art project, and the formation of guidelines and recommendations for other similar design projects. This may be in identifying which parts of the design process proved to be effective, which proved to be unpopular, and other things learned from the knowledge base of the project collaborators from other fields. As part of the project, there was also an investigation into the design potential of the use of volumetric video in an artistic VR experience. This involved creating a few prototypes and proof-of-concept for ideas generated from the design process. These prototypes are not exhaustively illustrative of the design possibilities in general, but still may serve as design inspirations for other artists and designers. Furthermore, in creating said prototypes, there was also an assessment on a few available solutions for recording volumetric video and creating interactive experiences with them. This assessment is done on both the quality and their ease of use in the project. 1.3 About Café Glaser-Müller The Café Glaser-Müller project, initially called Inflykt, was started by GGM Film as an attempt in creating an artistic interactive digital experience with the use of outside-in volumetric video recording. This project was deemed to be fit for this thesis for a few reasons. Firstly, when this thesis started, the project was still in its early stages of conception. It was a good candidate for the thesis’ design research in that we were able to discuss and collaborate on the process methodology of the project near the start of the project. The project timeline also matched the planned thesis timeline quite well. Secondly, it was a project that was expressedly experimental. It was also planned to involve collaborators from different disciplines of art (film, interactive technology, dance, to name a few). These characteristics 2 1. Introduction presented a uniquely challenging design problem, which much can be learned from. For the thesis, I was embedded into the project as a collaborator to study the specifics of the technical design process. My aim was to find ways to effectively apply the design theory learnt from the Master’s programme, so my involvement was mainly in the recommendation of collaborative design methodologies and the development of interactive prototypes. Throughout the course of this project, five people were involved: • Gorki Glaser-Müller: film/project director from GGM Film (background in film, theatre, some VR video projects), • Linda Brelin: project manager (background in film, dance, theatre), • Julian Namroud: intern whose involvement unfortunately was limited to the early phase (background in film and dance), • Rebecca Alverfors: intern who joined in the design formation process (back- ground in theatre), and • Myself as an interaction designer. There were plans to involve at least a choreographer and a dancer, but circumstances had prevented their participation within the time of my thesis. Before the start of my participation on the project, there have been a few early concept artefacts made for it. The artefact that was the main focus of the project at that point was a pitch document for the experience (Appendix B) by Pina Bausch. In it, dancers would dance through the stage with their eyes closed, while other dancers would rush in and move chairs that have been strewn about out of their way. There is also a moodboard attached to the pitch, which illustrated a proposed look-and-feel for the experience. Further development of the project used this pitch document as a reference. 1.4 Stakeholders 1.4.1 Audience The end users for Café Glaser-Müller will be the people who will experience the VR dance performance. The expected main target audience is people who may attend film festivals and the film community in general, so the experience is meant to be inclusive for people both familiar and new to using VR technology. 1.4.2 GGM Film The owners of the Café Glaser-Müller project is GGM Film. The direction and responsibilities of the project ultimately remain under GGM Film’s discretion, de- spite this thesis’s input on their design and technologies. The quality of the end product is a main concern for them, so the thesis project itself had to be careful not to undermine this for the sake of experimentation. 3 1. Introduction 1.4.3 Kulturbryggan Funding for the Inflykt project came from Kulturbryggan, which means that they have some stake in the project to be culturally relevant enough to justify the funds. 1.4.4 VR headset manufacturers VR headset manufacturers and platform owners such as Oculus and Valve may have an interest in projects like this one to improve the standards of VR experience design. These projects may also legitimise and further market the use of VR technologies to their favour. 1.4.5 Volumetric recording technology providers Providers of volumetric video recording tools, such as volumetric video recording studios and Depthkit, have similar interests to VR manufacturers in the use of their technology in projects like this. Design decisions made in the project may also influ- ence the demand to hire the service of those providers, positively or negatively. Since the project did not exhaustively evaluate volumetric recording solutions, and since volumetric video production is a continually developing field, we are not presenting that part of results of this thesis as a complete and up-to-date recommendation on these services themselves. 1.4.6 VR Designers Designers of VR experiences may take away the learning points from this project to apply in projects of their own. This can be especially applicable to exploratory and experimental experience design, collaborative and interdisciplinary projects, and even interactive art projects in general. To that end, this report was written with the intent to present potentially useful information as easily readable. 1.4.7 Thesis student (the author) As a Master student of Interaction Design and Technologies, I had a particular vested interest in studying the design theory and the technologies applied in the project, which may have potentially biased me against the completion or quality of the art project itself. This also extended to the faculty (including the supervisor) and Chalmers University, who have a stake on the quality of the thesis for their reputation and the thesis’s resulting knowledge. 1.5 Ethics GGM Film was informed of and verbally consented to the nature of thesis projects upfront, in that they are of public interest and the knowledge will be indepen- dently disseminated. Since the project involved a limited number of people, with an importance on their personal background and responsibilities, anonymity could 4 1. Introduction be difficult to achieve. Each person involved and mentioned in this report were individually asked the degree to which they would like to be anonymised, to which they all have given their consent to be fully identified and pictured. Audio recordings were taken for the final interviews mentioned in the report, to be used as a record for analysis and to be deleted after the completion of the thesis. Participants were informed of this and gave their verbal consent in the recordings themselves. 5 1. Introduction 6 2 Background 2.1 Term definitions • Volumetric video: Video recordings that contain or is paired three-dimensional data, such as depth data. This can be used to playback the recorded objects as three-dimensional animated models. Volumetric video contrasts with motion capture in that it is generally in a video-like generic frame-by-frame format. • Three-dimensional (3D): This term is used loosely in this report, but gen- erally refers to three-dimensional representations of reality in a virtual context. This can mean a digital 3D model of an object or scene, or a volumetric video being rendered in a 3D engine. • Inside-out/outside-in recording: This generally refers to how volumetric video can be captured. Inside-out recordings use a camera that records vol- umetric video in front or around it, while outside-in recordings are captured by using multiple cameras around a specific object or scene. The latter can cover multiple angles of a captured object, while the former has more issues with visibility blocking (e.g. the other side of an object will not be captured on the video.) • Virtual Reality (VR): In this report, VR refers to the use of virtual reality headset technology that immerses its users into a virtual context. It may also be loosely used to refer to the virtual context itself (for example, a scene made in Unity that the user will be placed in). • Degrees of freedom (DoF): DoF refers to the types of head/viewport move- ments a VR user is capable of doing by one-to-one motion. Three degrees of freedom (3DoF) means that the users may rotate their head around (three an- gles of rotation). Six degrees of freedom (6DoF) means that on top of rotations, they may also move their head around (three additional axes of translation). 2.2 Technology background Volumetric video is a technology that allows us to capture video data that can be represented in a three-dimensional space, as opposed to the more common two- dimensionally projected videos. This contrasts with other 3D recording technologies such as motion capture, in that while motion capture usually records the spatial mo- 7 2. Background tion of body parts or specific points on the body, volumetric video generally captures frame-by-frame voxel-like (3D pixel) generic data. This technology has become more common and accessible in recent years, with the availability of consumer cameras such as Intel RealSense and Microsoft Kinect, which can capture depth data on top of conventional 2D video. Volumetric video production is still relatively non- standardised, compared to conventional film production. Part of this project was to figure out how it could fit into an experimental creative production, and what off-the-shelf solutions are available to use in such a production. Virtual reality, on the other hand, is a technology not quite related to volumet- ric video capture, but has also become more popular in recent years. With modern virtual reality headsets, people are able to experience 3D environments in a more immersive way. This has resulted in many kinds of experiences made for it, some of which also used volumetric video as part of the experience. A fairly well-known example of this is Zero Days VR, which is a documentary on cybersecurity using a stylised volumetric video capture of an interviewee at certain points of the experi- ence. This relates to the project in that it is a film-like experience, but it differs in aesthetics and design goals. Since widespread use of volumetric video both in and out of VR is still fairly limited, there seems to not be much existing literature or annotated designs using it in an artistic product. Much of the recent literature focuses on the technical aspects of the technology, and not so much the design theory for utilising the technology or its aesthetics. For example, Kämpe et al. (2016) describes research in efficient encoding of voxel data with similar effect to volumetric videos [3], while Sheikhi-Pour et al. (2018) proposes 2D video coding methods for volumetric video data. This suggests that there is still much research to be done in solving the technical problems of volumetric video technology. The difficulty in finding practical knowledge for the use of volumetric videos affects the development of projects like this, with the need for more bespoke solutions and testing. There hopefully will be more interest in design research into the technology in the future. There have been artistic projects that used the technologies in similar manners to CGM (as an experimental interactive dance piece). Dust (2016) [4] is an example of a project that also utilised volumetric recordings of dancers in a VR experience. Dust used full-body recordings of dancers to stylistically recreate their performances in a virtual environment. This differs to the CGM project in its aesthetics and use of stylistic rendering on the dancers. VR_I [5] is another contemporary dance project using motion capture and location-based VR to deliver a social virtual dance performance experience. There have been various VR dance performance projects using stereoscopic recordings of dancers around a camera, such as Fugue VR [6], Half Life VR [7], and GGM Film’s own Skid [8]. These generally use sterescopic recordings to deliver a three-degrees-of-freedom experience, supporting rotational but not translational movement of the viewer. The main problem with trying to learn from many of these projects is a lack of detailed annotations on the design and technologies used, and a lack of access to experience them ourselves (as most of them are presented at specific locations and times). There are still quite some research literature on the use of interactive technology 8 2. Background in interdisciplinary creative works. Gherghescu (2018) described a research project on performance design using interactivity as a component in scenography [9]. Ben- ford and Giannachi (2011) drew on their collaborative artistic works over several years and identified the trajectories within them, although they largely focus on the audience journey [10]. Latulipe et al. (2011) reflected on the timing of the introduc- tion of interactive technology into a dance production, noting that the production with the most parallel development of the technology throughout the production was certainly the most stressful of them all [11]. Bluff and Johnston (2019) described a qualitative and interpretive research on interactive theatrical projects they were involved in, which is similar to the goals of this thesis project [12]. In their research, they argued that interactive technology and its relationship to the productions evolve over time. They also described the use of improvisational methods that has parallels to forms of dance used by performers, and intra-production trajectories that maps their development from conception to final production. Notably, they observed that interaction with the systems shift from the improvisational to more formal, as the technology becomes more robust. Research into VR user experience are relatively more well-established. The use of frameworks related to design of presence in VR [13, 14, 15], interaction fidelity [16], and the uncanny valley [17, 18, 19, 20] can be found quite readily. Presence is described by Slater and Wilbur (1997) as ”a state of consciousness, the (psycholog- ical) sense of being in the virtual environment.” [13], which is an effect of virtual reality technology worth investigating for this project. The uncanny valley, which was coined in 1970 by Masahiro Mori [21], refers to the phenomenon where a resem- blance of human appearances that do not quite replicate real humans can lead to a negative reaction from its viewer, which is relevant to the aesthetics of human actors in volumetric video in VR. It is unclear how usable these frameworks could be in an artistic design process, as quantitative results and usability are not often the main goals of an artistic project. Some of them were considered during the course of the project, but was decided against as they do not seem to align with the needs of the project. That said, the effects of presence and immersion in VR was considered as something difficult to simulate outside of VR. 2.3 Background on GGM Film GGM Film themselves have made a few video works and prototypes with volumetric video and/or VR. One example is the aforementioned Skid, which is a video that can be viewed in 3DoF VR. There is another volumetric video prototype called Beauty Vs Pain, which was made with a 360-degree camera. It allowed for 6DoF movement to view a recording of a dance with depth data captured inside-out from a central point, meaning that only the side of the dancer and environment visible from the camera is captured. In contrast to that, CGM was aimed to be an interactive 6DoF experience with a fully rendered 3D scene and a dancer recorded individually from the outside-in. 9 2. Background 10 3 Theory 3.1 Wicked Problem The problems of this project align with the definition of wicked problems as stated by Rittel and Webber (1973) [22]. There is no perfect solution, no enumerable set of potential solutions, essentially unique, and most importantly, likely impossible to understand the problem upfront. The lack of a perfect solution (or a test for a solution) is inherent to the artistic goal of the project, since it is not intended to solve a particular technical or social problem. Potential solutions are not enumerable as the possibility of using the medium for a creative work is too large to define. It is essentially unique as the results will be particular to the collaborators of this project (the director, the choreographer, the dancer, etc). The understanding of the problem depends on solving it, as we have no clear understanding of the possibilities for the project concept before we start working on it. The characteristic of it being a "one-shot operation" (where each attempt is consequential) also holds true. Part of the design problem is deciding on one specific creative direction for the experience to invest time and effort into, while foregoing other potential directions. It matters that the creative process progresses the design of the experience to a point where a satisfactory direction can be found before that attempt. To that end, an iterative design approach is to be taken to explore the possibility space for this project without committing into it. Because of these characteristics, it may be in question whether or not this design project will bear results that are generalisable for the design research and artistic community at large. The uniqueness and specificity of this project in particular may be counter to answering the research question in general, practical terms. As sug- gested by William Gaver (2012) [23], there is still a place for research through design in providing provisional, contingent, and aspirational theories and inspiring further research. To that end, extra care is to be given in documenting and annotating the process and results of this project, to provide provisional illustrations on what and why specific theories were formed throughout the project. 3.2 Iteration Process This project used the spatial model of artistic creativity proposed by Dahlstedt (2012) [1] as the theoretical inspiration for the goals of the creative process in this 11 3. Theory project. In this model, a creative process is described as "an exploration of a largely unknown space of possibilities" [1]. This exploration is represented by the possibility spaces of material space and conceptual space, each with unique topological path- ways, which the artist would explore and go back-and-forth between by forming a material representation of the conceptual work and re-assessing it. A diagram of this back-and-forth can be seen in figure 3.1. Figure 3.1: "The creative process as an iteration between idea and material." (Dahlstedt, 2012) [1] This model was adopted as it fits the needs of this project. First of all, the possibility space of the project is very vaguely defined to begin with, as the col- laborators are lacking in experience with the medium, and so needs to be mapped and explored first. Secondly, VR and other nascent technologies are mediums in which the material form and conceptual form can diverge in unexpected ways. In these ways, the model is a good fit as a theoretical basis for this technology-oriented process. For this exploration of possibility space, there are two things that need to be achieved in the project. One is to be able to quickly implement material repre- sentations (prototypes) of concepts, which is important in setting a rapid pace for design iterations. Second is to have a flexible set of tools for the prototypes, which will allow the mapping and exploration of the material space. For a relatively new technology, the topological pathways on the material space is not yet well defined, and so it is important to be able to map these potential pathways ourselves. These iterations generally followed the Wheel design lifecycle template proposed by Hartson and Pyla in The UX Book (2019) [2], shown in figure 3.2. Following this template is to have a step-by-step approach that guides the iterations in an 12 3. Theory orderly manner. In particular, the Prototype step is to transform concepts into the material space, and to Evaluate is to reconceptualize the work from the material representation. The larger phases of this project were based on the three steps of Analyze, Design, and Prototype, and these cycles also happened within each phase of the project (e.g. the analysis phase also included designing, prototyping, and evaluating prototypes). Figure 3.2: The Wheel design lifecycle template from The UX Book. [2] 13 3. Theory 14 4 Methodology 4.1 Theory on methods and frameworks The theory of design methodology in this space is not as well explored, and so a more creative approach was taken. Learning from informal discussions with the col- laborators, there are similar creative methodologies between the film and interactive design fields (e.g. the use of moodboards and storyboards), but their effectiveness in this experimental project is still fairly unclear. Bodystorming is an ideation and evaluation method considered to be a good fit for our project, since it encourages embodied interaction and empathy to users [24, 25]. This means to act out the experience using props through embodied movement, which fits with embodied interactions that can happen in VR. Since this is generally an exploratory artistic experience without clear use-case scenarios, this method was to be used loosely and closer to the Embodied Storming proposed by Schleicher et al. (2010) [24]. This means focusing more on ”acting first” and undirected role-playing as a source of analysis and inspiration. Since emotional impact is one of the stated goals of this project, design frame- works that deal with emotions in user experience design was deemed relevant to this project. Some examples of these that can be used are the MDA model [26] and Fokkinga and Desmet’s rich experience framework [27]. The MDA model was used as an early analysis and ideation method (further described in the second phase below). 4.2 Embedded research methodology A big part of this thesis is the embedded research approach taken. As a researcher, I was embedded into the VR project to provide input as a designer, with the intent of gaining an insight to the creative process through first-hand experience. This em- bedded approach has been used as a valid approach by other researchers in academia [28], and so was decided upon as a way to get an ”insider’s perspective” and work with professionals in a different field as per the topic. While this means that there can be potential conflict between what is practical and what is worth experimenting with for research, since the intent is to try out and learn effective design method- ologies, the methods in this project were proposed and tested in this project with the aim to fulfil both practical and the academic goals. 15 4. Methodology In carrying out this approach over the course of the project, extra documenta- tion on top of the ones taken for specific design methods was taken in the form of a personal project diary and photographs. These were used to capture my personal observations as an embedded researcher in a less structured but more rich manner, and were reassessed in the analysis and reflection phase at the end of the project. 4.3 Project phases The general structure of the thesis project loosely followed the Wheel template. The project started with an early analysis phase, to try to understand the project requirements and available tools. This was followed by a design formation phase for generating and solidifying design ideas through low-fidelity prototyping. This then moved onto prototyping at higher fidelity for more complete proof-of-concepts. At the end, there was an evaluation of the results of the project, to reflect on the project so far and discuss resulting design theories. Over the course of these phases, the iteration cycles in the Wheel template is also used as a reference for the smaller design iterations (i.e. from analysis to designing and implementing small prototypes, and evaluating them before moving onto the next phase). 4.3.1 Early Analysis 4.3.1.1 Literature review A literature review was done to gather existing academic knowledge that may be rel- evant to the project, extending to potentially useful design theories, research, other resources. This was carried out by searching relevant keywords on Google Scholar, IEEE Xplore, and the ACM Digital Library. Some examples of these keywords were ”virtual reality”, ”presence”, ”virtual reality ideation”, ”interactive dance”, ”volu- metric video”. The search results were manually selected by preliminary judgement on their potential relevance based on their titles and abstracts, and were then read through, noted, and lightly coded for reference. If they were personally deemed to be irrelevant, they were then removed from the review. The relevant knowledge gained from this review was then to be used in the project as described in the Background, Theory, and Methodology sections. 4.3.1.2 Opening interview To start with, the project needed to establish some basic design goals and knowledge background. A semi-structured interview was carried out with the director of the project (Gorki), regarding his creative experience and usual processes in previous projects. This was to gain an understanding of the gap between interaction designers and filmmakers, so that effective design methods could be identified and proposed depending on the needs of the project. The goals and stakeholder requirements of the projects was also established from this interview, to identify how the goals of the design process should reflect them. This interview was semi-structured, meaning that there was a pre-planned set of topics but free to diverge into less structured 16 4. Methodology discussions, because we were both not familiar with each other’s fields and persons to begin with. This interview was also not expected to be comprehensive from the get-go, so further informal discussions and inquiries could be carried out throughout the project to clarify the specifics or any misunderstandings. 4.3.1.3 Early prototyping The potential tools for the project were researched and analysed early in the project. This meant investigating potential technological solutions for the implementation of CGM, to identify what volumetric video solution and what VR set-up may be used for both the design process and in the final result. Since it is difficult to assess this on paper, some prototyping was necessary to directly evaluate the characteristics of each solution we found, through our own self-evaluations. These prototypes had to be at a high enough fidelity to get an idea of how they look in VR and how they would be implemented, so they came in the form of interactive scenes in VR that were tested with an HTC Vive headset. They were assessed and discussed in terms of development effort and budgetary costs, and the perceived qualities of their results. Said qualities for the volumetric video meant the look-and-feel, the fidelity, and other unexpected qualities that became apparent through the process. The assessments of these tools were important in terms of the spatial model of artistic creativity, as they will dictate the ”pathways” for exploring the material space for the project as per the Dahlstedt (2012) model [1]. The specific method used for this was a form of bodystorming. Simple proto- types with sample recordings from different solutions were made in Unity, and ran in development mode and modified on the fly. These bodystorming sessions were carried out with one person in VR and at least one person looking from outside VR on Unity. We discussed the different recordings in VR, and tried to play around with them with the Unity controls (e.g. changing in-scene positions, lighting, scale, etc.). This method was formed to allow for some freedom in testing the prototypes, while still ensuring that the prototypes are at a high enough fidelity to be effective for VR. In addition to bodystorming, in the process of making these prototypes, I also informally assessed the ease of use for each solution for the project. This partially informed the decision for which tools we planned to use for the rest of the project going forward. 4.3.1.4 VR experience review Along with the literature review, a review on existing VR experiences was also needed to be done. This was meant to get a sense of what is available out there, and to learn the standards and potential techniques used in similar/relevant VR experiences. There were two issues with this: one was that, similar to the issue mentioned regarding the literature review, it was difficult to ascertain what kind of VR techniques exactly would be relevant to the project. The second was that it was difficult to gain access to see similar VR experiences targeting a similar audience, as they are more often only shown on-site at an arts festival or performance venue at 17 4. Methodology specific times. Only some solo, unstructured testing was done on publicly available VR interactive experiences, while taking notes of critical incidents and personal observations. After the early prototyping phase, it was determined that the group collectively still had difficulties in distinguishing the design possibilities for the project. In terms of the spatial model, it felt as if it was difficult to see the potential pathways in the possibility space. It was decided that a group review of VR experiences was worth doing to alleviate this issue, as it may help everyone to get a more concrete mental image of what VR experiences could be like, and how existing ones have explored the medium. I prepared a set of VR experiences to let all collaborators try and discuss together, before moving onto more ideation. Notes on specific comments and critical incidents were taken for discussion in the following design formation phase. 4.3.2 Design Formation Following the Analysis phase was focusing on forming the design of the CGM expe- rience. After all the analysis work that has been done by this point, the main focus was to try out ways to form more concrete design ideas to form into material pro- totypes. This was done by holding several design workshop sessions, using different methods as appropriate. In a way, these workshops are to test the design methods used in them, with the collaborators as testers for these methods. To gather the tester feedback, when a specific design workshop was held, a quick unstructured group interview would also be held with the participants to gain some insight on what they thought about the specific design methodology employed in the work- shop. A structured interview would have made it quicker to gather this feedback, but an unstructured structure was decided on, as it was difficult to gauge how they would react about the methods before they used them. The interviews were infor- mally done with the general question "how did you feel about the workshop?" as the starting point. 4.3.2.1 Design frameworks A few methods and frameworks were proposed and tested in this phase, to be used as part of a more methodical approach in designing the CGM experience. One of these was the MDA framework. Although it was explained that it is a framework meant for game design, we tried to apply it to describe interactive experiences in general. A workshop session was held for presenting the MDA framework and trying to apply it in creating an MDA diagram of design ideas. It was done with the intent to create a more concrete theoretical basis for what the experience would actually entail, and to see the relationships (and possible conflicts) between our ideas. It was also meant to categorise the concepts we had already been discussing, and to present them in a more structured way. There were other frameworks that were proposed and lightly discussed, but not used extensively in any specific manner. They included Google’s Design Sprint Kit, Fokkinga and Desmet’s rich experience framework [27], and the Dahlstedt spatial 18 4. Methodology model of artistic creativity [1] that served as a basis for the process design theory. They were not directly used as they did not seem to naturally fit into the process. 4.3.2.2 Sketching A sketching workshop was held to form and show each other ideas in a more visual manner, and to decide on ideas we would like to prototype first. The sketching was done with the Crazy Eights method, emphasising rapid generation of ideas in a short amount of time. Making decisions on idea implementation can be difficult to do in the group, so a selection method was also used. We passed our sketches around and dot-voted by marking which specific sketches we find interesting. Any voted-on sketch was discussed as a group, and then finally decided on if we would like to implement something with it. All the resulting prototype ideas were then listed down. Figure 4.1: Photo of the volumetric video recording set, mid-recording. The volumetric video recordings based on the sketches were done in the studio for more rapid prototyping, with Depthkit and a green screen setup (fig. 4.1) as decided in the earlier prototyping phase. These recordings had to then be digitally cleaned and converted into volumetric video with a non-linear video editing software (for green screen compositing) and the Depthkit software. 4.3.3 Prototyping 4.3.3.1 Scene prototypes Following the ideation process, the list of ideas and video sketches were used to build two prototypes in parallel. This was proposed and decided on, so that the diverging ideas can be evaluated more fairly at a higher fidelity. These prototypes were built similarly to the earlier prototypes, but not including all of the features. The step back in features was to minimise implementation effort and potential bugs, especially when some features are more labor-intensive to produce (e.g. animating 19 4. Methodology collision objects for all the recordings). It was deemed enough to do a Wizard of Oz-style control of the playback in Unity instead, as it still allows for interaction and flexibility without as much effort. The evaluation setup for these prototypes was set up similarly to the previous bodystorming sessions, as it was already a familiar form of evaluation at this point. It was also a good fit since the prototypes still needed a Wizard of Oz, for additional interaction and to be quickly test or correct things. Unstructured discussions on the scenes were carried out throughout testing and after, both to gather feedback on the prototypes and to take note on new ideas. Impromptu questions were also asked to get more detailed responses from the testers, since they may not necessarily vocalise enough feedback during testing. 4.3.3.2 Story prototype Figure 4.2: Photo of the maps used for the story prototype recording. After the evaluation of the two scenic prototypes, one more higher-fidelity ex- periential prototype were built and evaluated. Since the director of the project was inspired from the evaluation of the previous prototypes, no further formal ideation process was carried out for this prototype. The intent for this prototype was to build a short dance narrative set in one of the scenes, from beginning to end. To make this prototype, a sketch of the experience was first drawn on what the director referred to as a ”map”, which can be seen in figures 4.2. It is a set of top-down diagrams used in the film industry to sketch where the actors, cameras, and other things would be placed in a scene. This method was suggested by the director as it is something that he was already familiar with, and had an edge over a storyboard as it is not focused on a single viewer’s point-of-view. These recordings were arranged and played in sequence in the Chair scene, and was then evaluated with the evaluation setup used for the previous prototypes. 20 4. Methodology 4.3.4 Process Evaluation Semi-structured interviews were carried out with the collaborators at the end of this thesis project, to reflect on the project so far. They were meant to qualitatively evaluate the effectiveness and efficiency of the interaction design methodologies em- ployed within the project so far, on top of getting general feedback on the project so far. Some topics discussed were the inspirational qualities of the methods for the project, and how they compare to the methods used in the other fields such as film-making or theatre. We discussed what we thought worked and did not work, and the potential in the technologies we used in the project so far. Semi-structured interviews were used for this again, as to let some freedom in drilling down into unexpected feedback and topics of discussions. 21 4. Methodology 22 5 Project Timeline The following lays out the timeline of the project schedule. This timeline has up- dated to align with the project diary, to show an overview of when and how long parts of the project took place. They are split into 4-week chunks for ease of reading, and to loosely show the distinct phases of the project. The report writing and anal- ysis is not included in this timeline, but took place alongside and after the project work. (Starting 4 February 2020) Weeks 1-4 (February): Literature review and planning Week 1 (4-7 Feb): Starting literature review, early discussion meetings. Week 2 (10-14 Feb): Surveying for volumetric video providers and resources. Week 3 (17-21 Feb): Reviewing literature. Week 4 (24-28 Feb): Finding thesis focus and project scheduling. Weeks 5-8 (March): Early analysis Week 5 (2-6 Mar): Replanning and carrying out opening interview Week 6 (9-13 Mar): Analysing and prototyping with volumetric video capture Week 7 (16-20 Mar): Setting up Unity scene, carrying out bodystorming workshop Week 8 (23-27 Mar): Further capture prototyping and bodystorming Weeks 9-12 (April): Design formation and prototyping Week 9 (30 Mar - 3 Apr): Planning meetings, MDA workshop, review showcase setup. Week 10 (6-10 Apr): MDA debrief, testing VR exps. to showcase. Easter break. Week 11 (13-17 Apr): VR showcase/review, sketching workshop, prototype recordings. Week 12 (20-24 Apr): Cleaning up recordings, setting up scenes (Cafe and Chair) Weeks 13-14 (April - May): Further prototyping Week 13 (27 Apr - 1 May): Finishing the two scene prototypes. Week 14 (4-8 May): Evaluations and bodystorming on Cafe and Chair scenes. Week 15 (11-15 May): Final prototype recording and interviews. Week 16 (18-22 May): Final prototype implementation and evaluation, and interviews. 23 5. Project Timeline 24 6 Execution Process 6.1 Early Analysis 6.1.1 Literature review The literature review was carried out in the first month of the project. It was difficult to tell at first, since this was an exploratory project, what type of knowledge would be relevant and how they would be useful, so an iterative approach to the review itself was taken to hone into it. Further searches were made based on the new knowledge from these references, including their references, related articles in the same publication, and new keywords (e.g. ”interactive theatre” and ”interactive performance” to explore further than just ”interactive dance”). These searches were done as potentially needed over the course of the review. This literature review was initially done with the assumption that the uncanny valley phenomenon would be a main focus of the thesis, but it became apparent over time that it may not have been a good assumption for a couple reasons. It may not necessarily be a good focus for the project, which makes it difficult to research, and the quantitative aspects of the research may not apply very well for the project. Because of this, the thesis was refocused in the midst of the literature review to focus on the design process, in the way this report presents now. 6.1.2 Opening interview The opening interview was carried out at the start of the Early Analysis phase, with the director as the interviewee. The semi-structured interview took about an hour to carry out in the meeting room at the GGM Film studio, with notes written down by the interviewer (me) while in the conversation. These notes were then reviewed and used as baseline knowledge for planning and carrying out the following design process. The pre-planned questions and topics for this semi-structured interview were: • Can you describe what forms of creative medium you have experience with? (Film, dance, theatre, VR videos, etc) • What sort of technology do you have experience working with? • How do you usually come up with ideas in the creative process? 25 6. Execution Process • How do collaborations usually work? Have you collaborated with a dance choreographer before? • How do you usually evaluate the results of your work? • In this project, who are the other stakeholders? Is there any specific goals or requirements that need to be satisfied for them? 6.1.3 Early prototyping Figure 6.1: Screenshot of Depthkit, a volumetric recording solution, in use. We discussed and looked into a few different volumetric recording solutions that may be applicable to the project. The solutions we ended up attempting to test were Depthkit (can be seen in-use in figure 6.1), EF EVE, 4DViews HOLOSYS, and Brekel PointCloud. Some test recordings were taken with Depthkit, but samples given by the solution providers were also used in the prototypes, for logistical reasons (e.g. HOLOSYS requires hiring their studio) and to see examples of what some ideal recordings would look like. A Unity scene with the SteamVR plugin was built for the these bodystorming sessions, using some free low-poly buildings and SteamVR assets to provide some scale to the recordings put in the scene. Sample recordings taken from a variety of volumetric video solution providers were placed in the scene when possible, to allow for direct comparisons between them. There were some difficulties with trying to put all of the samples in one scene, because of computing performance issues and conflicts between the Unity packages. The EF EVE samples had to be separated into its own scene, due to its usage of a different rendering pipeline. (Images of the prototype are included in the Results section and appendices.) Basic physics interaction was put in the scenes. This includes some physics collision between objects and the player’s hands in VR, the ability to grab objects using the controller trigger buttons, and the ability to teleport around in the area. An additional feature was also put in to allow the player to pause a recording by placing an object (including the hands) in the way of the volumetric recording. This 26 6. Execution Process was achieved by manually animating an invisible collision object along with the actor in the recording, which pauses the recording when it collides with a physics object. This was implemented by request out of curiosity (since it was one of the early ideas the director had), but was also useful to test what the recordings would look like when paused and played. Figure 6.2: A photo of a bodystorming session. There were multiple bodystorm testing sessions for different types of recordings used: • Test 1: sample high-quality Dephtkit recording, self-recorded Depthkit record- ing, sample HOLOSYS recordings. • Test 2: sample EF EVE recordings • Test 3: same as Test 1, with additional Depthkit recordings to test manually merged, independently recorded videos. These tests were each done on a different day, each taking about 30-40 minutes total, with the director and producer testing them in VR with an HTC Vive headset, and me on the laptop running it controlling Unity (see fig. 6.2, where I would be on the laptop if I was not taking the photograph). Group discussions out of VR were also held after the tests, to talk about the samples we just saw and what design potential we think is there for them. 6.1.4 VR experience review The first VR experience review was done on my own, with experiences such as Obduction, Until You Fall, Creed, Welcome to Light Fields, and Zero Days VR, each taking at least 15 minutes or more. I took notes of critical incidents and personal 27 6. Execution Process observations mostly after finishing my tests and getting out of VR, as it is difficult to write while wearing the headset. The second VR review (group review) was done at the GGM Film studio, with me setting up an HTC Vive headset and the experiences. The director, the producer, and the second intern participated in these reviews. The set tested includedMuseum of Symmetry, Invisible Hours, A Fisherman’s Tale, Superhot, and Blade Runner 2049: Memory Lab. Each was given around 15-30 minutes of testing, with free discussion among us both in the middle of and in-between testing. After the tests, we also had a small 10-minute discussion on them as a whole. Notes on specific comments and critical incidents were taken in writing. 6.2 Design Formation This phase was carried out at the GGM Film studio and involved four collaborators: the director, the producer, the second intern, and me as the designer arranging the workshops. 6.2.1 Design frameworks There was not a concrete plan on how long the MDA workshop will take, and it took about 2 hours total to carry out. There were some difficulties in explaining some parts of it, such as what Mechanics mean in the framework, so it took a little longer than expected to present at first. The MDA framework was first presented and explained to the participants using the original Hunicke et al. (2004) paper [26] and some examples from my previous projects as reference. Following the briefing, we attempted to model the concepts we have thought of onto a whiteboard diagram with post-it notes, using the framework. We individually wrote down on post-it notes and stuck them on a whiteboard where we thought it would go, and then discussed and drew the relationships between the concepts. The first round of post-its was stuck on the board silently, but discussions were allowed in later modifications of the diagram. The diagram was documented by taking photographs of it (fig. 7.2), and the rest of the feedback was noted down in writing. 6.2.2 Sketching We held the sketching workshop on a later day, and it also took about 2 hours. The participants were first briefed of what the sketches are for at the start of the session, and we proceeded to sketch. The sketching was done with the Crazy Eights method, where we each folded a piece of A4 paper into 8 sections, and timed ourselves to draw 8 sketches on them in 8 minutes. This sketching was done individually, and without discussion. After the sketching was done, we took turns to show and describe the sketches we had drawn to the group (pictured in fig. 6.3). To decide on which sketches we would like to discuss further and possibly implement, we dot-voted by passing our sketches around and marking the specific sketches we find interesting. Any voted-on 28 6. Execution Process Figure 6.3: A photo of the sketching workshop while it was in session. sketch was discussed as a group, and then finally decided on if we would like to implement something with it. All the resulting prototype ideas were then listed down. Three lists came out of this workshop: one listing specific virtual scenes for me to prototype, a list of specific volumetric video shots we would like to record and place into the virtual scenes, and a list of sounds to also place in the scene with the video recordings (see fig. 7.3 for the sketches and the list). The volumetric video recordings were done in the studio with Depthkit and a green screen setup (fig. 4.1) in a day, with multiple takes per idea. Ideas on the specificity of each idea arose during the filming process, so the different takes may vary technically beyond the natural differences in the human performance. In a way, this filming process unexpectedly turned into another sketching workshop. These recordings were then given to me to digitally clean and convert into volumetric video with DaVinci Resolve (for video editing and green screen compositing) and the Depthkit software. There were unexpected learning points gathered from this process alone, which are detailed in the Results section of this report. 6.3 Prototyping 6.3.1 Scene prototypes Following the ideation process, the list of ideas and video sketches were used to build two prototypes in parallel. The virtual scenes and assets were built in Unity, with the addition of 3D models taken from the Unity Asset Store and public do- main/CC0 sound assets from Freesound as needed. This included 10 Skyboxes Pack : Day - Night from Wello Soft, Snaps Prototype | Office from Asset Store Originals, Skybox Series Free from Avionx, footsteps on wood from Mydo1, Skipping heartbeat from under_the_hood, Woman reading a poem from LadyImperatrix, Breathing- WomNose.aiff from nickrave. The implementation was mostly done on my own, without much further input from the rest of the collaborators. Necessary details of 29 6. Execution Process the scenes that were undecided (for example, the lighting and the skybox used in the scenes) were single-handedly decided. These prototypes were missing the grabbing and direct pause/play collision mechanic, to minimise implementation effort and potential bugs. The first prototype was a relatively realistic scene called the Cafe scene, set in- side a cafe building with tables, chairs, and mugs. The second was a more abstract scene called the Chair scene, set on an enormous pile of gigantic chairs deep under- water. The volumetric video recordings and sounds were placed inside the scenes as deemed appropriate, played on a loop. Both scenes included basic physics collisions between the hand and the chairs. Further details on specific points in these scenes are also described in the Results section of this report. After making the prototype scenes, we met up once again to evaluate them. Only the director and the producer participated in this evaluation, and each scene also took about 15 minutes of testing. The test setup was similar to the previous bodystorming sessions: one tester was inside VR at a time while the others look on from the outside, and the host (me) controller the scene on Unity as the testers desired. Like before, short group discussions out of VR were held after the tests. The results were noted down in writing. 6.3.2 Story prototype (a) (b) Figure 6.4: Photos of recording set for the story prototype, with updated setup. Using maps and a recording set updated from what we had learned in the previous prototypes (see fig. 6.4), we shot a set of recordings for the story prototype. The Chair scene was then quickly rearranged in Unity, to remove the old recordings and put the new recordings in to be played in sequence. This prototype was then tested through by the director and the producer from beginning to end (which takes around two minutes each time), and then evaluated with the method used for the previous prototypes. This last evaluation session took about 30 minutes total. 30 6. Execution Process 6.4 Process Evaluation Two semi-structured interview sessions were carried out at the GGM Film studio meeting room for the final interview. The first was a solo interview with the intern who was part of the process in the design formation phase. This interview took less than an hour to complete. The second was a group interview with the director and producer of the project, which took about an hour to complete. The other intern who was part of the early phase was excluded from the interview, as he did not encounter most of the design process and was therefore deemed unable to evaluate it for this thesis. These interviews were recorded on audio, on top of the notes written by the interviewer. The list of questions/topics prepared beforehand (which were not asked in any particular order) were: • How do you feel about the current state of the project? Are you satisfied? • What did you think has been the difficulty in working with volumetric video? • How practical do you think these methods have been? Were they easy or difficult to work with? (Bodystorming, MDA, sketching workshop, parallel prototyping) • Do you think you will use some of these methods in future projects? • Do you feel like there was difficulty in communicating and developing ideas between us? Did the methodology help? • We often talk about techniques you are more familiar with (e.g. close ups, devised theatre, maps). Do you think it was more difficult to apply them in this project? • Do you think the previous knowledge makes working on this project more constricting? These notes and recordings from these interviews were then collected and anal- ysed to summarise our findings and reflections of the project so far, as can be seen in the Results section of this report. 31 6. Execution Process 32 7 Results This section describes the results of the thesis project, and some of the results of the Café Glaser-Müller project itself. First is the results of the early analysis phase, which prototypes helped to form two points of considerations when it comes to the use of volumetric video technology. Following that is the results of the design and evaluation workshops held in the idea formation and prototyping phases, and then the results of the interviews held in the beginning and the end of the design participation. Finally, a set of lessons learned from this process is outlined as design recommendations for designers in similar projects. 7.1 Early prototypes Figure 7.1: A screenshot of the first early prototype. In building the early prototypes (fig. 7.1, detailed screenshots in Appendix A.1), the differences between the ease of use of the different solutions quickly revealed itself. Two major factors were identified for their usage in a project: ease of recording, and ease of implementation. The ease of recording comes from the method of recording, such as the software provided and their requirements. A single-camera Depthkit recording was deemed the easiest to do, as it is quick to set up and record with just the provided software and a Kinect camera. On the other end, HOLOSYS requires the recordings to be 33 7. Results done at a studio. This ease can vary even with the same solution, depending on the design goals and setup (e.g. one or two cameras, Kinect or camera with separate depth sensor, green screen and lighting, etc.). Ease of implementation, especially for the purposes of this project (i.e. im- plementation of the recordings in an interactive virtual reality scene), also has a major factor in the exportable file formats of the solutions. Depthkit, for example, is able to export into a format (specifically a combination of video and depth bitmap sequence) that can be directly used with a proprietary plugin that generates a 3D mesh for the video playback in real-time in Unity. Meanwhile, most can export to a sequence of static 3D mesh or point clouds, which would be loaded into a Unity scene one frame at a time. Without further optimisation, that format could cause performance issues when used in a real-time interactive manner, and tend result in a slightly different look than what we wanted by default. This was less than ideal for the project, which was aiming for a photo-realistic camera look, and needed rapid prototyping iterations for figuring out the design of the experience. This ease of implementation extends to the flexibility of the use of recordings, e.g. whether or not it is easy to modify the look-and-feel of the recording, or to use the recording data for real-time asset generation in a 3D engine, and so on. These solutions also have different capabilities when it comes to multi-camera recordings (to get a multi-sided volumetric recording), which affects both their ease of recording and ease of implementation. Depthkit did not have the feature out of the box, which is why a manual solution was attempted by simply having individual recordings on multiple cameras be placed on top of each other in Unity. This solution was surprisingly usable, although not as good-looking as the other solutions. To summarise the results, the selection of volumetric video solutions assessed and the conclusions drawn at the time were: • Depthkit: Quick to record and to implement. Did not have multi-angle record- ing out of the box and has an ”unstable” look, but was deemed to not be an issue for prototyping. The one-sided look was also found to be interesting to look at, in its own way. Manually placed together multi-camera recording looked better than expected, but takes more effort and does not look seamless. • EF EVE : Exported mesh sequence or a proprietary format for a platform- specific viewer. Meant to have easy multi-camera recordings, but the seams between camera recordings were more noticeable than expected. Would have needed more investment in both recording setup and implementation. • HOLOSYS (4DViews): High quality recordings, and good out-of-the-box per- formance in Unity. The cost in recording is much higher, as it had to be done at their studio. It is assumed that other studio-based solutions would at least be on a similar level (one was looked at in the VR experience review). • Brekel PointCloud: Had the features needed, but the beta software was not as immediately usable as the other solutions. Ended up not fully testing this solution due to the effort required, and the point-cloud export format. Note that this analysis was based only on what was available at the time, and 34 7. Results therefore not to be used as recommendations for a future project, due to potential software updates and other changes. It was decided that single-camera Depthkit was the best fit for the design development of this project. It was the quickest both to record and to implement, despite some of its drawbacks in feature and quality. The effectiveness of the bodystorming method as an ideation exercise at this point in the project was not immediately clear, especially since the main focus was still investigating into the effects of the different recording setups. That said, all participants felt that it was at least useful for getting used to the technology of VR and volumetric video, and their potential uses. They also commented that it was simply fun to play around with the technology in a way they did not know was possible before. It also served as a prototypical setup for later test sessions, which has similar setup requirements anyway. The project manager also noted that this method is very similar to some practices in theatre, such as devised theatre. 7.2 Design workshops and sketches Figure 7.2: Photo of the final MDA diagram. First off, the MDA diagram arranged during the workshop did not end up being used much outside of the workshop itself, but did facilitate discussions on various parts of the desired experience. Participants felt that the diagram became complicated very easily (shown in figure 7.2), and was therefore difficult to parse as a whole. A common response was that they understood the framework and see the value in it, but were not sure if they used it properly. A positive outcome of the workshop was the understanding of some major themes and expected critical moments of the experience, which were highlighted 35 7. Results by seeing which parts of the diagram was connected back towards. The director explained in the workshop that it is important that vulnerability of the recorded actor is shown, while still prioritising comfort and safety for the players themselves. Relating to it, we agreed that it will be important for the experience to have a natural learning curve that accommodates the target audience, who are likely unfamiliar with the technology. Figure 7.3: Photo of the sketches drawn in the sketching workshop. There were initially some confusion with the intended outcome of the sketching workshop following it. Despite the participants’ familiarity with using sketches to communicate ideas in general, they were said to be more often only used as a means of communication for getting specific work done. After the briefing made it more clear that it is meant to be a collaborative workshop to generate prototype ideas, the response seemed positive, and the workshop proceeded quite smoothly. 30 sketches were generated in the round of Crazy Eights (shown in figure 7.3). Following the discussion and selection process, there was a final list of 3 scenic ideas (starting inside a pile of chairs, realistic cafe, big pile of big chairs), 3 volumetric video recording ideas (close up face recording, jumping/falling, pregnant woman with smaller recording inside), 3 sound effects (bump sound effects, heartbeat and breathing, ”thoughts” inside head), and 1 music track to implement (Theme from Twin Peaks-Fire Walk with Me). The process of converting the video recordings into importable Unity assets resulted in learning more about improvements that can be done on the recording setup. Since there is a gap in resolution between the video and the depth recording data, it was difficult to separate objects that are close to each other in the generated 3D mesh. When the actor moves out of the green screen background (due to limited space), it was also difficult to separate the actor being recorded and the background. There were also general difficulties with green screen keying, which was mainly 36 7. Results caused by my inexperience with video editing. All of these may be slightly alleviated by video editing and changing the settings on the Depthkit software and plugin, but would be more easily improved by a better setup (e.g. using a wider green screen setup, brighter lighting, brighter clothing). 7.3 Scene prototypes (a) Cafe scene (b) Chair scene Figure 7.4: Screenshots of the scene prototypes. To reiterate, the scene prototypes (figs. 7.4) were ran on the Unity editor on my personal computer at the recording studio, with an HTC Vive headset. There is no overarching story or experience in these prototypes; the scenes and recordings were simply shown like a gallery of sketches. Testers had the ability to move around (by walking and teleportation) and push objects with their virtual hands, but direct interactions with the volumetric recordings were not directly implemented in favour of a Wizard of Oz-like testing process. Minor adjustments were made live during 37 7. Results testing, to correct unexpected discrepancies in VR such as audio volume control and scaling of the virtual scene and objects against the player. The two scenic prototypes included ideas that were not previously discussed, but came up during implementation. For example, the pregnant woman recording was mirrored to give a background to see the smaller dancer against. I also took some liberty in choosing where the prototype recordings are placed, e.g. some are placed a bit above, or further away from the walkable area. Background music as suggested in the sketching workshop, from the soundtrack of the film Twin Peaks: Fire Walk With Me (1992), plays in the background of the scenes. 3D sound effects were manually placed and synced to the recordings in the scenes. This includes foot- steps, heartbeats, breathing, and ”thoughts” represented by a pre-recorded reading of poetry. Detailed screenshots of these prototypes can be seen in Appendix A.2 and A.3. During testing, the Chair scene generated noticeably more response than the Cafe scene. The testers in VR commented and looked around more curiously, and had more to say in potential ideas for future tests. The director mentioned that the Chair scene was surprisingly more inspirational than the other, and commented that the environment the experience is set in is important. He also commented that after testing it, he could envision a story that is doable and is the clearest yet so far. Another takeaway from the testing was that the interaction of stopping/starting the recordings tested in the last prototype may not be crucial all the way through in making the experience feel engaging, and that it may be better to have differing levels of interaction complexity throughout the full experience instead. The Cafe scene was said to not be as impressive nor as surprising, but it was still interesting in other ways. There were particular details in the Cafe scene that were found to have some creative potential, such as the way the recordings blur in motion, but was thought by the director as to only be usable for other stories and projects. The idea of putting a smaller dancer in a pregnant woman was also found to be difficult to look at, as it is hard to see an object that is placed very close to the viewer. 7.4 Final story prototype The final short story prototype (fig. 7.5), intended to prototype and illustrate what one section of the full experience could be like, was built on top of the Chair scene used in the previous prototypes. This was again tested with a similar setup to the previous prototypes: on Unity’s test play mode on my personal computer at the recording studio, with a Vive headset. In this prototype, the player starts from standing in the same position as the dancer, who has the ”thoughts” sound playing in the head. The dancer circles around the starting position, while looking towards it. The dancer eventually moves away to the edge of the chair they are on, and then jumps off onto a different chair. The jump was achieved by pausing a recording mid-jump and manually animating the video asset across the gap, and then sequencing a landing video taken from a 38 7. Results Figure 7.5: A screenshot of the story prototype. higher angle. An image set showing what this experience looks like visually can be found in Appendix A.4. This prototype does not involve direct interaction other than voluntary player movement and looking around, but audio and visual cues were intended to guide the player as clues for possible interactions. We encouraged looking around recordings by starting the experience looking at a one-sided recording’s back, and having the recording circle around the player with the audio following by. The dancer stopping by the chairs, looking at the player, and then jumping through them (which bumps the chairs away) is meant to show that the chairs are physical objects, which may come to play later on in the intended full experience. After the final testing session, the director was satisfied enough by the resulting prototype to use it as a proof-of-concept going forward. The prototype was then also exported as a standalone executable that can be easily run on other Windows computers, for the studio’s use in further development of the project. 7.5 Interviews 7.5.1 Opening interview The opening interview revealed that their creative process, at least the way they are used to, depends a lot on creative vision. A project in more traditional forms of art, such as film and stage, was often started from an idea or a premise. The ”core” of a project is often mentioned as a guiding principle behind most of them, which can mean a specific image or situation that sparks the inspiration for the rest of the project, often linked to emotions. For example, an interesting conversation could serve as the inspiration for what happens before and after that conversation, which could eventually extend into a whole story, which would then be narrowed down into the scope of a film. Collaboration in the film industry in particular was described more often as 39 7. Results a back-and-forth, relying on each other to understand the creative direction and use their expertise to realise parts of it. Points of communication and collaborative ideation would most often come in the form of meetings. Some tools that facilitate these were mentioned, such as moodboards and scripts. This sounded to depend personal preferences, as the director in particular claims to prefer using floor plans than storyboards, for example. The cost of film production is also mentioned as a factor in how productions are carried out, making it difficult to have an iterative process or to make changes at all once it starts, outside of editing. Evaluation processes did not seem to normally be a focus. There could be test screenings for a film, but if there was data gathering involved, it usually involves questions specific to the story to check if the audience understands it. There is concern for it in this project, however, as there are more specific user experience goals (e.g. user comfort). 7.5.2 Final interviews By the end of the project, there were a lot of concerns over the difficulties that arose from the global pandemic and other projects. They remarked that it ended up falling behind the initial schedule. Despite that, they said to be glad that the project was still steadily being worked on, more than expected. One comment was that the design process felt like a detour, but it was for the better. Finding the ”smallest part” (i.e. the ”core” inspirational concept, as mentioned in the opening interview) of the project from the prototypes was specified as a major progress for the project, All interviewees remarked on the research-like nature of an experimental project like this. One comment was how ”it is about learning”, and that it is ”healthy” to have had a relatively formal approach. Another was on how both the techniques and aesthetics of the technology are new, which meant that it was more difficult to realise artistic ideas. That said, they also believed that the underlying artistic thought processes are not too dissimilar, no matter the media. With regards to the specific methods and frameworks employed, the responses were mostly positive. All saw value in the MDA framework, in an interactive medium like in this project. They also saw use for it in other media they are familiar with, such as interactive theatre. Interestingly, the director was the least enthusiastic about how it was used in the project, while others felt that it helped to see his ideas represented on the board. They also felt that it was good to have an established framework and set of vocabulary in mind, even when they are not explicitly used. The sketching workshop was received with the most enthusiasm. While they were previously familiar with sketching in general, they liked the way it was used in the workshop. The specificity, quickness, and diverging nature of Crazy Eights was cited as a big plus, and they mentioned that they might use it more in the future. One comment was that ”different ways of writing down different ideas” is always good to learn, and that the exercise in abstract visualisation was inspiring, since it differs from how film ideas usually start from writing. It was also said that having a clear path from sketching to prototyping felt very effective. On the other hand, 40 7. Results the intern felt that the path from sketching and recording to the prototype was not quite clear, which is potentially caused by the lack of involvement in the first round of prototyping. Parallel prototyping was said to actually be fairly common in theatre and pho- tography, where testing different ideas have a fairly low cost. They felt that its use was important in VR, as it is something that is hard for them to imagine. Not only did they not have familiarity with the technology to begin with, it was also difficult since there can be a lot of material phases between idea and implementation, from sketches to flat screen renders to VR. The perspective in particular was cited as the most difficult to imagine. There was also remarks on the natural fidelity of the technology often feeling ”disappointing” when it comes to testing the implementa- tion, although it is quick to get used to. The director felt like this is an issue, since the aesthetics is attached to the photorealism of some of the results. That said, he also commented that a ”best we could do” attitude with the technology felt more inspiring and ”indie”. There was discussion in both interviews about the thought processes they rely on when it comes to making art. The subject of ”taste” came up in the second interview. It was said that personal tastes of the artists weighs heavily on how art projects go. This means that even when the results somewhat match what they thought earlier in the project, they may still personally feel that it does not match what they think it should be. One method cited as a common coping mechanism is to simply let go and compromise on it, and to instead focus on what is interesting about the result. This comes in especially often in collaborative projects, where they would have to simply trust in the other collaborators’ ”taste”. In the first interview, the intern used a metaphor of ”strings” to describe her thought process. She described that an idea could feel like strings in a web, that one would pull and follow along to find more strings of ideas. This is strikingly similar to the metaphor of pathways used in Dahlstedt’s spatial model, which she had difficulty in understanding when described earlier. This shows that although some creative frameworks may be difficult to explain and adopt, it may still be the case that the artist has a grasp on similar concepts intuitively. 7.6 Lessons learned My experience working on this thesis project had been quite interesting, and there are a lot of lessons that can be learned from it. As an interaction designer, there are six specific lessons that I would like to discuss and share with other designers that may have to tackle projects similar to these. Some of these lessons may extend beyond volumetric video and VR projects, and to collaborative projects that use technologies with less mature and/or defined design spaces in general. They will be detailed further below, but to quickly summarise, they are: • What an interaction designer can contribute to an art project: an interaction designer should be capable of helping to lay out and facilitate collaborative creative processes with yet-to-be-known possibilities. 41 7. Results • Why design methodologies should be used in an art project: formal methodology has value as a communication tool in a collaborative art project. • When ideation should be planned for: there is a need to prepare for constant ideation in an art project. • How design goals should be formulated: extra care needs to be taken not to make assumptions on design requirements and limit too many possibilities too early. • What other design methods can be potentially used in a similar project: there were gaps and opportunities in this project that could perhaps be addressed with other design methods. • What design potential we saw during the course of our project: there are still other design opportunities within the technologies that have not yet been taken in this project. 7.6.1 An interaction designer’s place in an art project What kind of contributions can a designer make in an art project? Beyond technical expertise in interactive digital technology (in this case, 3D game engines and VR), there was more I could contribute to the project than I initially thought. Interaction design is a field that is often interdisciplinary by nature, and often involving ideas across a variety of media. In this project, I found that although people in creative industries (artists) may have experience with different media, it is often that they specialise in one at a time. In this project, for example, the collaborators at GGM Film were mainly focused on film, even if they have previously specialised in theatre or dance. Having the experience and flexibility to lay out and facilitate a collaborative creative process for working with a less established medium is a valuable thing to add to the project. The use of design and prototyping methods in this project, as mentioned in the final interviews and over the course of the project, was valued and met positively, even when it did not quite work out (as was the case with the use of an MDA diagram). It was also valuable for me to identify the design process needs of the project to begin with, which included the need of rapid high-fidelity prototyping to evaluate the effects of both volumetric video and virtual reality properly. Based on that experience, I believe that an interaction designer is most valuable in facilitating the discovery and the prototyping of creative possibilities in an art project, through the use of interaction design methodologies. 7.6.2 Methodology as an interdisciplinary communication tool Communication can be one of the biggest problems to solve in a collaborative art project, especially an experimental one dealing with new technology like volumetric video. It was found throughout the project that artists of different disciplines may be familiar with a variety of practices that can facilitate collaborative design and 42 7. Results communication (e.g. use of maps during a shoot, devising theatre), but may end up not remembering nor proposing to employ them in a project in a new medium like this. It can be valuable to simply attempt the use of methodology, as it can help to connect the different disciplines and remind them of more familiar practices like how the bodystorming method and MDA model reminded them of interactive theatre practices. It is also worth noting that even though it did not seem to be particularly effective at first, the MDA framework helped everyone to establish a common language to think and discuss in. The bodystorming method also ended up being useful as the de facto evaluation method for our prototypes, despite it starting out as an ideation method. Taking the initiative in proposing and employing design methodology early on in the project, even when not immediately paying off, can be beneficial in those ways. 7.6.3 The need for constant ideation in an art project Although the design process was initially meant to have specific phases as per the Design Wheel, almost all activities done throughout the project ended up involving idea generation to an extent. As discussed in the interviews, artists tend to rely on basing a project on a specific inspiration, and finding that core inspiration is often a priority. It can be a bit difficult to capture specific ideas and chase it down, if it was generated during, for example, an analysis activity. It can also be easy to lose track of ideas during the implementation of prototypes. This happened in this project during the process of opening interviews, video recording, and Unity programming in this project, since new ideas keep incidentally surfacing (some examples are described in section 7.3). This may be due to the nature of art projects’ rather broad possibility space, or may also be due to the newness of the medium of volumetric video and VR. The bodystorming method accomodated for this with rapid, real-time prototyping on the Unity editor, although there was unfortunately no formalised method of recording those ideas. It would be my recommendation in an art project to always accommodate for ideation at any point in the project, to not lose potential good ideas generated during other non-ideation activities. 7.6.4 Risks in design goal formulation There was considerable risk in forming design goals very early in the project, since it encouraged making design assumptions on something that is experimental by nature. I personally experienced this early on, by deciding on the uncanny valley phenomenon caused by volumetric videos in VR as a main focus of the thesis, after some early discussions and analyses. This was first assumed to be a major factor in the project, but the more the project proceeded, the less it actually mattered. There were other assumptions made early on that very quickly got shot down by the limitations of the technology, or by simply finding out that an idea is less interesting than it seemed at first. Even the practical limitations of both VR and volumetric video technology changed over the course of the project in a variety of ways, with new VR headsets and features (e.g. finger tracking), volumetric video software updates, and a global pandemic preventing use of volumetric recording studios discussed in 43 7. Results the early analysis phase (section 7.1). This was the reason the focus of the thesis was changed to be more about the design process of art projects in general, as it was the larger design problem to tackle in the project. Although certain aesthetic goals can still inform the direction of the design process, designers have to prepare for design goals and requirements to constantly change over the course of an experimental art project such as this. 7.6.5 Potential design methods for future projects Most of the methodology employed in this project assumed for design collaboration with all participants, but an unexpected blind spot was found in the process of the prototype implementation in Unity. The effort in time needed for implementation was higher and involved more creativity than I initially expected, resulting in a lot of new ideas generated all by myself (as described in section 7.3). Although the others were happy with the results of my work, it ran a risk from me, as the programmer, to have a lot more influence over the results than intended. One potential solution to this is to simply involve more participation from other collaborators. There are also examples of this in other disciplines, such as the pair programming technique in software development, where additional observers would review a programmer’s work as it is being done. The downside to this would be the additional time investment needed from the observers, which may feel especially inefficient given that they may have less technical expertise. Another way of looking at it is that this is not necessarily a problem to begin with. It was mentioned in the interviews (section 7.5) that a back-and-forth collab- oration where collaborators trust each other with their expertise is more common in the film industry. That said, there could have been an attempt at having a more formal back-and-forth iterative design process during implementation. There can be potential in a design method that allows for more communication between col- laborators while in the process of implementation, so that the other collaborators can have a sense of the design evolution while it is happening. For example, in the context of this project, there perhaps could have been a process in which scene implementation and video recordings can be done in parallel to each other. One other thing to keep in mind in an art project is the importance of aesthetics and look-and-feel, and personal ”taste” (mentioned in the final interviews, section 7.5.2). Although this project had a target aesthetic early on, we found from the parallel prototyping that there still can be plenty of variations worth exploring. There may be potential in sketching methods that focus on exploring aesthetics and look-and-feel in VR, and is more rapid and collaborative than the parallel prototyping method we used. There could also potentially be better methods to be used for measuring and documenting self-evaluations on aesthetics, in a way that is more robust and clear for collaborators to keep track of. 7.6.6 Design potential of volumetric video in VR In the course of the project, we often felt contradictory interest in the aesthetics of volumetric video in VR. Although there is clear interest in the photorealism of 44 7. Results volumetric video, there also is often interest in the different ways they can look unrealistic. For example, in how single-camera Depthkit recordings look inverted from the other side, or how blurry volumetric video of movement looks when paused. From what we know, this did not seem to be explored as often as the technically more difficult clean and photorealistic volumetric capture. Another idea we had but did not investigate too deeply was the use of conven- tional filming techniques such as different lighting, make-up, and editing techniques in volumetric video. Some things we tested were putting on a fake belly costume to make the actor being recorded look pregnant (which looked fairly believable at the resolution of the resulting Depthkit assets), changing camera angles between cuts, and making ”close-up” shots. This is limited by some of the particulars of volu- metric video recording, such as the need for clear lighting and appropriate clothing for clear volumetric capture and the inability to make cuts as easily as traditional films do, but there may still be a lot more potential in achieving different effects and look-and-feel without having to resort to software solutions. These sorts of solution may be easier to implement than programmatically applying different aesthetics and other effects to the resulting volumetric recording data. Regarding the limitations of volumetric video recording, one difficulty we en- countered was the limited recording space that the Kinect cameras can handle at a time. A technical idea we had (but chose not to implement) was to repurpose the VR tracking technology to record the position of the camera, along with the video recording, so that it can be synced to the video and allow for free camera movement. A simpler setup would be to pre-program the movement of the camera to begin with, and mimic it in the 3D scene. There may be some further technical considerations with this idea, but if it is possible, it could open up more design possibilities with volumetric video recordings over a much larger space. 45 7. Results 46 8 Discussion 8.1 Limitations 8.1.1 Data gathering and analysis limitations As this thesis was written by a sole author, there were issues and risks that come with it. Due to limited manpower, there are risks even on the qualitative aspects of the data collection, as a lack of a second observer potentially brought personal bias to methods such as interviews. On top of that, a more rigorous and structured method of writing notes and project diary entries would definitely have helped in the long run, for both the thesis study and the art project itself. 8.1.2 Generalisability I have to admit that the results of this thesis is fairly limited in scope, especially since there were only up to four participants over the course of it. A small data sample is natural for a qualitative study, but that combined with the specificity of the project means it may be difficult to generalise the findings of this project to make assumptions