សូវាប់បានប់បានប់បានប់បានប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់� Thank you all for coming back. Thank you all for coming back. This has been a really thought-provoking day so far, and I guarantee you will have way more creative and inspirational works to come. So thank you for coming this afternoon. We're going to get right Going on it so that I don't cut into anybody's time. I'd like to introduce our first speaker Slide please Camille Baker Camille come on up We've got a slide for her, so we're going to give her a moment. There we go. Okay, so Camille's piece that she's going to be talking, or paper she's going to be talking about is Audience Onboarding and Offboarding for Emotional VR Storytelling. VR storytelling. Camille is an interdisciplinary digital artist, curator, and researcher in immersive experiences. Welcome, Camille. Is this on? Yes, yay. Okay, yes, I'm going to jump in as well. I prepared for 15 minutes, but I have 10, so I'm going'm gonna whiz through and I might skip some things but I'll just give you some context I've made two installation VR and haptic works in the last four year five years and prior to that I've have a long history in digital art and various aspects of electronic and wearables and that kind of thing so I won't get into my history but I come to XR from a different place of performance and and the physicality of digital mobile and wearable technologies and came into it from a different place. I'm going to give you some background around this topic first however. I've discovered this work Larissa Whitaker's Onboarding Offboarding in Virtual Reality after I was already doing it but it really helped make things slightly more succinct in my mind. And so I'm going to I'm not going to I have text, but I'm going to try to summarize the text before I go into it. But she says basically, when you're designing a work, for those of you who might be doing that, you need to think about these five aspects in terms of the audience experience platform,, place, user, genre and time. And so what I'm gonna do is I'm gonna cover those briefly, but I'm also going to tell you how that works within the context of the work I've been making. So platform is really about the equipment ultimately. What are you using and how people can get into the experience through the technology? And so you want to think about how easy it is to use these kinds of technologies, but also what does it mean when you're using a Vive versus Oculus versus all these different platforms? The next one is place. Where are you doing this work? So is it in the home? Is it in a museum? And what kinds of, in my personal view, what kinds of safety measures are there? Because people have gone into VR experiences and museums been given a headset and left on their own and then they don't know what to do next and so the context this is really context where is it showing and what what's happening there the next one is genre so what type of engagement is involved is it is it something that they have to do a lot of is it a game is it So what type of engagement is involved? Is it something that they have to do a lot of? Is it a game? Is it sitting in a chair and moving around? Is it, you know, what do they have to do and what kind of work is it? Is it training? There's a lot of XR work that's around training. Is it a movie in like 360? What is the genre? The next one is time. So what Larissa found or what Larissa Whitaker found was that time changes. Time is experienced very differently in VR. So sometimes it is really slow and seems to be going on forever, other times no time passes. For a quick example, and I'll get more into this later, but the work that I'm showing here is 23 minutes, but people come out saying, oh I thought that was only 10. So the time really shifts and changes and people feel like it's longer or it's shorter depending on what the work is. Who are the users? Who are the audience members, participants? I really like the term immersants and VR pioneer in in this area in the 90s, Char Davies, used the word immersants and I noticed a few people here and there are still using it and I'd like to bring it back as well. But there are things that are happening in VR that people aren't used to, especially if they've never done VR before. So it might be too uncomfortable on their head, it might make them motion sick, it might be the content is too disturbing for them, depends on the head, it might make them motion sick, it might be the content is too disturbing for them, depends on the work, or it requires something that they're not able to do in terms of accessibility. So understanding what people need and a lot of times again as Larissa Whitaker found out in her research, that's not considered a lot in the design. It's like, here's the piece, now let people see it. And it's not considered in terms of how people are going to experience it physically, emotionally, all those other kinds of things. So, like I said, I prepared too many things. And so this is the idea of thinking about how you manage the experience, how is it being supported in the context of wherever it is, and how you bring people from the physical into the virtual and then back out of the virtual into the physical. So there's two points that she talks about. Management of equipment, who does that? Who's the persons or persons who do that, and what is their role, how much information do they need to know, what happens if things break, and then how do they bring people into that experience. Yes, so I'm going to skip that. I have more time. That's okay, I'll get into my own background. So the first project I made in VR after researching it from an external point of view was this project called Into Her, and I showed a little bit about it I think last year around haptics and it's going inside the female reproductive system in various different stages it's it's not literal it's not educational it's very metaphorical and poetic but you do hear real voices talking about their experiences and I'll just run through it a little bit it's talking about post-reproductive it's not even post reproductive diseases fibroids polyps endometriosis and cancers that are not normally talked about that much in society with especially with you know anyone else usually maybe women will talk to each other but but it's not usually discussed So we did this whole development about how do we bring people through the experience? What, this is from inside, but also from outside. How did they go through the experience? What are they meant to experience at different points? How does that connect to, it's not on this chart, but how does that connect to the wearable that we're putting on them? How does that connect to the space? And I did this really rudimentary drawing when I started the process, but it's kind of still relevant, you know, bringing people through. We made this, I'll show you more pictures, we made this tent. They come out and they spend some time talking to a guide. This is like really early on the process. They have these very abstract flower kinds of images that are meant to represent their diseases, and there's a story from a real person talking about what happened to them in their treatment, etc. They go into the physical part is when we start we bring them we invite them into a womb like space and they sit down on a on a small squashy pillow and they're in a safe space and then they they start the experience we have the haptic that we put on them before they go in and so what the haptic is is it's got vibration at different points on the lower abdomen, and depending on where in the body we're talking about, it vibrates at different points in the story. The next project, oh, did I show this one? Where is it? Yeah. So this is what it looked like from the outside. It has been touring for two and a half years, and now I'm trying to connect it with healthcare professionals maybe a scaled-down version not this big tent version but definitely a way to explore gynecology clinicians and how they might work with patients in a more empathic way new project this year with two other artists, Mamrie Mountain, it's about the breast cancer treatment experience and we spent a lot of time trying to figure out again what we wanted to do and part of that was doing a creativity workshop with breast cancer patients and trying to understand, give them something, a workshop around how to express their emotions, and then collect stories from them about their experiences. We have some images. Again, I'm not going to go through all this because this is a different talk, but very much thinking about what are people experiencing. This is in the, I brought it in the campus exhibition with the RCA and if you're interested you can come and see it. But we wanted to take people through this journey of very, very emotional, difficult topics. People don't always talk about what happened for them in their treatment, whether it be breast cancer or any other cancer, and we talk about diagnosis, and then surgery, and chemotherapy, and radiotherapy, and all these really really tough topics and so the point that we wanted to think about is how do we connect them back to their body so another wearable garment that has vibration motors at different places in the upper abdomen and the back that are where the lymph nodes would be so So they're gonna feel this and so it's really important at the beginning of the process we tell them you're gonna have this really emotional journey, it will be triggering and you're gonna have this garment on you that's going to also be at different moments in the storytelling. And we dress people, we made some bespoke chairs, we dress people, and we bring them into, this is our first show, it's in Dundee in Scotland, we've now been showing it in Venice, and they made us a bespoke clinical environment that terrifies people because they feel like they're actually going into a treatment. And that was kind of the idea set set people up into go into the world physically and then go into the world mentally I'm almost done so I'll speed ahead and just to say we use same things thinking about how help people use the controls help them to think about how they're going into the world, think about the timing, and then think about how you train the invigilators. How do you tell them what to do, and how do they work with the technology, and how do they support each of the people who are experiencing this, and what do they say to them at different points, and what are the different things that might happen? So I'll just stop. I think that's it. Great. We have some time for questions. And we have students that are passing around microphones. So please raise your hand and a student will come and take your question. I'll get started. Okay. I can imagine after going through a traumatic experience, not knowing whether you're not knowing whether you're gonna live or die, right? That's about as traumatic as it gets. And then having to kind of relive it, but maybe in a different way. What kind of responses are you getting from the participants that are, the immersants that are in this space? Yeah, so we got a lot of crying but it's all quite good actually we had one woman in dundee who came into the experience really angry and said you know i'm really angry with the health care system and this and this this is the part where we actually talked to them for 10 minutes after their their experience and that's that's a big part of what i'm talking about it's really traumatic tissues Tissues are available, all those things. But she came out and she said, oh my god, I feel so much better now because I know that I'm not the worst. It's much worse for other people. So there's that sense of validation, sense of being seen, and so we get a lot of that actually. A lot of people actually, instead of feeling, I mean, yes, some people who are re-traumatized sometimes don't do the whole experience. And Tara, whose story is the thread that goes through, she was re-traumatized making the work. But ultimately, it's cathartic. From all the people we've talked to, it's cathartic. But we have to spend that time we can't just go see ya it's that really important kind of piece of caring taking time and okay maybe there are some pieces that don't require that if you see a VR piece that's not emotional maybe it doesn't require that but I think still the point of making sure that you train people who are helping people out of these experiences reconnect reconnect back to the world. If it is emotional storytelling, then it needs a little bit more. Definitely. Very therapeutic for some. We have time for another question. And actually, if we could get our next speaker to get set up. Reva? Thank you. Hi, Camille. I just wanted to really just give you some brilliant encouragement for your project. It's so important to draw visibility to what's going on with women's gynecology. We are still very marginalized, I think, despite being 50% of the population. Women's health is really marginalized overall. And I think there would be women in the room who've probably had traumatic healthcare experiences. I'm certainly one of them. So yeah, just thank you for advancing this field of study and hopefully this kind of research is going to be beneficial to society for the future. Yes, I hope so. I second that. Thank you. Let's give her a big clap. Very, very important work. Our next speaker is Jinsil So. And I seem to be running into her in every conference I go to. I don't know if she's following me or not. I'm following her. She's going to be presenting Mosaic Immersions, Stacked Realities, Merging Physical and Virtual Experiences. Jensil is an interactive artist and designer and researcher at Texas A&M. Welcome. Thank you so much. Hello, nice to meet you. I'm very excited. Hello, this is my first time in the Arts inica, that's why. Nice to meet you. I'm very excited to be part of this expanded animation conference and also RCN Electronica. I'm from Texas A&M in the United States from the visualization program. Today I'm presenting a recent interactive installation that deals with augmented reality and multi-sensory experiences. This work was done with my former student John Albersy as well as Wesley Taylor. These two images kind of showing you know that kind of core concept of core experiences of the project, a physical environment with physical props as well as VR headset and also augmented reality environment. So the people walking around the physical environment and they see the augmented contents. And then this augmented reality contents was recorded from a MetaQuest 3 headset. So I started this project from cultural and personal experiences from Korea. I grew up in Korea for many years, but now I'm in the States. So you probably have seen stacked rocks on hiking trails everywhere. It's not just in Korea, right? It's everywhere, and also like whole history. But having grown in a Buddhist family that hiked every week in Korea, and then stacking rocks has always been kind of meaningful and something special for me. So I started from that experience. In Korea, there are also multiple communal activities as well. One of them, one of the famous thing is tapje, ritual performing art done in, you know, Seollal, in Korean new, Lunar New Year. And tapje is a community ceremony that the villagers construct conical stone stacks at the village entrance to pray on everyone's health and well-being. And then these traditions demonstrate how the simple act of stacking stones can evolve the profound expression of faith and community and the human connection to the natural world, serving as tangible reminder of the power of collective individual and spiritual practices. So with that kind of background and foundation, I kind of started asking myself these questions. How might our sense of belonging change if we could see their trails of everyone's interaction layered in a shared immersive space? So really, like people touch and then hug, you know, and then all kinds of interaction. Can we see it after they, you know, went through? And what if the emotions expressed through our touch could become visible, creating a shared visual language? How would collaboration evolve if the invisible energy we give and receive through touch could be captured and shared visually? But why hands? As an artist, hands hands also like, you know, pretty important for me and for all of us, I believe. I was always impressed by the role of hands in our bodies, especially the expressive and communicative, also perceptive, you know, you share your emotion through hands as well. I also really like the concept reciprocal relationship within the hands. So when you touch yourself, and then you being touched at the same time, right? So I really liked, you know, giving and receiving and sharing information together. So the first project we did in 2006 from Simon Fraser University, collaborate with David and Diane Grimala and at that time we kind of captured hand movement of the audience members and they created kind of flying creatures. That was simple exercises but it was very important or interesting for me to see how they kind of like see their own hands from the third person's perspective. So kind of like, you know, making the relationship, different relationship to their own body. The next project I was recently working on is a similar concept, but we worked on a more complicated, more sophisticated creature's format. So the tree shape sometimes, sometimes snakes and birds, and also audience members can put own characteristics and behaviors and emotions into their own characters and then displaying them in the virtual reality environment. And after that, the users using virtual reality headset and then kind of like navigate around and see their creatures. They made it, but they own their own body. And also they see, they kind of make their own world, their own community, and then they interact with other hands as well. So based on the concept, we designed the user experience in three main tasks. So the capturing hand movement is kind of sharing with previous projects as well. In this one, when people kind of stacked, you know, rocks, they usually, I mean, many people usually kind of like, you know, pray, right? Kind of like, you know, pray something for their friends or their family themselves. So this time we recorded the story, like to the virtual rocks, and then those are connected. So the rocks with the hand textures around with the story. And then that's been placed in the physical environment, the story, and then the rocks will be shared with other people. So this is the recording of the project. people. So this is the recording of the project. So this is a hand scanning or hand capturing box. It has a webcam inside and also we provide the augmented reality interface introduction what to do within the headset. And then once the hand motion captured, they talk about their story based on the prompt we provide. Like, help each other. It's a really hard time. So sometimes, just ask them to tell your story. And then people don't know where to start. And then also people are like, mm, don't want to talk. So we provide the prompt. And then once it's generated, people can actually grab the virtual rocks and they can hear what other people recorded and also their own story. And they can see the kind of like, you know, blinking, but they can see the hand motions around it. So we really wanted to generate the concept into the virtual world. They touched and then their trail stays on the virtual rocks and also it has their own story or prey. So the setup of the installation, as you saw from the beginning, it has main hand capturing box as well as props, also projection screens. In terms of technology system overview, it has four components, hand scanner with a webcam and desktop computer getting data from hand scanner with a webcam and desktop computer, getting data from hand scanner. Also headset is wireless, but you know, wirelessly sending data, you know, like virtual, the hand scan data as well as WAV file display within the headset. Also we use Photon Fusion to record and store the virtual objects that are displayed within the headset. Also we use Photon Fusion to record and store the virtual objects location. And then these are the examples of the captured images of their movements. A lot of people actually kind of like play with some objects, not just their own hands, so like keychains and others. In terms of participants' experience, of course rock generation was the most important and the core experience. People really liked it and then people kind of tried to create multiple rocks in this space. In terms of recording stories, sometimes people provide long, detailed personal stories, but sometimes just testing, testing, blah, blah, blah, and they don't really want to leave their own personal stories in the public installation. And then stacking virtual rocks, it's a little challenging dealing with virtual physics inside of VR, but when they actually made it stacked, they were very satisfied. This is an ongoing project. We're currently working on anonymizing the voice. This is an experimentation. We don't know if it's going to work or not, but hearing their own voice with their own story may be feeling intimidating. We like to change their voice to someone else's, like using AI. Also, we're providing alternative platform, for example, like web platform for browsing virtual rocks. Thank you so much. Any questions? We have time for some questions. The afternoon crew is not so talkative. This morning they were asking lots of questions. So anybody? Some questions? Yeah, very impressive project. How long did you take to create this? How long did you take to create this? How long? Yeah. The first prototype version we exhibited, it was about six months. But also, I'm not only working for this with all other jobs, so it took a little bit longer than usual. But also, we've been... This project been evolving with different, you know, components. For example, like, you know, anonymizing the voice as well as, you know, providing different interfaces. So it's an ongoing project. So it's been a year so far. Yeah. I hope we can finish sometime soon. Another question back here. Could the students pass a microphone in the back? Does anybody have a mic back there? Okay, good. I don't have to jump over the camera. Thank you. So your project contains a lot of user interaction and gathered data. Do you expect to work with those experiences that were shared as a separate art piece or to evolve it further with those? Yeah, so as I mentioned, we'd like to create like another platform like a web or something else. So based on the people's recorded like stories and a captured movement as well. But we are not too sure and you know, it's kind of like projects becoming bigger and bigger. So I don't know exactly what's going to be as a separate installation. But also this installation, you know, like kind of embodying that experience, you know, putting your hand and capturing that also part of the experience, right? So maybe if people see everything just on the web or just virtual reality, they kind of lose that embodiment experience. So we're kind of debating what we want. Thank you. Hi. Thank you for the great presentation. Thank you. Very interesting work. A question from more... Well, I work in a more technical area, and I was wondering, do you evaluate the experience of the users in the installation in some way? Like I don't know, some user experience. That's a great question. Yeah, for some other projects, instead of evaluation, I study user experience. That includes evaluation, analysis of their experience right but this one I think it's still explorative I'm a little sensitive about utilizing their own story and own you know body data so you know like if if we want to do study we have to go through like you know ethics approval stuff right so we're working on it and I'm not not too sure how I'd like to study. But definitely, I'm very, very interested in what that means, especially using AI voice instead of using their own voice, like changing to AI and how they feel about it. Is it like they do they still connected to that voice or not connected at all? So I think those are interesting research question for me. Thank you. Hello, hello, yeah. Great presentation. I have a question about how, can you talk a little bit more about how the rocks are being generated and is it related anything to the text description they're describing the story? Oh, that's a great point. So we had an intent to connect what they say and then what they see. But at this point, they're not connected. But it's on our list. One of them we really want to try. In terms of rock generation, so currently we have a randomized 50, 60 shapes, real poly rock shapes. And then it's randomly selected. And then these texts, I mean the GIF file, and then we create actually pattern like this. So once it's, you know, once we created a pattern, it wraps around the rock shape. So that's current process, but hopefully we can connect it to, you know, what they said and maybe we can, you know, again like study, analyze, you know, like their story and then try to find what's the relationship with what they did as well as what they said. Thank you so much. Thank you. Let's give her a big clap. That was very interesting. Okay, let's get our next speaker up here while we'll get set up with the technology if they wanna get that computer working. And is our next speaker here? Oh, nice to meet you.