Thank you. Good afternoon on the last day of Ars Electronica and welcome to the last track of expanded animation. We will right now kick off the third edition of Synesthetic Syntax. of the third edition of Synesthetic Syntax. Synesthetic Syntax is a collaboration between Ars Electronica, the University of Applied Sciences Upper Austria, and the University for Creative Arts, Farnham. I'm co-organizing this chapter with Birgitta Hosea since 2020, and it is part of the Ars Electronica Animation Festival and Expanded Animation Symposium in these terms that it's more focusing on the research. So we have this hybrid vision of bringing artists, researchers, experts from different fields and also researchers in the field of animation, sound and expanded media together. We came up with this idea with a call of paper to spread it in the academia and to see how the researchers are responding and sending ideas to us. We started with the keynote speaker Rose Bond. She's with us and she will also join the closing note at the end. Would she not? She will not join, maybe not. And this year we have a new topic and it's a pleasure to hand over to Peggy Ter. So hi everybody, just to give you a little bit of background on this symposium. We use the term synesthesia and synesthesia, as you all know, it's a condition where you get the senses mixed up in your brain and people often use this as an idea to refer Mae'n ddigon haws, fel y gwybod chi. Mae'n gyfandod lle mae'r syniadau'n cael eu cysylltu â'ch ffyniad. Mae pobl yn aml yn defnyddio hyn fel syniad i ystyried ein bod yn defnyddio ein syniadau i ddysgu am y byd o'n gilydd. Mae'n trwy ein syniadau ein bod yn dysgu am y byd. Ychydig o flwyddyn diwethaf, rydyn ni wedi edrych ar y cyfathrebu rhwng swn a fision a chynyddu animaeth. between sound and vision and expanded animation and this year we're looking at touch gesture and proprioception and the relationship with that and expanded animation so why are we interested in the senses so we're interested in looking at the senses because this brings you back to the body it brings you back to the human being who's using the technology, who's making things with the technology, who's experiencing the technology, rather than simply focusing on the technology itself. So in taking their discussion back to the human, we hope to open up a space for creative, critical and ethical discussions about the use of technology and the making and function of animation in all its most expanded forms. So that's a little bit of background about the symposium. I'd like to welcome our first speaker, Andy Buchanan. Andy is an independent researcher formerly of Purdue University. Welcome. Thank you. Which to the other screen? Ah, there we go. Thanks everyone, hi. My name's Andy Buchanan. I'm going to talk about some processes involved in digital sculpture. To get there, I'm going to talk about making a whole lot of physical objects and drawings and other things like that. I take the title of my presentation today from this paraphrasing from Malcolm McCulloch, who wrote about digital craft in the late 90s, and used the phrase, a pure phenomenology of sensate presence, which was his sort of goal that he believed software might enable us to return to a sort of contemplative, fluid interaction with software that he described as this pure phenomenology of sensate presence. Throughout that book, he writes quite a lot and sort of draws on the work of Henri Fosillon, who said, art is made by the hands. They are the instrument of creation. But even before that, they're an organ of knowledge. And Fosillon's famous essay, In Praise of Hands, talks about the depiction of hands but also particularly about hands as a knowledge making instrument and a way of interacting with the world and so on so i'm a maker of animated images of lots of different kinds this is not not playing in a second This is not playing. Hang on a second. Oh, that one's not animated. How disappointing. So I'm sometimes using natural media, sometimes using 2D techniques, drawing, painting. I also have done a series of projects doing improvised drawings. This is a sort of a speed drawing attempt to capture the process of thinking. This is a slightly more considered but also improvised sequential art form of drawing as a way of capturing the visual impulse of thought. And in these graphic works, I'm inspired by the artist Frederick Frank, who wrote a great book about the zen of seeing, where he talks about the connection between the eye and the hand and the ability to connect them as closely as possible being a sort of a way towards using drawing as seeing and drawing as meditation. And so it becomes a form of process art, of course, and he's not so much interested in the resultant drawings, but says, in the end, we must recall that a drawing is not a thing, but an act, especially if we're using it as a process, using it as meditation, and so on. As I said, I also make a whole lot of improvised graphics in different formats. make a whole lot of improvised graphics in different formats but to come sort of to the point of today's discussion I also use improvisation in digital animation and here is both 2d digital animation and 3d animation based on digital sculpture as well as digital sculpture for more conventional purposes so this is sort of pre-production for a short film which is hopefully completed sometime next year. Last few years I've really focused on this idea of digital sculpture as a form of animating. The point here is that these forms are all improvised, there's no pre-planning, which is very sort of in very much in contrast to the usual way that 3D animation is done. Normally there's a very rigorous planning scheme, everything's sort of pre-organized. But these are created by simply approaching the digital material itself and seeing which forms want to emerge from that material during the process of creating the sculptures. And so it's related somewhat to the direct carving movement, some examples here from the 20th century, where the sculptor would approach the material with some sort of impulse but then let the material speak back to them about what it wanted to become. And so I'm trying to do something similar with the digital material, allowing the sort of polygon grain and the forms to express themselves during the process of making the different shapes. At the centre of all those different improvisational practices is the idea of temporal collapse, trying to get a hold of something in a present moment rather than being involved in thinking that extends through time. So rather than trying to think ahead about like, so what does this 3D model have to do? How should I design it? How should I arrange the polygons and being very structured in that way? It's more about trying to get everything reduced down to the moment of the gesture, the moment of creating the form itself. To begin talking a bit more about digital sculpture, I want to talk about the production of or the creation of physical objects. I was one of those kids who at a fairly young age discovered tools and discovered that I could take things apart with them and, you know, this screwdriver fits this screw and I could, you know, do interesting things like that. So I took apart my dad's stereo when I was quite young. And I remember I was sort of in quite a lot of trouble for that. He was, like, made me feel like I'd done something very bad. And so subsequently I took apart his bicycle. And this was sort of a process of exploring the relationship between tools and how they interact with the world and how they sort of deliver you potentials to do extra things. And as you become more familiar with the tools, those potentials and those potential interactions with the world expand. I replaced a damaged handle on this kitchen implement. Using probably the wrong material, certainly the wrong tools, trying to make a handle that would fit the hand of my, I think this was for my mother or whatever, so that slicing the cheese would somehow become easier. So this thing would sort of become more integrated with the hand. And you see like most handles on kitchen implements and so on, they're sort of designed for left and the right hand. But being right-handed, I thought, okay, let's make one that's customized for that. I still do this sort of thing. This is some recent knife handles that I made. I carve spoons out of wood with, you know, little knives. I'm on a bit of a quest to, at some point, create all of the furniture in my own house so that I can that's sort of a different topic altogether but so I've designed furniture I've created a range of different physical objects and the process for each obviously varies quite a bit. During the pandemic I started making guitars these are some of the guitars that I made in the last couple of years and my my initial idea for this, sort of leveraging a background in industrial design, was I know I can make everything in a computer, and I can design everything perfectly, and then I can instruct a robot to do the carving for me. And at the time I had access to these wonderful robots, and that's fine as well. But the thing is the robot doesn't have any respect, or doesn't have much respect for the material. You know, if it comes across a knot in the wood, it just grinds straight through it and we sharpen the bit later on. When I decided I would be making these myself in my basement, I started thinking about the different ways of interacting with the curves and the shapes and the different hand tools that I had available. Each tool sort of gives you a different opportunity to interact with the material in a slightly different way. Each different shape that you have to take out of the wood requires different sets of tools and you start to get familiar with how those tools interact with the wood and you see here there's certain grains which resist or support certain forms that they would like to become in a way. So when we think about this idea of a pure phenomenology of sensate presence and it's more than time for me to come back now to digital sculpture we're obviously in the territory here of what Merleau-Ponty established as embodiment relations and Merleau-Ponty's work on the virtual body described a relationship with tools whereby they can become extensions of one's sort of thought body and and in the very best occasions, can do so without drawing conscious attentions to themselves. So, you know, if you think about driving a car, the sense of the edges of the car become your sense of your own body. Henri Fausselon, who I mentioned earlier in his essay In Praise of Hands, described this as, between hand and implement begins an association that will endure forever. One communicates to the other its living warmth and continually in Praise of Hands, described this as, between hand and implement begins an association that will endure forever. One communicates to the other its living warmth and continually affects it. The new implement is never finished. A harmony must be established between it and the fingers that hold it, an accord born of gradual possession, of delicate and complicated gestures, of reciprocal habits, and even of a certain wear and tear. Now the inert instrument becomes alive. So all the different instruments that can be used to act on the surface or the wood or the material all have a different sense of distance and all present themselves as available to becoming a proximity to your body and you know and different distances from your hand the sort of modern version of this idea of embodied cognition generally falls into these two categories the second being the one that we're most interested here that is thinking about how thinking works as a you know as a part of the psychological, physical, cultural context. I find this diagram particularly useful. And we see at the bottom here we have the sense of embodied cognition and embedded cognition. But then to bring this back to a discussion about digital production, But then to bring this back to a discussion about digital production, this second layer of inactive cognition where we're interacting in environments is particularly relevant because of the virtual space of the digital model. It exists inside this sort of purely digital, weirdly Cartesian perfection of XYZ space that never actually exists in the real world but only exists in the digital world. And to mention again, usually creation of 3D models, 3D assets for animation require this very structured process. It's sort of like you have to know what you're doing in sort of a complete view before you can do anything. Like the image that I showed before of the robot making the guitar, the robot needs to know the completed design before it can start any carving. There is no sort of emergence or feedback along the way. And that's typically how 3D models were produced, changing now of course. 3D sculpture software presents a different interaction paradigm. It allows you to be a lot more interactive and sort of work forwards and backwards along that line. But there's still a sense of teleology where you're still sort of thinking ahead about what this might be used for, how it might be animated, what the shapes and polygon arrangements and so on should be like. This is the sort of setup that I use. And so it's such a contrived and sort of manufactured and fake interaction space here. My question then becomes, what are the possibilities and impediments to creating embodiment relationships between this sort of abstracted interaction and abstracted space and the final models and what are the sort of issues involved in the feedforward and feedback. How or is it possible to consider digital sculpting as present or sensory experience? I'm just going to briefly mention three issues which I think are central to this. I'll just very quickly mention tools because it's such a huge field in itself, that discussion is so deep. And then I want to talk about the idea of a digital surface and then some issues related to interacting through touch, through sight, through a screen. So in terms of hardware and software, the thing that I observe is additional levels of pressure sensitivity, resolution, putting the screen on the surface, all those sorts of things only get us so far. The areas for development here are likely to be more in unencumbered interfaces, haptics, force feedback, that sort of thing. But it's very hard for me to imagine an interaction space in the digital which can effectively mimic something like carving wood with a chisel. I suppose the outcome of my thinking there is perhaps that shouldn't be the goal, that the digital should remain as this sort of weird, hermetic, precise, strange space, rather than trying to constantly emulate the reality, emulate the physical. In terms of software tools, I think there's a risk of what Heidegger referred to as gestell, or in framing, whereby we start to adopt the thinking and the frameworks and the limitations of software and we adopt it as the limitations of our own thinking so if i'm thinking about digital sculpture tools i start to think in terms of what i know the software can provide rather than thinking about you know what could i possibly bring to bear on this digital surface. So it goes. This is the case, I suppose, with almost all software. The discussion the other day was interesting when we were talking about writing prompts for AI, and we start to learn about which words are like no-go zones. So we start to naturally not use those words as prompts, and then perhaps in the future we start to naturally not think in those terms, because we know there's sort of limitations there. So the second of these three issues I just want to mention briefly is the idea of a surface. We sometimes, neither of these are playing, here we go. Is it playing? No. The codec is not good. Oh. Okay. In any case, never mind. Let me continue anyway. We think about these digital surfaces as sort of pure and infinite and because of their mathematical capabilities sort of perfect but you'll notice you can even see from these still images here that as you start to carve and shape these polygon mesh surfaces they do start to exhibit what we might say is like natural digital qualities. Like they have a grain. They have different response rates when you interact with different parts of the mesh that's caused by the density of the vertices. And when you apply the same amount of pressure over here or over here, the reaction of the mesh is different. So one sort of development in my thinking in this area has been rather than thinking about the digital mesh as having properties, which is sort of a mindset from, you know, an engineering mindset, like what are the properties and positions of this surface, rather to start thinking about it as something that has behaviors. So I can get into sort of an interaction with this digital object and it behaves in certain, and I behave in certain ways in a sort of reciprocal response. In her book, The Storm of Creativity, the architect Kynaleski describes the process of using a stubborn or temperamental material in these terms. I chose the material. This is in the case of giving architecture students a particularly challenging situation. I chose the material because I knew that it would not respond to anything imposed on it. A stubborn material is indifferent to what I might want. And so it's quite a different situation trying to be interactive with the digital mesh, whereas usually we're sort of trying to problem solve. I have this design. I need to just impose my shapes, my forms on this mesh to get what I want at the end. We sometimes take our interaction through screens for granted. The visual sense is really important in this case because visual is our means of consumption. It's an interactive part of our eye, hand, brain, Wacom tablet sort of process. But it's also the form in which we're later going to consume whatever we're making here, usually. But what's interesting is that just like the digital surface that's being sculpted the idea of the screen is somewhat malleable and we totally just assume this we start to take it for granted we can zoom in zoom out we can bring up multiple views which is very much sort of again an engineering way of thinking but we might also consider it almost like a cubist approach like this is reminiscent of a cubist artist looking at an object from multiple perspectives at the same time again fausillon has interesting things to say about the relationship with sight and hands sight slips over the surface of the universe the hand knows that an object has physical bulk, that it's smooth or rough, that it's not soldered to heaven or earth from which it appears to be inseparable. The hand's action defines the cavity of space and the fullness of the objects that occupy it. Surface, volume, density and weight are not optical phenomena. Man first learned about them between his fingers and in the hollow of his palm. He does not measure space with his eyes but with his hands and feet so despite that we're using the sense of sight as our whole sort of means of understanding the digital space and like I said with these discussion on tools perhaps it's just a matter of time before we get good enough force feedback and haptic interfaces that we can start to progress beyond there. So, by way of a brief conclusion, coming back to this diagram here between ways of sort of physicalized, embedded and embodied thinking. of sort of physicalized embedded and embodied thinking. As we move up through the embodied and embedded phase into an inactive phase where we're interacting with digital software, we also have this sort of pinnacle at the top of the diagram here which is situated cognition which includes you know sort of broader social and technical contexts. And here is where we have issues related to sociotechnical understanding, techno-ontology, so understanding what is the digital, how does it exist. And sort of as we move up this chain, and for example, like I said, we start to take on the ways of thinking from the software we maybe lose contact with the bottom levels which is the more sensory and the more direct so an easy example is any time that i need to pull up a list menu in the software i'm sort of snapped out of that moment of interaction with the surface and brought into a different analytical and abstract way of thinking about the software processes that proceed. Finally, this block-by-block sense of technical rationalism, like the approach of engineering thinking where everything is logical and in a chain, is very useful. It works. It's good for making animated images. We've relied on this for sort of 30 years plus. Thankfully, now it's changing and getting a whole lot more sort of fluid and complicated. But it's important to think about this as a phenomenon, as an issue, because itself, this technical rationalist sort of mini-production flow is a part of a broader consumption and production scheme that is in sort of every level of society and you could sort of label it capitalist if you like but this idea of means to an end thinking where I'm doing this process in order to this in order to that is sort of at odds with anything emergent or anything improvised or anything ultimately I think think, sort of expressive, and certainly in the case of digital sculpture as processed art. Thank you. So we've got time for a few questions. Does anybody have any questions? Yep, over here. Have you got a microphone? It's OK. Maybe I can do it. We've got people on YouTube, so it's quite good to use a microphone. OK, good. So I have two questions, actually, regarding the workflow of sculpting with hands one is what do you think about gravity sketch or VR environments for providing a platform for this which on one hand creates that kind of embodiment in the space but on the other hand still you have to negotiate with these digital behaviors of the material and the second question if you ever thought of, I mean, we are working on something similar. We are capturing physical models while modeling and having an immediate digital version of that on a screen, which you can then manipulate with your digital tools. And I was wondering if you think that kind of method could bring us closer to hand sketching or hand sculpting in the digital. Like Kinect capturing or... Yeah, sort of photogrammetry? Yeah, I mean it's more photogrammetry like camera and distance mapping or also lighter. Yeah, yeah great thanks, great questions. I think the first question relates to tools for digital sculpting in VR, which there are very many and they're getting better really quickly. I think it's fantastic, but there's a trade-off. We trade-off being immersed in a space, which is obviously beneficial in terms of being able to see in three dimensions rather than trying to imagine three dimensions through a two-dimensional screen but there's no interaction so you don't know when you're touching something we we end up in my experience anyway in this really weird drifty sort of dance with the object where there's a little bit of visual feedback sometimes sometimes you see an icon sort of running on the surface of a model as you try to sort of touch it. But there's nothing preventing your hand going through the model in space. So it becomes sort of a weird dance, I think. And that's traded off against, you know, the benefits of being able to sort of tap on the tablet, but then you're confined to a purely, you know, unimmersed experience. So I'm not sure that one is better than the other. Obviously, the future is probably some combination of both, where you'll get a VR handle that will sort of stop at the edge of a haptic interface. In terms of the second question, I'm not entirely sure. I mean, I would love to see the workflow that you're working with. But some of my hesitations are about, you know when you're learning software and you're learning how all the list menus and everything operate and where all the tools live, those moments when you're taken out of sort of a natural interaction. You're forced into, now I have to do this step and now I have to do this other step. I don't know if your camera capture is very fluid or if you have to sort of break one way of thinking to jump into a different way of thinking to acquire the model to then work on it. So I don't really know what the answer is, but I'd love to see it. Okay, thanks. Then love to see it. Okay. Thanks. I think we have another question. Yes, thank you for the presentation. You spoke a lot about fluidity during the creative process, and I was wondering which parts of this creative process of modeling with these tools that you're using now, the digital ones where do you see the fluidity in which which segments uh because some of them might be more fluid than others i suppose yeah it's i find it's really unpredictable um over the course of the last decade or so i've tried a great many different things to try to get into that space of fluidity. I created joysticks with a single button to advance frames for 2D animation. I made an animation machine where you slide something across and it just advances one physical page. We can do things like hide all of the interface except for the model so the opportunities for let me perhaps rephrase your question the other way around like what breaks the fluidity for me is being forced to think practically about what the software needs you know we've all probably had the experience of learning the keyboard shortcut for zoom in zoom out you know and when that becomes part of your physical knowledge not part of your sort of thinking process you start to work much faster and more fluidly and you certainly don't think about it anymore like if you can type we don't think about the locations of the the letters so for me it's more about um removing as many things as possible that might sort of break you out of the moment of being in the interaction. But, you know, I'm a little bit of a scatterbrain. And, like, even when I'm working on sculpting something, I'll get distracted every, sometimes every ten seconds, sometimes every two minutes. I guess it's just a bit like meditation. It's just trying to find a moment where it's working and then trying to increase the frequency of those moments however you can technically. Thank you. Do we have any more questions? Yep. Thank you very much for your presentation. If I understood right, your argument, you described us, Thank you very much for your presentation. If I understood right, your argument, you describe us the transition from the physical sculpturing to the digital one. And I would like to ask you if your experience with digital sculpturing somehow influenced also your practice or your thinking about that physical sculpturing? For example, if you use prototyping in digital media and then you do it physically, or how are these relationships between physical and digital sculpturing? Thank you. My background originally was in industrial design and so I have sort of a hangover of this engineering mindset of like understanding things like manufacturing processes a little bit and that certainly made 3D animation and 3D modeling make sense to me from that perspective and I think I would say that I'm sort of a reformed designer tending towards artists as I'm trying to, you know, trying to drop some of those habits. And I think that applies both to physical and the digital space, almost the same way, I think. Yeah. I think that we need to draw it to a close there and move on to the next speaker. Thank you very much, Andy. Thank you. Andy's still here if you want to ask him questions. So our next speaker, Yanis. Welcome. While you're plugging in, just to introduce Yanni Skarags, who's an artist and immersive media researcher. immersive media researcher. And you're currently based in Cologne. Is that correct? Actually, no, Riga. Oh, Riga. Back in Riga. Yeah, but now I'm back in Riga. You're from Riga originally. I'm from Riga and back in Riga. Great. And you're going to look more at the idea of kind of gesture and instruments. Great. All working? Yep. Hello, everybody. Thank you for inviting me. Today I'm going to talk about investigations into gestural and touch interaction with anisotropic metaphors for audiovisual virtual instruments. This presentation highlights main concepts and reflects on examples of projects by some other people and also a few of mine that feature mappings of change of multiple values, audio-visual content or for example visualizing or sonifying various data. And there's a little video. So, this is actually, even the quality shows it, it actually made long time ago when I was a student in 90s, mid 90s, when I was trying to mix physical reality with imaginary reality. of playing with this ephemeral medium of digital image. And what I do is actually I do not map hands. I did it opposite all the way around. First I filmed my hands and then I made some kind of warping of the object. So this is some of the departure point from my previous education being a painter and going through classical art and then moving into digital audiovisual installations that I quite often use as stereoscopic imagery that interact with audience. I've been playing with a generative algorithms. This is like time series of evolvement over, let's say, 15 minutes of time or five minutes, depends on the piece, all loop, which usually were exhibited in galleries. Sometimes in 2D, sometimes in 3D, using stereoscopic image. Also, kind of involving human motion and kind of motion capture data, and so it was playing around and actually some of this stuff I was showing at Ars Electronica 20 years ago. This is again like a series of mutation from abstract shapes, and tapestry, like live painting of manifestation of human presence, like physical and conceptual, even body parts in different languages and so on. I did also some number of other projects, collaborator projects with the people who like visualization of, sorry for jumping, and this is kind of, I spent a couple of years in, first being an artist, I spent a couple of years in a serious research, full-time job in one European research project led by Fraunhofer Institute called Live, where the focus was some kind of concepts of the future media delivery system, sort of next generation of TV that would use, coming from content sources that would range from multiple cameras, statistics databases, and that would feed the usual control room and some kind of director which would work on some kind of score translation into live show experience and that would deliver in the end some kind of choices of channels and feedback. It was a really complex system involving algorithms analyzing the content, for example, recognizing a type of sport, even from algorithms. To make it short, the focus was to work on also content delivery, not only content delivery, but production. The new metaphors were introduced, and one of them was called the video conductor. From classical music that you would work with curves of emotional tension and saturation of the timeline and that was a departure from this rigid mechanical of switching through cameras so there would be more parameters to be taken care of and issues like collaboration and dynamic timelines. That also in a way reflected to my earlier work that I was thinking about the audiovisual objects as some kind of different levels of timeline that kind of global, which deals with kind of global synchronization of audio and image and going deeper into objects, elements that create that longer score, and even down to the elements of working with the pixels and the volume frequencies. I think I noticed also in recent years that there is actually trend to integrate the various aspects of sonic interaction, human-computer interaction, audiovisual performance, and even cognitive psychology. And the researchers Nunno Correa and Atau Tanaka recently published a paper called Audiovisual User Interfaces, AVUI, as kind of novel type of user interface linking interaction sound and image. That would extend the concept of graphical user interface. This is one of their case studies when they were working on a series of audiovisual performances, and they were integrating the kind of shapes as a kind of a, it was growing into kind of content itself. And this is an example of AVUI layout pointing to different user interface elements. And they were proposing that this could be probably in the next level of problem solving that usually a lot of artists try to, they are forced, actually, to develop their own tools. And because some of the popular software addresses some of the needs, but then there's due to stylistic concerns or collaboration problems with musician and visual artist, you need some kind of common platform. There is also a new field which is called immersive analytics as a proposal title. It's a field of investigating potentials of novel interface features. It is some kind of fusion of recent developments of visualization, auditory displays, computing, machine learning, and neuroaesthetics as some kind of a category that is applicable also, not like really neuroscientists, but also from point of view of artists, how they work with the psycho-emotional involvement of the audience and also the producer itself. Later I will propose some of my approaches to this field that, for example, this field that, for example, use of 3D interfaces sometimes have not led to the, let's say, not follow the hype of the, that we can come in some wave of excitement of using VR for solving a lot of design problems or kind of visualization problems, because it comes and goes into a certain disillusion. And so I've also put my own artistic interest and also research interest in some kind of dynamic interaction models that also pick up from other areas like immersive experience design or data process data and process visualization and sonification and interfaces and this is my also kind of a This is my toolset from probably 10-15 years ago when I was playing with, also I think that you mentioned with the whole range of MIDI inputs, some kind of 3D navigating game joystick and 3D Oculus first version when it came out was quite exciting and these are maybe some of you recognize also some of the favorite tools for a lot of people who work in 3D the so-called space mouse where you could have a usually for your other hand of for example a left-handed, you could control six axes of movement. People actually have stopped using that. They felt like, somebody said that they felt like their left hand has been cut off when they had to be somewhere else. So much you can get used to that. These very simple various tools to capture the gesture and also even finer details that this famous tablet from Wacom. Is it Wacom? In tools, they also have this stick or pen can have also, for example, angle sensitivity that increases the, well, at least they say so, the control of detail, trying to mimic real brush. This device has been used also for not only graphical design, but audiovisual performances and mapping these parameters to, for example, MIDI or open sound control and so on. Talking about MIDI, let's say 20 or 30 years ago when some of our best video resolution was 640 and then going up to, you see the actually relationship with the MIDI standard values, which is 128 degrees or increments, that relationship goes quite high up. For example, the 8K system here probably already has 62 pixels, which is quite a lot if you talk about some kind of contrast between small and big detail. And of course there have been a number of efforts to extend this wonderful standard MIDI, and there have been new instruments developed with the kind of augmenting the MIDI standard that includes kind of various strategies of capturing of really sensitive, like finger tip pressures that are not possible with, let's say, electronic keyboards, maybe 10 years ago. And there are some number of experiments I did with putting audiovisual content coming, as I said, this painterly approach to generate a new painting every 30 or 60 seconds, a new picture, and trying to put them together as some kind of audiovisual composition and playing with that. So let's hope the video will start. Maybe it will sound a little more... That was previous then also, like 15 years ago, with a kind of playing with a joystick, and this one is a painting in a space with some input, various, so that was trying out various input devices. That has to be observed within stereoscopic imagery, which is creating somewhat sculptural objects in the space, which is difficult to see in 2D, of course, but the progressivity of sound maybe kind of algorithmic variation, so every time it was different result. There are many kind of synesthetic relationships. One is of course this medical or kind of physiological condition of some small percentage of people experience certain sounds or kind of sensations in their, there is of course a way of creating algorithmic relationships and there is also some interesting examples from people who have been animating for example bird sounds and so on. But that was kind of rendered, animated, and there is a great deal of literature. I left it for a couple of seconds for those maybe who want to stop frame it later. To deal with these various challenges, there have been a number of proposals to create the taxonomies and notation of spatialization in terms of all the visual content. And that could be kind of a various aspects could be also put in different hierarchies and internal relationships and therefore actually I kind of propose an isotropy as a kind of metaphor which comes from nature that there's a concept describing orientation direction dependent properties and materials and space which comes from like a cellular level to space and that also the same kind of structure could be used to organize all the visual content and kind of using multi-channel sound. And there have been a number of efforts to work with kind of augmented reality interfaces. This is from Behringer, which in a way kind of looked like a cool gimmick, but I don't know how useful is it actually to work on real performance. So there are new devices so I have a couple of minutes left I think. I'll show, my proposal is to kind of approach this problem of kind of dealing, separating the physical and virtual space and creating kind of intermediary interface that is kind of same time part of the, can become part of the content and also as a tool. There's applications that could be useful, for example, financial data visualizations, which I'll skip through today, but just to show a couple of more minutes. So the space is kind of like a hidden map of various... Do you have a sound? Because it should be sound. It's kind of like a study, it's not finished artwork, it's kind of generating sounds and images on the same kind of closed loop within the scene. a closed loop within the scene. As I said, this is intentioned to work as a kind of immersive experience with the stereoscopic glasses and multiple sounds. So the sounds you hear are somehow kind of mixed into stereoscopic. Thank you. Anyway, putting this kind of VJ tool set into artwork itself. Two minutes. One minute. In conclusion, I think that multimodal interface elements that can combine artificially induced synesthesia and adaptive dynamic concept mapping to spatial out of your structures can enhance triggering of flow and efficiency in performance in complex, focus demanding scenarios. And I think the future probably is also to getting certain machine learning into this kind of feedback loop between interface, or kind of helping map the performance space, and measuring various latencies between, for example, sound waves in large large distances or loop within kind of different converter systems, that has to be also taken into account. And therefore the metaphor of anisotropy, I think that could be, it has inspired me to work for some time. Thank you. Mae'n rhaid i mi ddatblygu'r cyfrif ar gyfer amser. Diolch. Diolch yn fawr. Alla i ystyried beth rydych chi'n ei ddatblygu? Ydych chi'n datblygu gwaith gwaith gwaith lle mae cymwys neu unigolyn yn dod i mewn, yn symud ac yn gyflwyno sain a gweledigaethau? generates sound and visuals? I think I would like to work on this continuum between a tool and an artwork experience. For example, the installation could run it by itself, but then it could also be performed by the expert, the one who has created it, or has spent more time to do it, and so it becomes like a performance piece. And also to give certain input for visitor to kind of, if he wants to be a bit more active, to participate. And blending the tool with the result, which usually kind of stays, you have that interface and the product, and I think there's certain connections that I think are interesting. So are you generating this through some kind of motion capture, a gesture capture? That could be, that would be probably, I think I was, there is also kind of a collision of, for example, how you deal with with really finite finger movements, especially instruments like violin or piano. It's really important how you move your fingers. But then the other extreme, if you use a full body, and then there's the question of the scale. I think that in my case, I have a certain set of really personal way of dealing with that, but I'm have a certain set of really personal way of dealing with that, but I'm proposing a more abstract framework that would be adaptable, adapt a really expert approach and also some kind of really brutal approach where somebody tries to break this almost physically, and that would be able to accommodate that. break this kind of, you know, almost physically, that would be able to accommodate that. I think it's a kind of problem for a lot of artworks and museums, and this is certain energy, also physical energy or the energy of other types of people who are kind of really much more, let's say, meditative or detail-conscious. There's a big range which has to be kind of accommodated. OK. Are there any questions from the floor? Otherwise, I might ask. Oh, yeah. Thank you for your presentation. I like very much the artworks you show us. And I would like to ask you if they were live or it's recorded. Some of the works were just screen recordings of the live, and I'm working on the live. The live is my current focus, because there were some... I mean, there's still pictures, but actually in short, yes, live, yeah. Yeah. So just to be clear, you're working on kind of a visual, an instrument for visualizing, for visuals in a way, in order to for visualizing, for visuals in a way, in order to sort of play or perform them? And I guess, is it always generative, or can you have cued works? You mean sort of like a score? Yeah. I think this, I would like to, I consider myself also as a person kind of shifts between structure and impulse and emotion. So I think the score is important even if you have at least kind of basic dramaturgical line or kind of a playing field and then you play within it. And so yeah, I think the score is important, kind of at least a sketch for it. And so, yeah, I think the score is important, kind of at least a sketch for it. Yeah, another question here. I'm wondering with the investment that you are putting in, how you may be expanding your research into involving other users, the sort of the potentials and capacities of a visual language involve diversity. And perhaps I'm wondering if what you're generating is coming solely through your own desires, or if you're involving others and their potential aesthetics. I briefly mentioned some projects I've done together with other people that my input was certain visualization using, for example, a specific scenario or even it could be combined with a live video. But that's actually maybe a different project. I was at some point dreaming of some kind of a festival infrastructure system that could build sort of like a blend between two artworks that the two artists come together and they pick some elements from one artwork and kind of mix together into kind of new life form. And that would be this some kind of symbiotic or kind of fusion that also should be possible symbiotig neu ffusio, mae hynny hefyd ddylech ei fod yn bosibl gyda algoryddiau arbennig. Felly rwy'n hefyd, fel y dywedais yn fyr, rwy'n gweithio weithiau'n unig, rwy'n gweithio gyda phobl eraill. Ac mae hynny'n amlwg, weithiau mae nid yw'r canlyniad yn rhywbeth arall yr ydym i, mae pawb wedi ei Sometimes, no result is something else that everybody has expected. Unfortunately, we need to move on now to our next speaker, Jana. We can pick up this discussion at the end because I think there's fascinating resonances between your two papers with the idea of haptic knowledge and what the user might bring to that interface. For example, a dancer, the embodied knowledge they might bring compared to a non-dancer a fydd yn dod â'r gwaith i'r cyfranogwyr. Er enghraifft, bydd y gwyllt yn dod â'r gwyllt yn y cyfranogwyr. Yn ogystal â'r gwyllt yn y cyfranogwyr, mae'r cyfranogwyr yn dod â'r gwyllt yn y cyfranogwyr. Mae'r cyfranogwyr yn dod â'r gwyllt yn y cyfranogwyr. Mae'r cyfranogwyr yn dod â'r gwyllt yn y cyfranogwyr. Mae'r cyfranogwyr yn dod â'r gwyllt yn y cyfranogwyr. Mae'r cyfranogwyr yn dod â'r gwyllt yn y cyfranogwyr. Mae'r cyfranogwyr yn dod â'r gwyllt yn y cyfranogwyr. Mae'r cyfranogwyr yn dod â'r gwyllt yn y cyfranogwyr. Mae'r cyfranogwr yn dod â'r gwyllt yn y cyfranogwr. Mae'r cyfranogwr yn dod â'r gwyllt yn y cyfranogwr. Mae'r cyfranogwr yn dod â'r gwyllt yn y cyfranogwr. Can you play the intro slide for Jana while she gets the computer set up? Thank you. And our next speaker is Jana Holakova, who's Associateol a New Media Art o'r Unifersiad Masaryck yng Nghymru. Felly, a yw'r holl bethau'n gweithio? Gobeithio y bydd y sain yn gweithio. Ac yn y sesiwn cyflawni ar y diwedd, And in the wrap-up session at the end, we will have a chance to think about the links between all the different papers and have a further discussion. Welcome Jana. So hello everyone. It's my big pleasure to be here and to be part of this symposium. So first of all, let me thank you to organizers for having me. In my presentation called Searching for Gestures of Freedom and Intensities Within Interactive Media Art, I would like to share with you some theoretical concept because I think I am the only theoretician here among the speakers. My presentation is informed with the theory of interactivity and by the redefinition of new media art as programmed art, which happened under the labels of software studies or software art at the beginning of the 21st century. This presentation introduces basic outlines of theoretical framework for the study of interactive media art as gesture art. This redefinition enables us to overcome the reductionist trap of defining new media art with a focus narrowly on the material part of the artwork. To approach the interactive media art as gesture art means to think about these artworks as programmed media consisting of scores and scripts and of performative, collaborative and participatory events and experiences they provide. To define the program media as gesture art means to treat them as one of the live art formats, which allows us to explore their poetics using the vocabulary of language of art by Nelson Goodman and poetics Elements by Gaston Bachelard. What I mean when I refer to media art as gesture art. Let me start with definition of the gesture itself. The dictionary defines a gesture as a movement, especially a movement of hand that expresses something or emphasizing speech. There are various theories of gestures, theatrical, anthropological, cultural. Media philosopher and cultural theorist Wilhelm Flusser called for establishing a new theory of gestures, and he called it the general theory of gestures, which goal would be to search for gestures within the interactions in the information society, in his terminology, within an apparatus-operator complex. This theory of gestures is supposed to be a counterpart to information science. Its goal is not to strive for scientific objectivity or to teach some technical skills like programming, but rather its mission is to become a bridge between the critical academic reflection of cultural phenomena of the information society and the field practice of so-called engaged science. According to Flusser, this new science goal is to search for gestures among the overproduction of mere movements within the apparatus operator complex, whereas movements are, according to him, thoughtless actions which just follow the prescribed protocols. It doesn't matter if they are very simple or primitive or at virtuoso or expert levels. On the other hand, the gestures are, according to Flusser, rare expressions of resistance, freedom, and deflection. In his book, The Language of Art, Nelson Goodman addresses gesture art, which he named allographic work. According to him, allographic works include works of music, theater, or dance. All these art forms consist of scores or scripts and their executions, and none of these executions can gain the stamp of the original. In opposition to gestural allographic works, he placed orthographic works, like paintings or sculptures, and it means such works which are given their final form by the author and for which we can distinguish between originals and their copies. Goodman indicates the close relationship between gestural art and the allographic work in the chapter devoted to dance. He wrote, the possibility of notation for dance was one of the initial questions that led me to exploring notational systems, because dance is visual like painting, which lacks notation, and yet ephemeral and temporal, like music, whose notation is highly elaborated and standardized." On end of the quote. In this context, Goodman refers primarily to Laban's notation system that you can see on the picture, which is meant to be used not only for dance, but generally for human movements. From approximately the beginning of the 20th century, we can observe artistic efforts to liberate gesture Z příkladu začátku 20. století můžeme vidět kreativní způsoby, které vytvoří gestu z subordinace do jazyka a zjistit je jako jen ilustrativní funkci předmětným způsobem vývojů. Koncept gestu jako původní vizualizace, vývoj o sebe a způsob sensoryčního vývoje trajektory a rytmu vývoje vývoje trajectory and rhythm of the release energy began to be applied. Among the early manifestations of these efforts, we can include the Italian futuristic photographic technique, photodynamism. Its authors, Antonio Giulio Bragaglia and his brother Arturo, applied the futuristic aesthetics of universal dynamics to photography. To capture the energy released by the movement of the gesture, they used several lamps that bathed the scene in light to divide the gesture or the released energy from its source, like a movement of the body parts. from its source, like a movement of the body parts. In my opinion, a modern artist expresses the inner world in his work, in other words, he expresses energy, movement, and other inner forces, wrote Jackson Pollock. A typical example of artistic efforts to liberate gesture is gestural painting, particularly by the leading representative of American abstract expressionism, Jackson Pollock. Since 1947, he has created paintings in style reminiscent of dance with a brush or other source of colored pigment, which, as if by accident, leaves traces of the creative act on the canvas spread out on the ground. His paintings are kind of exposed scores capturing the dynamics of bodily gestures. Those, they can be interpreted as records of the primal movements of the mind taking place in the depths of unconscious. mind taking place in the depths of unconscious. Identifying this writing as a special kind of text would not solve the problem, because the purpose of a computer program is not to be written or read, but to be executed, wrote Stefan Holtgein in his paper How to Do Things with Keys, Assembly Programming as Kind of Gesture. In the list of allographic works or the gestural art, we can also include works of art created in the programmed media. Viewing a programmed work as an allographic gestural artwork leads us to its definition as an open structure made up of programmed game rules and processes, the scripts, and their executions, the performances, in the interaction of the user and the programmed work. Noach Valdry Pruin refers to such works as expressive processing, but perhaps it would be more accurate, at least in the context of this paper, to speak of programmed creative gestures. Vardy Pruim describes programmed work as a system created from the relationship between data and processes and the interaction between the surface or interface and the user. A user activity, his or her gestures, is the trigger mechanism that sets programmed processes into motion. They are performed in certain time and particular spatial constellations, for example in situations defined by the interface between hidden processes, their effects manifested on the computing device's surface, and the user's actions. This definition of interactive programmed artworks point to the performative gesture nature of the programmed work. The gesture character of interactive programmed art can be described as series of movements that gradually fulfills the script. Thus, it can be compared to the performance of a dancer executing choreography. Dance can take many forms. It can be formal and tame, like dance in a ballroom, passionate and sensual like tango on the street, wild and unrestrained like a rave party, and sophisticated like classical ballet or any other dance style. The same applies to programmed interactive artworks. The gestures inscribed into the programmed media can be very subtle and minimal, spectacular, subversive, anarchistic, or else. Thinking about programmed artwork as gesture art can be done by applying Gaston Bachelard's poetics of elements. For example, his poetics of water allows us to differentiate between types and intensities of interactivity provided by programmed media. From the mere splashing or the surface play of scrolling the content of social media, through the immersive experience of computer game players or immersions into virtual reality, to deep dives of those who intervene directly into the code to influence the very behavior of the given media. Since the late of the 20s and the beginning of the 21st century, we often meet with gesture-programmed artworks, which are more than anything else, amplified, extended, or postponed results of gestures of their creators, like artistic viruses or code works. We can effectively relate Bachelard's poetics of air to them. According to Wilhelm Flusser, the air is the primary medium. It's an archetype of all communication media, actually. Airwaves are characterized by easy availability as they surround us, and at the same time, the human organism is well equipped and adapted to use air as communication medium to share information encoded in natural languages. The gesture programming media can gain different forms and intensities of these air artworks. The poetics of air can help us articulate the intensity of the interactive artwork. It can be as subtle as the breath through which we constantly interact with our environment. However, it can also take form of windstorm, hurricane, making the air this transparent, evasive medium, explicit and even overwhelming dangerous element. To illustrate what I mean by programmed gesture art subtle like a breath, I would like to refer to a schoolwork by Andrew Spitz from 2012, Me Doing This. This was made in MXMSP, the records and visualizes the program's gestures by programming the software. The author compressed the programming process lasting more than nine hours into two minutes and 30 seconds long recording. Spitz records the programming processes taking place on the level of the unconscious at the moment of the creation of the work. The result gains character of a gesture painting or score from which we can reconstruct the choreography of gestures of programming retrospectively. We can say that this work is kind of remake subconscious primal movements of his mind working on this software. The gesture-programmed artwork can be also wild and overwhelming as a windstorm or hurricane too. For example, I refer to Alex MacLean's Forkbomb, the artistic virus written in parallel programming language. The output of the approximately 30 line Forkbomb.po program is binary data, seemingly randomly arranged ones and zeros on a black background, creating the impression of a kind of minimalist score. The output on the monitor is a result of the fusion of an artistic algorithm and the operating system in which it's running in a speed of sprint. Dirk Piesmans, one of the members of the artistic duo JODI, who has been active on the scene of artistic activism in program media since the 90s, often offers an analogy to understand the wideness of program media. He regularly conducts events called air plays, where he uses oxygen bombs to shock the audience by releasing strong air currents. Using the air medium, he evokes the feeling of danger spread from confrontation of the audience with the element of air that has gone out of control. It gains enormous power and turns against everything that stands in its way. Just like a computer virus, the element of air also has a, let me quote, naturally excessive character, Peaceman says. And to conclude, reflection on the nature of gestures in the context of programmed media led us to include programmed and interactive work among gesture art or holographic works alongside music and dance. The definition of interactive programmed art as a gesture art is also game changer for preservation policy. It calls for contextual anchoring of the artwork as a process or event in the network architecture of worldwide web as well as in the context of ideologies surrounding technical progress and especially emerging from the ethos of cyber subcultures based on non-hierarchy of network and hackers ethos, which is manifested in culture production based on the recycling, sharing, or contextual sensitivities. The gestures as manifestations of freedom and resistance within apparatus operator complex, as Flusser calls them, can be preserved only by keeping them alive as a part of living culture as an everyday repeated dancing ritual. Thank you for your attention. So thank you, Jana. That was brilliant. That was really thought-provoking and interesting. So I like your idea about the gesture as the kind of free choice of the human that's setting the performance in the computer off. But I wonder how you would respond to looking at gesture in another way. Gesture is something that is socially defined, that we're haunted by. So I'm thinking about Judith Butler and performativity and the idea that we're haunted by gestures and we reproduce the gestures that we see in the society that surrounds us. So rather than being freed by gesture, perhaps we're oppressed by gesture. I use the terminology of Wilhelm Flusser, and he suggests to use the word movement when we just repeat by the society imprinted movements into us. It's just mere movement. Standby behavior is mere movement for him, and he doesn't differentiate between human or machine. Whatever is programmed this way is just moving, not gesturing. And maybe I should reveal my background because I used to be ballet dancer. That's why I know that you can somehow build your own gesture language, individual and highly artistic at the same time. But it needs some self-discipline, self-awareness, and yeah, discipline, self-awareness, and not just mimicking the behavior of other people. So I was talking about artistic gestures, and I think that they are mostly, of course they can be parodies and exaggerations of the social movements, or they can be gestures of resistance and critique of the society. So yeah, Flusser used different, I think that he would call movements the gestures Butler is talking about. That's fascinating. Has anyone got any questions? Yep. Thank you for your talk. Very inspiring. You mentioned Laban notation. I was wondering, so I'm not familiar with the state of the art, but we discussed yesterday to use Laban notation in the context of animation, as there are some quite interesting aspects to control, to define animation. So I think in terms of character animation it failed, but you also showed some particle animation where you have forces and you can give energy. I was just thinking as you were coming from dance. So this connection to animation, do you see that there is a need to bring the Laban notation to animation, or is there something that is? I think it might be not Laban's notation, but it is one of the most famous. What is interesting, that dance doesn't have generally used notation system. That's what the Goodman said, that in the contrary to the music, we usually learn from our older colleagues these movements. We don't use notation in practice, but Laban tried to do it. And I have to say that I was here on the presentation where they show it and I like it very much and I remember that they made a kind of database of movements and they connected with emotions and I think that it can be like enlarged also with the category for example of that intensity of the gesture because it's not only the like of the gesture, but also the speed, or how deep I'm able to go into the scrolling the social media, or I'm really reprogramming, or I'm building my own tool, or hacking the software to prepare my own personalized medium to express myself. So it's like enlarging this. I have screenshots of these presentations, and I would like to contact these colleagues. It was really interesting, and for me, very much I like that they stress that movement is maybe one of the last human characteristics which are such individual that you can't reproduce, that it's kind of like fingerprint. So I think there is there is potential, but I have to say I share with you some ideas. I would like to elaborate more further. And yeah, I think that there is potential because it's definitely the new media art is more like time based art than for example, visual art, but in the virtual space. All right, I because my background, I feel like this. Then again, we get back to Andy's point about haptic knowledge and with the dancer, the body is the archive. Body is the archive. Yeah, that's another possibility how to use the digital databases because that's something that is of my interest too, that I use the digital media as art theoretician to work with like artificial intelligence to discover some schemes or repeated motifs or in this big data sets. So I think that we can use it also to protect, for example, knowledge of folk dance, but also, for example, of the classic dance of the certain ballerina, because no one do it the same way. And it's just a very ephemeral artwork that just disappears after like 15 years when you are too old to dance and yeah so it would be very nice to use this digital media to find the essence of the individuality of every movement style We had some more questions I think I think you've got the microphone next Yeah just on that point thank you for your talk, your redefinition was was really intriguing I thought but just on your point about the like individuality of of movement that potential there I know in his like kind of conversation around photography Flusser was concerned about kind of like the apparatus and the presence of the apparatus in the process ultimately leading to a form of ymgyrch o'r ymarfer, a chynnydd ymgyrch yn y broses, yn arwain at ffordd o ddysgu'r ddynion yn yr ffordd gwaith, lle roedd y bosiblrwydd o ddelwedd y ffyrdd o ddelwedd wedi'i ddysgu gan y cyfrif o camera sy'n cael ei ddifynu gan y gweithiwr. Mae rhai o'i ddechrau yn dda o ran ymddiriedaethau pobl i'r trageddoriaeth sydd wedi'i gynnwys i wneud gwaith drwy'r fath o fath o fath, ond yn ymddygiadu'r optig a'r canolbwyntio o'r potensialitâd sy'n ei gysylltu â'r fath. Rwy'n credu bod hynny'n wir iawn o ran gysylltiadau a chasglu'ch data. Efallai bod chi'n twitio am faint o'ch bod yn hoffi Twitter yn fwyaf yn fwyaf yn gwella'r set data mae Twitter yn ei gynnal. Ie. Felly roeddwn i'n edrych ar... Roeddwn i'n meddwl a oes gennych chi unrhyw fath o sylwad ar hynny. Gan ddod â'r gesturau hynny i'r maes o ddigidio a thamu, a phynoriad bwyd binar, ac y math hwnnw o fath o fath, yn hytrach na chyfleu newydd, and digitizing and like binary information and that kind of apparatus that actually rather than opening up new possibilities the kind of horizon of what's possible is being removed from our individual control and also accelerating towards us in quite dramatic fashion. I think that I understand your point, thank you very much. Maybe I would refer to another dichotomy of terms Flusser used concerning the possibility or possibility to be really creative and free within the apparatus operator complex. He defined two users of this media, robots who just follow the instructions. For example, he's talking very often about camera and he says when you use camera as it should be, you are just following instructions and you are just robot making the camera work. Actually, you are in the service of the apparatus to make it work. But then there are players, and they are able, as he refers to chess play, you play the game, you are not quitting, but you are able to overcome the, what is, the other player, or the system, the programmer, maybe by learning to program or reprogram the media. I think that he's aware of the dystopia of the operator-operator complex and there is no escape from it. But we are able to perform some strategies of subversion, searching for some escape games, escape gates. And yeah, so I think in this thing, he is quite realistic. There is no way to go to, I don't know, to forest to put away your phone, but that you can be aware, a very user, and not to become like, he calls it robot. Yeah, so, yeah, I like on his philosophy that he, there are every time like two meanings in it. You can see the dystopian character of his, for example, chamber music, but he use as a metaphor for us, just clicking onto the screens and sharing our pseudo arts or art together and making remakes of us are part of it. But at the same time, he offers some hope, some possibility. But he approached to the future, so he's not like sociologists who would talk about the present. So it's more like speculation on his side. So that he reminds us that we have choice, that we should be aware of our behavior, our usage. But there is no like simple answer, I would say. Shuf, one more question over there. Hello. My mind is spinning. You bring up so many challenging questions and ways of reimagining how it is we might identify an authentic and authentic quality of gesture. quality of gesture. And I guess at a really ordinary kind of level, I'm wondering if we think about perhaps the data of gesture that might be captured. I'm wondering if how you're imagining it, is it abstracting the sort of the biometrics of that gesture? Is it abstracting the psychological history of that person's gesture, that emotive expression of the gesture, how all of these things are such an incredible mesh that is involved with that arisal and performance of that momentary gesture, however long or short it might be? Can this recording of that gesture actually exist without those things in tow and be called authentic? Of course not, because dance is the art form that is like life art. It's by definition something that's happened here and now. But still, I think, as an art historian, that as an art historian, that we are missing the means of protecting this kind of artwork. For example, to be able to study the style or the gestures of some dancers from previous time. So that's one thing that the recording can, it depends on the purpose. It can be like archiving because of the research needs, but also it can be of course used as a kind of means of enlarging our possibilities of expression because human body has some means of enlarging our possibilities of expression, because the human body has some limited possibilities how to express, even within the dance. And the technology, for example, these different devices of visualization of human brain's activity, and EEG, EKG, EEG and so on, they can be seen as another means of using or expanded possibilities of human body used as a medium of expression itself. But then of course, yeah, so it's another thing. But of course, it can be also misused or somehow dangerous to collect this data. Yeah, it's everything. Life is very risky. So we've finished there to go on to the break. But we can return to these fascinating topics at the end. So we've got those. Felly, mae'n dda i ni ddiwedd yma i fynd i'r traws, ond gallwn ni ddod yn ôl i'r pethau hynod o ddiddordeb yma ar y diwedd. Felly, mae gennym hyn. Felly, diolch am gyflwyno'r ffynciau cymhwyso cymhwyso'r cysylltiadau o'r perfformiad yma o ran y perfformiad, o ran ymdrech, o ran y llifrwydd a'r gweithredau ac mae gennym hefyd y syniad hwn o'r automatisme yn y apparatus felly llawer o bethau i feddwl amdano. Diolch yn fawr iawn i bawb. Kepala Thank you. Thank you. Hello, back. Welcome back to the last track of Synesthetic Syntex. We will continue with four presentations. And the first one is by Lilian Kolinsky-Galellos from the United States and from Mexico, exactly from the border. And she will give an insight into a project that is still in progress and will be probably starting or be presented next year and we unfortunately could not have you here or you couldn't make it to Linz but you have prepared a very inspiring presentation and we will watch the presentation and then we will have the possibility to talk to you thank you before the sixth son there was already a land full of life and color. The fifth sun said, grow, and life flowed out of every corner. Under the hot beams dwelt the cactus and their flowers, the apache plume, the Jericho rose, white sage, washokoyoling, chuparosa, dandelion, marigold, and devil's claw. So many more grew under the recurrent blessing of the southern breeze, caressed and watered by the raindrops of the north. We learn to see naturally occurring plants and herbs as weeds or as eyesores to the imposed version of nature that we call a garden. This is how current white supremacist ideology portrays and imagines natives, African people, and their descendants living in colonialist nation-states as perpetual foreigners, as weeds ruining the landscape. The evidence is in how we are continually represented in mass media and in films, and in how we are treated by institutions where individuals keep uttering, All lives matter, echoing a dissonant ignorance that will be forever recorded in history. Living in the desert area of the Inland Empire of California has opened my eyes to the stupidity that is to obsess over the maintenance of a green grass and flower-based garden, which at best is not desertic, and at worst has a dire impact on water resources. It is too reminiscent of living in accordance with coloniality, forcing what is not because we're told it should be. This is an example of how even through the choices that we make for the aesthetics of the home, it can be so easy for us as individuals to blindly and unconsciously follow supremacist ideologies and actually serve as local agents of the matrix of power of coloniality, upholding ideals rooted in hierarchical and exclusionary methods of white supremacy. To showcase the forces of coloniality and its counter-resistance through decolonial esthesis, or what Walter Mignolo refers to as the processes of thinking and doing, of sensing and existing beyond the impositions and absurdity of coloniality, and doing, of sensing and existing beyond the impositions and absurdity of coloniality. Before the sixth son, a codex for our children began with the idea of writing a children's book which incorporated audio, tactile, auditory, and AR components that would retell the story of coloniality, the system of ideas and ways of thinking that remains following the invasion, injustices, and issues of indigenous lives and land stewardship through the illusion of the relationship between native medicinal herbs and important settler flowers, represented by roses, which is the national flower of England, although, plot twist, it originates from Central Asia. The textile book slash codex is a work in progress produced over burlap and muslin fabric with designs and backdrops and embroidery and crochet using cotton, wool, acrylic, and silk threads. Before I can even begin offering an overview of my project, it is both my pleasure and burden to provide you with some historical, theoretical, and personal context, as some of the topics and peculiarities that have informed this project have been deemed as forbidden historically. Though this minoritized, marginalized knowledge, or this maybe very highly specialized realm, depending on how you want to see it, I will attempt to introduce information while I demonstrate examples of coloniality. One of the most prominent indigenous nations of Abiyayala was the one referred to as the Aztecs. Abiyayala is what millions of indigenous people call the same landmass that many of us have been taught to refer to as America, and it's a word that comes from the Kuna language used by the Guna people, who inhabit the region from northwestern Colombia to southeastern Panama, meaning mature land, land of vitality, flowering land. It was Takir Mamani, leader of the Animara people, who popularized the term Abiyayala because should we continue referring to our continent after an Italian 16th century explorer, Vespucci? As Takir Mamani has explained, quote, to give the names of foreigners to our villages, cities, and continents is the equivalent to subjecting our identity to the will of our invaders and to that of their hearers. End quote. I would add, the issue is not that it's simply a foreign name. It is the context of said name in relation to imperialism and the colonial history of the world. The way that it so nonchalantly upholds coloniality. the way that it so nonchalantly upholds coloniality. According to anthropologist and historian Robert Barlow, Aztecs were those who left Aztlan and migrated to the basin of Mexico. But what we refer to as the Aztec Empire was founded in 1325 by two main forces, the Tenochca and the Tlatelolcas, descendants who called themselves Mexica. The reason why you know them as Aztecs is because a Creole Jesuit teacher, Francisco Javier Clavijero Echegaray, used the term in his book Historia Antigua de Mexico, which was referenced by the German explorer Alexander von Humboldt in describing his own 1803-1804 expedition to Mexico in Views of the Cordilleras and Monuments of the Indigenous Peoples of the Americas. Later, William Prescott then popularized the word in the English-language world with his The History of the Conquest of Mexico. As you can see, the history of coloniality discovers a legacy of patterns of domination from the perspective of a Western European Anglo-Canon with a terrible inherited habit, a perverted logic described by Frantz Fanon in Wretched of the Earth, and I quote, Colonialism is not satisfied merely with pulling the people in its grip and emptying the latest grain of all form and content. By a kind of perverted logic, it turns to the past of the people and distorts its figures and destroys it, end quote. As a border identity, one that goes beyond inhabiting a physical border, Mexican-American Chicano, Chicana, Chicanx identity results from the westward movement and the invasion of Anglo-Dutch settlers into what we refer to today as the U.S. Southwest, with the final reach of the U.S. nation-state in its land grab from the Mexican nation and its imposition of the U.S.-Mexican border. Chicana border discourse is about our experiences while conscious of our indigeneity, surviving and living in an Anglo-dominated empire. Mexican people were differentiated from whites in what they were considered half-breeds tainted by indigenous blood. This hierarchical perception of Mexican identity as inferior, purged many families in varying degrees of violence to relinquish their indigenous culture and language and internalize the Anglo language, culture, and ways. This also happened in Mexico, with white Spanish settlers imposing Spanish language, culture, and ways over indigenous and African peoples. After all, one of the vilest projects of the U.S. westward expansion was encapsulated in the words of Richard Henry Pratt's 1892 speech delivered at the National Conference of Charities and Corrections in Denver, Kill the Indian in him and save the man. The violent enforcement of assimilation went beyond separating families in the name of civilizing children through torturous practices that murdered thousands of children in boarding schools. Coloniality came in variant doses and it manifested itself interdimensionally. Mexican people who were born in the lands that suddenly became the United States faced quotidian violence, forced deportings and lynchings. Children who spoke Spanish at school were asked to stop barking. Justifications for stealing and burning property, throwing rocks, conducting hangings and lynchings included walking around too proud of being Mexican, speaking Spanish publicly, or being what was referred to as too uppity. So yeah, doblemente jodidos, twice death over, the Chicano in their consciousness of their history as survivors of the disfigurement and distortions of their indigenous roots are left with the longing to reconnect with their indigenous self and origins. The Mexica Aztec mythology, being the most prominent in studying and recorded, has become the last vestige represented of the culture and identity that were viciously seized from so many of the children of colonization who struggle in their search, embraced also by Barrio and Cholo culture, as well as academic, theoretical, and philosophical proponents like Gloria and Saldúa. Parallel to the Negritude movement of the 1930s to 50s, when Francophone, African, and Caribbean writers such as Le Foult Sedar Señor from Senegal and Messé Zer from Martinique and Léon Damas from French Guiana living in Paris expressed protests in their writing and artistic productions against French colonial rule and assimilation by embracing an Africanity, a cultural identity once lost due to the historical trauma that was experienced intergenerationally. Mexican Americans also had the Chicano movement that burst in the 60s, pushing for a new identification while advocating for social and political empowerment through borrowing mainly from Mexica and other indigenous mythologists of Abia Yala. So Before the Sixth Sun is part of this legacy and is working on the continuous efforts of having our version of history told. And to me, the way it is told, it is equally important, if not more. Before concluding with the details that make this a piece of experimental expanded animation, you must know that before the sixth son, a kodesh for our children is in homage to indigenous kodeses, which were painted colorful pictographic books, which recorded precious knowledge that came to be forbidden during colonialism and beyond. Only 23 codices survived. Some were burned inside the buildings that kept them called Amoshkali. Others were burned in piles during acts of faith because they had been painted and created under demonic inspiration. During later colonial times, indigenous people converted to Christianity would then report the existence of codices hiding amongst the people. Later when the Catholic religious leaders decided they wanted to use knowledge of the Bucco region to indoctrinate them further, they began to commission codices that would include their idolatries and their stories. These new codices were based on the codices kept by wise Indigenous knowledge keepers who would bring the books along to read them and explain them to the Spanish authors who would then immediately destroy the original documents. The original antique books surviving were all sent as gifts to the kings of Spain and many are still held as plunder in Europe. We only have two of them in Mexico. By the way, in Austria you have our Quetzalpanekayotl, also known as the Hdress of Moctezuma, and the Weltmuseum Wien. With this presentation, I ask its prompt return. And while we're at it, amongst these, I would also like to demand the return of two deerskin 15th century Mixtec Codex titled Becker I and II, one sculpture of Quetzalcoatl, one starved stone piece used for ballgame, one representation of Chico Mekwati, one green stone mask of Chipetotec, one shield which belonged to Aizoti, Montezuma's predecessor, as well as multiple Mayan funerary pots, Olmec masks, and jade figures, Mexica and Zapotec sculptures, among many other unique Mesoamerican objects. But I digress. Before the sixth sun is also alluding to an important aspect of Mexica mythology. We are currently living under the fifth sun, Nawi Olin, or fourth movement. Nawi refers to the previous four suns and Olin to a movement. It is foretold that the fifth sun will end in a series of earthquakes and one specifically large one that will lead to famine and darkness. It is a moment in the history of humanity and creation where things are both orderly and chaotic, where earthquakes are shaking off the ideologies of global oppression. The period of famine and darkness has been quite long for much of the world. The fifth sun began in the time of colonialism. It is a weakened sun, one that has required lots of blood to survive and for humankind to begin to see the full picture, the light of the sun. It is foretold that at the end of the fifth sun, cosmic equilibrium will be achieved. Some believe that the arrival of the sixth sun will be marked by the return of Montezuma's headdress, which makes sense, because it would mean that the hearers of colonialism would have understood why it was so important to return it. Before the sixth sun can rise, the world needs to learn and come to terms with all that happened, and come to understand how the formats of supremacy and oppression still remain, how we as individuals can be working to uphold them without even being aware. As is with indigenous textile tradition and Trans-Frontaresa knowledge production, Before the Sixth Sun creates an opportunity for a non-linear morphing genealogy of presentations of fragmentary versions of this piece, because the way the piece is exhibited will continue to change as it is embroidered and being completed, always in a nonlinear way. Initially I thought of creating a multi-sensory children's book as a way to tackle issues of accessibility. The completed textile codex will be composed of at least 35 pages of 18 inches wide and 14 inches high each. As I continue to work on this piece, the pieces will not be attached to each other so that they can be arranged in different ways, as long as the order is coherent and there is continuity expressed even through the space of the venue itself. Once it is finished, the piece can be showcased spread out with a motorized roller that can roll the text as the story is heard via audio. In addition, each page will be accompanied by one to three lines of text, as well as a 3D printed version of the surface of the textile so that people can touch it at their will, as the original piece is a work of art that I would like to preserve as it would have taken me years to produce. In addition, when a person places a smart device over each of the pages, it will bring augmented reality elements to the story. The herbs and plants come to life with blinking eyes and mouths. For example, the breeze will move through the page as well as come out and be seen by the person in the environment around them. In addition, the roots will be seen coming out under the page in the museum environment as layers of unexpected movement and light. The sound effects and the audio will include the drone effect of an earthquake at pivotal points of the story where supremacy and homogeneity take over the visual parts. I plan to include an Arduino Interactive Diffuser that can receive signals at appropriate times to diffuse the scent of white sage and copal resin and the scent of roses when the introduction of the imports and finally a mixture of scents for the ending of the story into the sixth sun. The colors in the animation as well as in the textile go from colorful shades of greens and vibrant cactus flower colors slowly becoming the homogeneous dim pastel that overtakes the entire landscape as the vegetation comes to the realization slowly that homogenous color becomes imbued with even more color variations into a rainbow-colored sixth sun. The same effect will happen with the music, which has been created and produced by Edmund Musicologist Juan Carlos Portillo and which you have been hearing in the background of this presentation. He has used anthropologically accurate reproductions of Mexica instruments to play the first part of the song, European sound components during the tension and conflict, and finally a song mixing sounds from indigenous and African and European roots. The color and sounds, as well as the textures, provide an effective field of complexities involving issues of supremacy, homogeneity, coloniality, resistance, tradition, and liberation. The transborderate aesthetic that prolongs itself beyond the page, mixing analog with visual, static, mobility, and interactivity, helps to create a changing text that produces the experience of nepantlat, or the liminal space, the in-betweenness of the border experience that produces the double consciousness that is divergent and multidimensional, complex, and inflowing conflict. Furthermore, the burlap represents the bounty of indigenous and feminine, as well as feminist forms of data storage and storytelling, alluding to the presence of hidden elements found in the augmented reality portion. Here, direct and unfiltered decolonial roots are not shared underground, but through an invisible wireless technology around everyone. The digital element makes it possible to universalize the function of Chicano and indigenous pedagogies of the home, making them public and seeing them as futuristic. And I quote, communication practices and learning that occur in the home and community and serve as a cultural knowledge base that helps Chicanas negotiate daily experiences of sexist, racist, and classist microaggressions. This represents the overlapping versions of history I received while I was a child at home, recreating the dissonance I experienced as a Chicana trans fronteriza and Kumeyaay descendant while receiving the legitimized, whitewashed version of the coming of two cultures, the European and the indigenous. With this codex, I am making a case by recording the lesser-known depths of coloniality. The design at the bottom is the design found in the Codex Huichol Cinco of 1531, a codex used in the court of law which included a detailed list of deaths, which led the Nahua people of Cochocinco to win a legal case against representatives of the abusive Spanish colonial government in Mexico. For me, working on this artisan multisensorial textile AR art animation piece recreates the sensing of the embodied person of color experience in receiving overlapping oral histories and constant movement with the distance from receiving a static legitimized version of history from the outside. Finally, this piece of hyper design mixes folk artistry with technology exposing the artisan's hand as a counter story to mass production and capitalism in the uniqueness of presence, making the moment and the event more valuable than the object itself, and bringing materiality back to its connection to ritual and meaning, as well as community building. This work in progress is scheduled to be part of Mexicali Bienal exhibition, The Land of Milk and Honey, with its first installment at the Chichamarin Center for Chicano Art and Culture of the Riverside Art Museum, slated to open in late February 2023. As part of the included works, it will become part of the U.S. Library of Congress Mexicali Biennale Archive and Catalogue, and you're all invited to come visit. Finally, I want to say thank you to everyone involved, and especially to Brigitte Josia and Juergen Hagler, and the committee for this invitation is each and every part of the Synesthetic Syntax Gestures of Resistance Symposium, part of the 10th Expanded Animation Session of the ARS Electronic Thank you for this great presentation, insight into a very important topic that you are tackling. And we are really looking forward to see the start of the project, the first exhibition. And we are also very looking forward to have the possibility to maybe bring it to Linz, to Austria. to have the possibility to maybe bring it to Linz, to Austria. And you can, as you have mentioned, this object in Vienna. You could also take the opportunity to go there and talk with them. I would like to start with a first question in terms of animation. You mentioned augmented reality, expanded animation. Can you give us an insight into the animated world of the project? Well, for me, animation is really just a tool. The point of my work is the theory and the history that is embedded within it that I try to express in different ways. Like I'm also a writer. I'm also an artist. Yes. I'm also like a community, you know, hub for my students and the community around me. So animation is just another tool that I would use to be able to bring you into that life experience or that experience that is usually not present in mass media or that is ignored because it's seen as something that is irrelevant to this future modernist perception of the world. So for me, I didn't want to go completely in animation. I wanted the animation to kind of be at the surface because it represents that dissonance that I experienced. Like I said in the video, it represents the dissonance that I experienced as a person of color. That's what we're called here in the United States. I don't know, you know, but that's what people who are not white are referred to as. And that the way that I grew up in my home and the things that I learned were very different from what I was receiving from the television set, but also from the school and from just the communities and stuff. So the communities outside of mine, the hegemonic ones. So then what ends up happening is that I think, okay, the augmented reality works for what I'm trying to show, which is this overlap, right? You see a static image in front of you, you don't think there's anything else. But if you use a program to show the augmented reality, the roots coming out. So for example, you saw the first piece, the first page, you don't see the roots there, you know, because the roots will come in through the AR component as you're observing them through that object, right? But in a way, I also want to express how our ways of doing things that are for the purposes not only of survival but of preservation, they have been present, ongoing, and just because people cannot see it doesn't mean that it has not been there and that it hasn't been strong, you know? So that's the hope. maybe I hope that this answers the question like this is why I am using augmented reality in this way you know that it it creates this like border feeling but it's trans trans border it's surpassing these borders through different dimensions simultaneously. Thank you. Are there any questions? Further questions? Helen? Hello, thank you for your wonderful presentation. I have a question for you, for someone who is as well. I'm half indigenous. And I also work a lot with virtual reality and augmented reality, because I feel that these are, in my particular tribe, that these mediums express my thinking the best especially in in uh thinking about liminal spaces which is why i work with them and i can see that you're nodding that you understand what i mean but maybe would you like to speak a little of this about what it is with us that really connects with these liminal spaces? Sure. Thank you so much for your question and for sharing your own experiences and for making me feel like, you know, we resonate in that way. I also, I can understand what you're saying. I also, I can understand what you're saying. We are a very musical people and, you know, history was passed on through songs. And so this is resonating with our way of thinking and storing information, but also sharing it and maintaining it, right? This is our digital realm. And so it has existed before, you know, the internet and all these things, but we didn't we even need them. That would be my question, you know. So I can share with you. Maybe I can tell you through my own experiences how this works for me. So, for example, when I was in my dissertation, I was doing my dissertation in UC Santa Barbara. when I was in my dissertation, I was doing my dissertation in UC Santa Barbara, and I was asked to express myself in a thesis. And that was very difficult for me to articulate what I was seeing, what was in my brain, the visualizations of the images fighting with each other at the border. My anger was with the way that Tijuana was portrayed in mass media, but also like in the way that people in the street would talk about it as if they were from there, they've been there, because it had a bad reputation. So then I thought, okay, have these people ever been there? You know, it was like anger that I felt. And I saw these images in my head of things that I've seen on TV, words that I heard in the radio, things that I've seen in history books, comments, stupid comments. So then I thought, I want to write my thesis about this. Had no idea how to articulate it in the language of the place of the institution where I was. So then my professor said, well, you know, if you work visually, why don't you just try to pretend like you're making a cover for a newspaper or a magazine for your thesis? So then I created a collage. And then when I did that collage, then we read it together. And from there, I developed my thesis statement, my methodology, what I was, you know, everything. my methodology, what I was, you know, everything. And it was so easy, you know. Then also, for example, the image that you see back here, this is how I design. So this is a design for a VR project that I also made called the Cuyochao Comparative 2020, which is to show like the ongoing parallel experiences of women of color in the Americas in relationship to loss, but also the different dimensions of coloniality. So I'm tripping this in my brain every day, you know, and so it's kind of like, here, come in, I'm inviting you to my brain for a little bit, check this chaos out. But it's an organized chaos, but this is that you won't understand all the symbols that are there because i want to welcome you into that moment where you're like what does that mean that's how i felt many many years of my life so it's kind of like trying to show this way and create an empathy so i i hope that this kind of expresses a little bit of how i i experienced that myself thank you so much for your question thank you for your answer i think we have a time for a very short answer and question is there a one further question pegitta hi liliana i was just wondering you talked about 3D printing and so that people could handle this book in the museum. I just wondered if it would be the same experience of touch as the one from the original fabrics and what you think about that. It would not be the same. Thank you for your question. Sorry, Brigitte. It would not be the same. Thank you for your question. Sorry, Brigitta. It would not be the exact same experience. No, because we receive history in fragments. We receive like ghosts versions of our culture. It's in pieces, at least for me. And I've had to be recovering these pieces. It's not like I could touch or feel who I am. It's like I'm constantly in that search to figure out this part of me that has been detached or taken from my family for many, many generations, for several generations not many but several and so when I print out a version of this you will receive textures that are different from each other in the printing but no you you will not be able to have the feeling maybe that will also create that longing that you may sense and feel what I feel you know with that longing to touch or to be where, you know, others are in terms of their history and their ancestry. For us, also for the people in the diaspora, many don't even know where their families originate from, right? Because slavery, enslavement of human beings forced people out and completely cut them off. But something similar happened also with indigenous peoples in the Americas. You know, not all of them went to reservations. Some were, you know, the men, the European men would steal the women, take them away, away from their people. These kind of things happen. So it's kind of like my way of also saying here, you can touch as much as is possible, but at the same time, you won't be able to touch the actual physical object because again, also it's precious at this point, right? It also has its own value as something that took, will take many, many years to finally complete the 35 pages. So yes, that would be my answer. It's part of the show, Brigitta. Liana, thank you so much for joining us. We had really the feeling that you are here with us. joining us. We had really the feeling that you are here with us. Please submit to Priya's Electronica next year and let's continue this discussion next year again. So have a nice day. I think you have breakfast. Yes, I'm going for breakfast now. Thank you so much. Take care. Bye-bye....production as we are late. Next talk is entitled from screen to space the haptic experience of exhibited animation. Wes Anderson's piece Fantastic Mr. Fox is the case study. And presenter Farsaneh Yasandus, PhD candidate from the Film and Media Department at Cambridge School of Creative Industry. And we'll start the presentation. Hi, my name is Faazaneh and today I'm going to talk about the embodied experience of visitors in Wes Anderson's exhibitions where the settings and puppets of his animations are displayed. I discuss the difference between perceiving the sets and puppets on the screen and encountering them in the exhibition. To do so, I first argue that the images of Anderson's animations are highly haptic due to the techniques he employs in his animations. Then I will discuss that visitors unable to touch the sets and puppets in the exhibition could gain a haptic experience in the absence of those techniques as well. Anderson's animations are populated with different types of organic textures, which are rough and bold. In addition, textures are even more highlighted through using camera's capabilities and specific condition of puppet animation production process. For instance, the image's deep focus makes every part of them sharp. This amount of focus is difficult to be achieved in puppet animation due to the very close distance between the miniature sets and the camera. sets and the camera. Tristan Oliver, the director of photography, explains that in addition to use different ranges of lenses, different parts of one scene are shot separately through the assistance of a green screen and combined in the post-production to achieve sharp focus. This technique has been used in the first scene of Fantastic Mr. Fox in a shot that depicts Mr. Fox's point of view. This shot is made out of three separate sets depicting the background, the middle ground and the foreground. Although the scales of the objects textures and the puppets are different in each layer to create the illusion of perspective, the consistent sharpness of all parts of the image makes it seem flat. Moreover, Andersen is well known for his symmetrical compositions and his stop-motion animations are not an exception. The symmetrical compositions cause the image to seem flat. What reinforces such flatness in his animations is the position of the camera which is primarily perpendicular to the sets, particularly in the scenes where the camera is apparently horizontal. In Bird's eye-view shots, the camera is horizontal and perpendicular simultaneously. The opening credit of Fantastic Mr. Fox is particular in this respect, which depicts Bird's eye-view shots of farmland. Another instance is a shot of Isle of Dogs, which illustrates the crash of Atari's plane. The flatness of the images in Anderson's animations carries out a significant function. It causes the bold and rough textures to evoke a sense of touch in viewers. The sets and puppets are mostly constructed through organic physical materials instead of resin or clay, which makes the images textured. These textures are underlined by being magnified when photographed. Since the sets and puppets are miniatures, the camera has to be situated very close to them. Such close distance magnifies the quality and size of the textures in the final image as if they are seen through a magnifying glass. For instance, regarding the opening credit of Fantastic Mr. Fox, the farmland is made of dyed towel with various patterns. The fibers of the towel stick together in some parts when thicker paint was applied. Being magnified and entirely visible, the shot reveals the type of material and the brush by which the towel was dyed. Laura Marks regards flawed images as haptic due to being two-dimensional, which is specifically capable of conveying tactile quality. Comparing haptic and optic visuality, she contends that with optic visuality the eyes seeing things from enough distance to perceive them as distinct forms in deep space, whereas haptic looking tends to move over the surface of its object rather than plunge into illusionistic depth, not to distinguish form so much as to discern texture. In Marx's account, images with deep depth of field that create the illusion of three-dimensional space are most attractive to the eyes. In contrast, the texture is more emphasized and perceivable in flat images representing surface instead of depth. Her argument is that flat images are the most attractive to the sense of touch because the viewer's eyes move over these surfaces and stimulate a response akin to that of a caress. In addition, different types of movements such as the movement of puppets, camera movements, and a type of motion in flowing materials that I call arbitrary motion lead viewers' eyes to move on the surface of the images instead of staring at a specific part of them. Jennifer Barker likens the inclination of the eyes to move over the surface of a haptic object to a caress. She notes the viewer caresses by moving the eyes along an image softly and fondly without a particular destination. In Barker's account, camera movement and the movement of actors' hands reinforces the act of caressing. It is worth mentioning here that Marx and Barker's account of caress appeal to a form of erotic, whereas I use it in a different sense to highlight the way the texture of a surface is examined and understood better by moving the palm of a hand over the surface patiently for a couple of times, focusing on how it feels. Certain movements in Anderson's animations, specifically in the shots of Puppet's hands, function similarly. For instance, the scene in which Mr. Fox tells Ash that he loves him as he is, the way he touches Ash's face conveys how it feels to touch a furry face. Arbitrary motion is a specific type of motion to puppet animation, particularly Anderson's animations, that is rendered to animate a constant touch on puppets, manipulating them frame by frame. Arbitrary motion as such stimulates caress. Fair and fabric, through which puppets are covered, are flowing materials, so their position can be changed arbitrarily each time an animator touches the puppet to change its position. As all of these changes are submitted in each frame, along with the changes in the position of different parts of the puppet, a secondary motion is rendered. The arbitrary motion of puppets' flowing parts is a mark of animators' touches and thus haptic because it evokes the feeling of touching flowing materials. Also, the arbitrary motion causes viewers' eyes to move on a puppet instead of being fixed on a specific part of it, and stimulates the sense of touch. It's typical of Anderson's animation in which arbitrary motion is rendered not only in the moving part of the puppet but also in the stationary ones. In conclusion, the quality of the textures, the miniature settings and puppets, the function of the camera as a magnifier, the flatness and extended focus of the images, and the arbitrary motion of flowing materials make the images of Anderson's animations haptic. The eyes become organs of touch that see and touch at the same time. Such synesthetic engagement links the constructed world on the screen to the physical world of which viewers' bodies are a part. Anderson also exhibits some of the sets and puppets from his films in real spaces such as galleries and museums. Mr. Fox's study room, which is an exact copy of Roald Dahl's actual study, is located at the Roald Dahl Museum and Story Center. Also on the 30th of March 2018, when the Isle of Dogs premiered in the UK, an exhibition was organized at the Store X gallery to display 17 sets of the animation, accompanying the puppets. Similar to animations, the settings on puppets in the exhibitions could not be touched. In I Love Dogs exhibition, some lines on the floor in front of each set kept visitors from getting too close to the miniatures. Also in Roller Museum, the study is protected by a glass which separates visitors from the set. In contrast to the animations, no cinematic techniques fill the gap between the miniatures and the visitors in the exhibition. Neither movement nor camera is used to magnify the textures and underline the physicality of the materials employed in the sets and puppets. However, visitors can encounter the miniatures in the exhibition more overtly than the animation. Most of the viewers of stop-motion puppet animation indeed know that the sets and puppets are miniatures. Still, there are rarely direct references to their actual size in the animations. There are restricted attempts to conceal such references to keep viewers' attention within the world of animation. However, the direct encounter with the miniature sets and puppets has implications. miniatures, sets, and puppets has implications. Exhibiting sets and puppets in real space gives visitors plenty of time to see and scrutinize the miniatures. In a review of the exhibition, Ewan Franklin mentions the prominent role of time. He writes, the level of detail injected into every one of Anderson's gloriously symmetrical images can't be fully absorbed in a single viewing. In the animation, some sets are only shown for a couple of seconds, so visitors might not even remember them when encountering them in the exhibition. This unlimited time to visit all the aspects of the sets and puppets reverse the process of recognition and haptic perception that viewers experience in the animation. Textures affect viewers immediately in the animations due to the techniques employed to underline them. The viewers are influenced before or at the same time they distinguish what the texture represents. Whereas in the exhibitions, they encounter the entire representation and perceive what they illustrate first. Then, as they get closer to the sets and spend more time exploring them, the visitors gradually engage with them haptically through the assistance of the details. Going through the reviews of Isle of Dogs exhibition, the most notable aspect of the exhibition in the writer's idea is the mind-blowing amount of details it can be found in the sets and puppets. For instance, encountering the setting in which the Mayor Kobayashi is taking a bath, one at first glance sees the set as a whole illustration. The mayor stands in a round top facing a wall covered by tiles that illustrate a spring scenery and two servants are heating water for him. Some less significant objects such as the phone in front of the top and the wooden bench are seen at a second glance. For the viewers who have already watched the animation, the size of the setting can be a surprise, comparing how it seems bigger on the bigger screen since there is no reference to the size of the sets and puppets throughout the film. Some visitors immediately take their cell phones and cameras out examining some possible frames or trying to find the exact angle they have seen in the film. They then try to get closer to explore more. Using a camera is also a way of having a closer look at a set. Some just drag their necks to get closer trying to stand behind the white lines while some others cross the line inevitably to try the rare moment of examining actual miniatures at the cost of receiving a warning. In Marx's account, such an unavoidable desire to get closer is the inherent quality of miniatures. She claims that details created throughout miniaturism evade a distanced view, instead pulling the viewer in close. Marx also alludes to a proliferation of figures that miniatures are fair, which perhaps is due to the small size of the figures in physical miniatures that without a magnifier one cannot see the textures exclusively. Marx concludes that, encountering miniatures, the viewer perceives the texture as much as the objects imaged. One can see how this might be the case with respect to the Andersen's exhibitions. When we spend a long time looking at the sets and puppets, we come to face fragments, specifically the physical materials through which the sets and puppets are made. This is due to the fact that meticulous details appear gradually. Here, spending time becomes equal to getting closer to the miniatures. Gaston Bachelard's account of miniature details chimes with the gradual appearance of textures in Anderson's exhibitions. Batchelor notes, details reveal themselves and patiently take their places one after the other. One has to take the time needed to see all these little things that cannot be seen all together. So, for instance, in the set of Mayor Kobayashi, we do not perceive the details of the two puppets in kimonos, the texture of their kimonos, the fibers of their hair, the copper boiler, the wooden laden, and their footwear at the first couple of glances. These details reveal themselves to us slowly. As we spend more time scrutinizing them, they become separate fragments from the whole set and come into view one by one. separate fragments from the whole set and come into view one by one. The closer we get to the miniatures, either by reducing our distance between ourselves and the sets or looking at them for a long time with narrow eyes, more details and fragments come into view. Hence the FIGUS, Fade and what Marx regard as optical perception, which chimes with distance viewing, transforms into haptic perception throughout the exhibition. Haptic perception, attracted to textures, requires viewers to be close enough to an object to be evoked by them. The viewer's soul remembers how each texture feels just by looking at them while they can feel them on their bodies. Batchelor's account of imagination helps to clarify the engagement of viewers' memory in the process of perception. He also considers time as a determining factor in perceiving a miniature. In his idea, as a result of lingering over miniature images for a long time, one will experience a sort of coalescence of unlimited values. Values become engulfed in miniature, and miniature cause man to dream. The values he mentions include meticulous details which he believes foil the imagination. Batchelor does not put imagination in contrast to objective reality. He rather believes that imagination goes further than objective reality with its assistance. Objective reality is a gate that imagination passes through. Such a relation between objective reality and imagination can be best experienced when entering a miniature world. For instance, Bachelard refers to how a botanist describes a flower in the Dictionary of Christian Botany published in 1851. The botanist begins with an objective description explaining the flower's shape, color, texture, and other formal characteristics. After some lines of objective description he starts using metaphors and analogies or as Bachelard puts it the description is accompanied by a daydream. The botanist draws analogies between the flowers pistil and a woman, stamen and a man, seeds and children. He also draws analogies between their function and the way members of a family treat each other at home by anthropomorphizing the different parts of the flower, giving them human characteristics such as jealousy. Batchelor believes liberation, a special characteristic of the imagination, leads the observer to mix objective description with analogies and metaphors as he enters a miniature world. Batchelor then emphasizes that not all the observer who look at the same flower on their magnifying glass could see the features of the flower in the same way that the botanist described in the second part of his account. However, there is an interesting point in Batchelor's explanation of how the botanist ends up drawing an analogy between the flower and a home. For instance, he mentions that the botanist ends up drawing an analogy between the flower and a home. For instance, he mentions that the botanist has felt the gentle warmth preserved by the fur. The feeling particular to the sense of touch that one has got as a result of seeing a texture is more common among viewers encountering the same object rather than the analogy they can draw between an object and the other. For instance, seeing fa is likely to bring warmth and smoothness to some viewers due to the memory they have of petting a cat or dog, wearing a fairy coat, or walking on a fairy carpet. But whether this feeling can remind one of a holiday house or someone else, a gloomy wet night in the wood is totally an individual matter. Marx regards haptic images as a subset of what Gildolus refers to as optical images, those images that are so thin and un-cliched that the viewer must bring his or her resources of memory and imagination to complete them. When viewers focus on miniature sets for a long time, the details of miniature settings, particularly textures, become very thin and at times isolated from the whole work. This focus works like a close-up shot in which even the fibers of a puppet's clothes are visible. So the visitors can guess what it feels like from how it looks on the puppet, such as weight, softness, hardness and other features that can be tested through touch. Here visitors' eyes are the examination tool and the memory is the resource. The visitor brings her prior experience of touching different materials saved in her memory to find out what each part of the setting would feel if touched. In addition as Marx also mentions about haptic images which force the viewer to contemplate the image itself instead of being pulled into the narrative, the sets in exhibition allow visitors to explore the settings disregarding what they communicate as a part of a narrative. As I mentioned before, in Anderson's animations, a balance is created between following the narrative and having a tactile experience of the sets and puppets through the assistance of cinematic techniques such as depth of field and close-ups, as well as arbitrary motion. None of them disturbs the other, they instead coexist. However, the exhibition aims to create a pure experience of the sets and puppets since there is no narrative to follow. Neither are any movements or cinematic techniques to highlight the tactile status of the sets and puppets. The bodies of the visitors, they leave the experience of the material world, the unlimited time of perception, and an unmediated encounter with the sets and puppets lead them to have a haptic experience. Thank you so much for listening. Thank you for your inspiring talk. And Joanie, thank you for coming back because we lost connection. You will be on screen and we have, of course, some questions. Do I see any? And first question, Birgitta. Pazone, how are you? Hi, Birgitta. Thank you so much. How are you? Hi, Brigitta. Thank you so much. How are you? I'm good. So I'm interested in your idea of haptic visuality. And of course, Laura Marks talks about the idea that touch and haptics can be used to evoke memory. And I wonder what you think about that in conjunction with something I've written about, which is the nostalgia of the craft in stop motion and the labor that's in stop motion and the kind of fetishization of some kind of memory of labor that is present in stop motion that you will see in the exhibition you're saying people are marveling at all the details so i'm wondering um what you think about that the fetishization of labor in stop motion um exactly i um talk about labor in stop motion animation through that arbitrary motion and the idea of nostalgia that Anderson brings into his animations through somehow forcing us to see some of the textures from a very specific time and that the corduroy suits and things like that. These are very connected ideas, I think, in specifically Wes Anderson's animation. And yes, I think that they are completely connected to each other. And in viewers visiting the exhibitions, we can see that idea, you know, with bare eyes, even more than when they are watching his animations. touch them but the the memory and um what i call um lived experience um is is more visible and um touchable in his animations another question from the audience audience. Thank you for the talk. I'm from Portland, Oregon, which is a stop motion city. And so Guillermo del Toro is finishing Pinocchio right now, and I believe there will be an exhibition. But I think about having seen the Leica show in Portland, which Laika was the studio that produced Coraline and Box Trolls and whatever, and seemingly has an endless font of money from Phil Knight of Nike. And so the level of craft has a kind of perfection to it that I see as different in Wes Anderson's, which seems rougher. And is there a way that you would differentiate those? Am I making myself fairly clear? Yeah. Yeah, thanks so much for your question. I think there is a very, you know, completely, they are completely different. I think the practice of Wes Anderson animation and Laika, because there are so many 3D CGI and these kinds of technology, 3D printing in Laika's animations, whereas in Wes Anderson's animations, everything is completely quotidian. You know, whereas in Laika Studios' animations, everything is manufactured through CGI and then 3D printing. That's why I think these two practices are completely different to each other. We have time for a last question. Thank you so much. Thank you so much. It's a very interesting topic, haptic perception in this context of exhibiting animation, stop motion, but also thinking about bringing animation in galleries as we here at Ars Electronica have this challenge from animation competition to go to galleries and we always have to find a proper way to present it in a gallery space. So there are a lot of things to consider and I like especially the idea of giving the audience the possibility to see the details. They cannot touch it but they can zoom in with their mobile device and you see really, okay, they would love to touch it. Thank you so much. Thank you so much for having me today. Bye-bye. Thank you. Bye. Thank you. So we will continue in real space with our next presenter, Yuichi Nagashima from Japan. He is a composer, researcher, professor, and doing a lot of things with music instruments. He has composed over 100 choral music pieces. He is a professor. He is a key member of Japanese computer music community, et cetera, et cetera, et cetera. Welcome on stage. Thank you.お、OKHello, I'm Yoshinagashimaあ、これやFrom Japanこんにちは。私はヨッチ・ナガシマです。日本から来て、奇妙な音楽を持ちました。あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あ、あズムで書いたのです。そのため、プロの方に、インプロビゼーションでプレイすると言いました。この3つのスコアを選びました。そのため、彼女にインプロビゼーションで 各フェアを選択してくれと言いましたこのフェアは 私の学生の中に 身体にあるセンサーがたくさんありますなので スコアはグラフィックスタイルです私が作るオリジナルの 特別な音楽の作品は私の作品の一部ですこれは32チャンネルのテレビこれは私のプロデューサーですこのフラッシングドラムはニューヨークで演奏しましたこれはもちろんフラインストラメントです最後に、アタウ・タナガさんの名前をお伝えします彼は私の友人です彼はバイオミューズ、バイオセンシングインストラメントを開発しました私はミニバイオミューズを開発しました私が作ったものと、自分で作ったものです16チャンネルのEMGセンサーです私は、ドイツ、カナダ、ネザーランド、フランス、台湾、ロシア、ノウエーなどの多くのパフォーマンスを演奏しました最近、私はコンピュータ音楽の研究者で、このように、両手にMU-EMGセンサーを使っています。MUSE、オープンBCIのブレインセンサーです。これはロシアでの演奏です。今日のテーマは、タクタルインタラクションです。手と指は特殊なセンサーとアクチュエーターです。ピアノ、ギター、サックスを使って、人間にとっては、このチャンネルは人間にとってこのチャンネルはとても重要ですこのシートは私の主な意見ですタクシャルインタラクションは笑いをしていますこれはエモーションです最後の結果は私たちの健康・健康の気持ちですThis is emotion.So the final result is our wellness, well-being, feeling.I talk later about this later.Tactile interaction.So at first, this is a…OK.どうしたらいいんだ。こうやって…こうやればいいか。これは日本で発表されたPAWセンサーです。このようにウレタンの形で指で、指を軽く操作します。この中には、2つのLEDと2つのオプティカルセンサーがあります。ウレタンの形状を押して、光の濃度が変化します。このセンサーは4つのパラメータを4つのエリアにディテクトできます。これだこれはこのインスタントの初の試験ですこの4つのバリューは、個性的にコントロールされていますそして、このセンサーで面白い作業をしています。このインストレーションのテーマは、柔らかく、可愛く、セクシーな声を作ることです。この声は、自分で録音しています。センサーは、音のバランスを、実際に調整しています。全てのプログラムは自分で作成されています次だこのセンサーは日本で唯一売られていますが今すぐに日本からインターネットショップを買っています。20ドルです。私は多くのインターフェースを発展していますが、センサーインターフェースの中にはソフトフィールが失われています。例えば、トレーニンググリップは最高値を制御していますが、このセンサーはソフトなニュアンスを発見し、指の反応することができます。とても興味深いです。私はコンピュータビジョンが好きではありません。私はそれらを身体に分けたのです。そして、それほど長いレイテンシーで、コンピュータビジョンセンサーが好きではありません最後のプレゼンテーションの部分でイントロセプションと呼ばれるものを紹介しますこれはキーワードだと思います違う、違う、違う4世代のプロトタイプを開発しましたこれは最初のプロトタイプですそして、これからこれからこれから。このセンサーを紹介します。このインストルメントはMRTiと呼ばれています。このパッケージは、卵のようなプラスチックケースです今、私がやっているのは、 現実的なデモです時間が足りないので、一人一人でやってみてください。でも、時間が足りないかもしれません。さて、行ってみよう。これだ。これだ。さあ、どうだ。音はどうだ?10本のセンサーがありますこのように10本の指がコントロールできるのですこのアルゴリズムはこのアルゴリズムはこの情報をリアルタイムコンピュータグラフィックスに生成します。このグラフィックは、フラクタルアルゴリズムで発生されています。マスマティカルで発生されています。音は動物の声の一種のものです。音の生成アルゴリズムは、フォルマントシンセスタイプの一種のものです。このセンサーの作り方は、自分で作ったものです。この写真は、私がモスコーバーでのワークショップです。ロシアでワークショップを行うことはできません。ロシアでのワークショップですこの画像は映画ではありませんがここにデモンストレーションをしていますこのタクタルインタラクションは笑顔をすることです笑顔は感情や気分を意味しますこれは私のハードウェアスペシャリストのハードウェアでのスケッチです私がこのセンサーを持ってきましたこのセンサーを使っている 多くのスペシャリストがいますMITでの研究者が多く インジニアやGoogleやインター、マイクロソフトの エンジニアなどの人々がこのセンサーを使って 音とビジュアルを体験していますみんな笑っています子どもたちのように笑っていますとても感動しました。この触る感覚、触る感覚、軽くのみの反応が人々に笑顔を与えました。そして、このシンガポールのアートと科学のミュージアムで、私は言いました。子どもたちも笑顔をしています。このキーワードはインターセプションです。後で話します。次に、私はこのキーワードを発表しました。8チャンネルです。しかし、これは音楽のインストラメントではありません私の計画はこの機械はウェルネスエンターテインメントシステムですこの8つのチャンネルのセンサーですこれは撮影ですこれは8チャンネルのセンサーですこれは撮影しています全て手作りですここでこのコンセプトをご紹介しますこのコンセプトはソフトタッチと半レベルの…This concept is soft touch and half level.OK OK OK やりましょうを 受けとれを止めてだほっin インディシステム0 h チャンネルフォークがプルー ヒアの実はワンセンサーいやー インフォメーションいいんこのような情報が…この4つのレベルのレベルを ソフトに使ってみてくださいそして、この4つのレベルが 同じですこれが最高の位置です。実際のグラフィックでは、この赤い部分を見てください。軽く押して、レベルは同じです。ブラックホールに入ってください。音のフレークが低いです。最後の目標はこれですこの8つのポジションがブラックホールに移動しますしかしこのシステムはリアルタイムレンダリングがとても難しいです。このコントロールは、スポーツジム、深いリバーブレーションで、ピッチとソロを入れます。この音楽を弾くとき、この画像は、非常に暗い空間と大きなスクリーンです。良い感じがします。今日は時間がないので、次の画像をご覧ください。この画像は、タクタイルインタラクションを小さくすることです。2019年にデトロイドを持ってきました。このスペシャリストたちが使っていて、笑っています。笑っています。They all smile.So this physical soft reaction makes our feeling very happy, I think.So, okay.And finally, this sensor,This is using only two sensors.これは2つのセンサーを使っていますこのページは日本語で書いてありますが翻訳プログラムで書いてありますこれはオープンソースシステムですみんながこれを作ることができますプログラムの中にオープ作ることができますプログラムの中にWebが開いていますこのCPUボードやセンサーを買ってプログラムを開いてみてくださいみんなが作ることができますあ、違う、こっちじゃない。すみません。ここ。W。これはデモンストレーションです。2つのセンサーが4つのバリューをコントロールできるように。4つのバリューを作りましたこのようなものを作りましたどうやって作るかというとページが開いています20つのセンサーを作りました Then I produce total 20 this type of sensor I have. And I bring everywhere, many universities, many hospitals. I have a workshop with my collaborator of the biofeedback therapy, biofeedback rehabilitation specialist.バイオフィードバックセラピーバイオフィードバックリハビリテーションスペシャリストコラボレーションプロジェクトを行っています私は多くの大学や大学に行っていますでも私たちは病院でワークショップをドクターコースのセラピーSo using thisFor example this oneControlling two sensorsコントロールする2つのセンサーです1つのセンサーはポジションを変えることをして2つのセンサーはサークルの値をコントロールしますこれはイリュージョンを作ることになります。このセンサーを使って、私の学生が作った作品です。So, the sensor softly controls the cute animation character moving.This is a very simple program.Like this.Other example, Ogata-kunのなんだっけ?OK.これは、アハハ、このセンサーが私の顔を操作しているのです。カレイドスコープ。彼らは私の顔が変わっているのを操作しているのです。私の顔が変わっているのを確認していますこのプログラムを作りましたこの2つのセンサーが正確な位置にある場合、左側の画像が開くことができます。これはスコアです。センサーが良い位置にに当たると、写真が開くことができます。私たちは、このビオフィードバックリハビリテーションスペシャリストと話しました。これは、老人にとって良いです。何だっけ?まあ、そうです。リハビリテーションブレインのリハビリテーションこの写真は、母親の頃のことですブレインのリハビリテーションは重要です同時に筋肉の操作や視覚、音などのチャンネルが活性化され、その同時に脳のリハビリテーションにとって良いです。このツールはこのように発生されました。 is very good for the brain rehabilitation. So this tool is produced like this. Yes, this one, this one. Okay, so this I want to discuss to the keyword interoception. We have many sensor channels. Exteroception means ear, nose, mouth, eye.エクストロセプションは 耳 鼻 鼻 鼻の多くのセンサーチャンネルがありますエクストロセプションは 特定のセンサーがありますこのセンサーは エクストラセプションと呼ばれます違うな、どれだっけ例えば、この視線は脳にネーロンと接続するのが簡単です。しかし、このチャンネルは、例えば、血液についての感情についての関係です。このようなものを探しているのです。このようなものを探しているのです。アタックですリサーチそうですん 実はもでううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううううまずはこのモデルですこれはソマティックマーカーハイポテシスです通常の感覚は身体と感覚です細かいヒポテシスは直接的な情報ではありませんが脳は何が起こるかを考えていますこれは高速的の決定ですこの2つのセンサーですこのセンサーでは強力な押しをしていませんスポーツチームでは、このようーは、プッシュだけです。しかし、このようなサンサーは、軽くプッシュして、軽くインタラクションを発生します。この軽いフィジカルフィードバックは、インタラクションをすることで、感情は柔らかくなります。 So physical feedback makes interception and our feeling is very gentle. And we have a smile. I love this situation. So my recent research is this rehabilitation while we're reading. And what time remains? Okay.このリハビリテーションは良いです時間が残っているか?大丈夫ですか?私はこのスケッチハードウェアの 特技の専門家です多くのデザイナーや研究者が集まっています今年はこのようなことはできません私はこのような多くのワークショップを 開催しています昔はインタラクティブマルチメディアの ワークショップでしたが最近はこのような リハビリテーション・ウェルフェアのワークショップを開催していますレハビリテーション・ウェルフェア・ワークショップ例えばこのワークショップはグリスプセンサーでキャラクターが行ってこれはプロフェッショナルマッサージテクニックシステムがディテクトしてテクニックを知っています。人々が良いマッサージをすると、 成果が上がります。これは、呼吸センサーやハートビートディテクトセンサーですこのシステムでリラックスしているので、緊張しているのですこれはロシアで4つのワークショップを行っています I make four workshops in Russia. This is Moscow, and this is Poland, some years ago. So yes, call me, and I will do it. OK, that's all. Thank you. Thank you so much for this wonderful presentation. Please come over to me. My colleagues will help setting up the next presentation. Okay. Okay. Okay? Okay, okay. I think we should use your interfaces every day and everyone so that everybody is smiling. So please push your research. I would also love to have this interface as a sculpting and animation tool. I think everybody agrees. If Ars Electronica calls me to bring here, I can come next year. Perfect. Mr. Nagashima, so a serious question from the audience. Yanis. Thank you for your presentation. Please speak slowly. Yes. This thing about smile. Have you tested it or some other, your student or after the workshop, have made a system which maybe is not funny? Because it is one of the reasons maybe people smile i don't know maybe sounds of some kind of some some you know a stressful sound or stressful image how that would affect the result have you but but you feel that sound is not good butでも、あなたは音が良くないと感じます。しかし、多くの情報をコントロールすると、非常に奇妙な音を生み出すことができます。人々は何を動かすか、何を体験するか、 I can experience, they go into the interaction, maybe. I think so. I did not think that specialists don't play like child, but there are many specialists. So after this, you can play this. You will understand, maybe. Is there another question to Mr. Nagashima? Helen. Helen. Hello. Hello. Do you believe that's a neurological connection between the touch and the release of happy endorphins, which then make the smile? happy endorphins which then make the smile. The opinion of the psychology is not directly connected. The interception, the physical reaction to our muscle reaction. When we make our muscle, the brain command directly here, but physical soft reaction of muscle is detected as an interception. Then interception is strong connected to our emotion and feeling. So then my opinion, strong connected our emotion and feeling so then my opinion, my challenge is the tactile and softly control makes us happy a smile I think this is my hypothesis thank you running out of time This is my hypothesis. Thank you. Running out of time. Thank you so much. And you will be around, so we have the possibility to test the interfaces. Thank you so much. Thank you. Yeah. So now we move over to our last presenter. Aristophanes Soulikias is an architect and animator from Montreal. Yes. And we will hear about visualizing the city in handmade film animations. Yes. Thank you, Jürgen. animations. Yes, thank you. A step back from the frontier of the cutting edge technology. Trying to slow down a little bit. Actually, it's a step sideways, I would say. So yes, I'm currently a PhD student at Concordia University, doing an interdisciplinary study combining my architecture and animation background. It's a research creation project, so there's theory, but there's also some art. Animation has always been a technological affair, and its evolution is clearly in response to the evolution of technique and the technical. Technology is there to facilitate, to save us energy and time, to offer comfort or even abundance of something. It can also offer completely new things that no one imagined existed or that were ever or even needed. In doing away with bodily movements and energy and whichever discomfort is incurred by physical toil or contact, humans have relegated physical activity, giving primacy to the cerebral and mostly the visual. This is true in most facets of life, and as an architect, I have experienced this distancing very acutely in being part of the first generation of architects back in the 90s who transitioned from the architecture school's drafting table to drafting on CAD in the architect's office. In suspecting that for anything gained with each technological advancement, something is lost, and by sensing that there is merit to imperfection, that in other words, the very values pursued by the progress may be questionable, I turned to architecture theorist, the Finnish, Juhani Palasma, who has written extensively on the value of hand drawing for architects and architecture, and considered these positions for the act of hand making for animators, for the ever-moving, ever-changing city, a place made of real materials and lived bodily experiences. In his books, The Eyes of the Skin and The Thinking Hand, he questions the domination of computer-made drawings, the role in design, and the total prioritization of the eye, and advocates the significance of the hand in both creating and sensing artworks, and particularly architecture. According to Palasma, the computer has confined design into a retinal survey, depriving it of the merits of that special relationship between the maker and the object, and thus compromising imagination. I quote, Imagination is an inseparable part of the handmade spatial-based practice. While the material computer image flattens imagination, the material-based craftsmanship is the very unconsciousness of vision, where the line traced by hand is clearly a unique act of making and a spatial one at that. Hence, ironically, it is through the tangible, I say, that one achieves the intangible, the essence of place. The distance, I quote, between the experiential and the flawlessness of newness and exactitude is manifested through the hand's uncertainty and hesitation, crucial states of being that foster curiosity and all of the necessary ambiguity that art hinges on. And while the computer has been proven efficient in many stages of architectural production, there is one where the spontaneous mind-driven gestures of the hand are crucial, and that is the early conceptual phase of any given project. If one thinks about conceiving the city, Palasma has expressed concisely the role played by the unconscious first steps. Atmospheric characteristics of spaces, places, and settings are grasped before any conscious observation of details is made. That sensitive stage in film animation, I would say, is the pre-production stage of the storyboard, which also seems to be resisting the switch to the digital in small animation studios, mostly due to the document's ability to prompt and focus cognition. I quote Palantin Price. I can give you the bibliography later. So as you can see, this created a theoretical framework for my work in research in film animation. An animated documentary I created in 2014 was already an indication to me how the city's layers, human imprint, and imaginary can be experienced and cut out silhouettes which I discovered at Concordia at my animation studies. And here are some production stills of the street facade of the St. Laurent Boulevard in Montreal and how the story was about the demolition of these historic buildings and how these burlesque dancers resisted the expropriation and actually managed to keep their burlesque venue from demolition amidst a horror of glass buildings. Excuse me for my bias. So and this is how it would look on the table, and how it would move. And it's cutouts, and it's also a series of photographs, basically of also collages that I made to represent this building, its traces, the human experience that was carried on it, construindo suas traços, a experiência humana que se levou a isso. Então... E depois, foi para 65 festivais, e eu percebi que estava em direção a algo. E outro projeto, de novo, I was on to something. And another project, again, just to give you a sense of what I have been doing. This is a collaboration with a music group, mostly Persian music. They play, and it was based on the Shahnameh, the part of Persian mythology, 11th century epic. And here I have a little excerpt where, again, I somehow even unconsciously incorporate the idea of the city, in this case a city in flames. After the conquest, it's part of the myth and destruction. And then questions of viewing arose, the possibilities of situating the work within space, make it site-specific, bring it into a dialogue with the built environment. And this is a workshop that I participated in, which was run by Rose Bond, who's here, and Pedro Sarazina, Dr. Pedro Sarazina, where here I was relating the built environment behind this glass window and some thoughts I had about what was going on. Again, cutouts, and it's projected on a cloth. And in neighboring Salzburg here in Austria, two years ago in an artist residency, which I had the honor to be part of, I did a makeshift light table and told the story of this neo-medieval tower by having these slices through the wall. And maybe it's too bright here to see, but one of the slits is showing an imaginary of the architectural intention, and the other one supposedly the actual function of the building, which was an ice storage place. As I began my PhD studies at Concordia University to research the building environment through film animation and through techniques that involve the handmade, I recognized the unique place film has with regard to the city. And the deduction was clear. If film is the medium par excellence in depicting urbanity, as claimed by numerous scholars from Benjamin Krakauer and filmmaker Eisenstein all the way to Giuliano Bruno and Richard Koeck today, and if hand-based ways of making images about the city, such as drawing, are those that connect more meaningfully our bodies to the built environment, then handmade based film animation must be the optimum medium for sensing and expressing the city as a temporal entity. There may be a parallel between crafting animation and the way we design, build, and experience cities. The effect of CGI, automated interpolated animation techniques, may be comparable with the effect of the computer-aided design buildings, or more contemporarily, with that of parametric designed ones. Buildings begin to resemble a sort of virtual reality. In fact, with the merging of the physical with the virtual, as buildings become immaterial, screens and as physical space is often substituted with the virtual space of our smartphones, the question about what is left behind, what is lost, and what is worth keeping become important. These questions became even more apparent as the pandemic restrictions forced a certain virtualization of multitude of everyday physical activities. The emptying of city streets and public spaces and the transferring of human communication and contact to online and virtual avenues offered a glimpse into a very possible future as a digitized image became ever more pervasive within both the public and the private spheres of our lives. Vision and the culture of ocular centrism was emerging triumphant once more. And this is, well, ironically, the pandemic began just when I had started exploring the haptic in art and specifically the haptic nature of my art. Just when I began looking into the tactility of handmade animation its feel and perception touch had become taboo to me this new reality did not vindicate the world the computer screen on the contrary it made the question of the need for the physical city even more urgent and i was stuck happily during the pandemic in Lisbon as I was about to do an exchange program with Dr. Pedro Serrazina. But it gave me ideas about making an animation, which I haven't started really, the physical environment and the bodies that are trapped and yearn to touch the city and celebrate the city. So I had different iterations of that. Yeah, so could handmade film animation, its making, its sensing, be a paradigm for how our bodies can still be fully present and engaged in the real city during this ongoing dematerialization of our world? This is not a question that tries to reinstate artisanal forms of film animation solely as subversive or resistant to mainstream practices. Besides, as Dr. Brigitte Hosea was here, in her contribution in a recently published book, The Crafty Animator, insightfully concludes, crafting descent can be done either manually or digitally. My interest lies in recognizing that despite the new possibilities that digital technology can offer to the animator, including tools that facilitate the very handmade techniques I favor, the total dominance of the digital is not fatalistically inevitable, but a choice, in fact one of many. Just as there are choices to be made with regard to how much we are willing to abandon the physical city for the apparent conveniences of Zoom equally. There are choices between the many hybrid configurations of manual and digital, the many imaginative ways with which we can allow the body to engage with the animated artifact with all its senses. I argue that beyond feelings of nostalgia and luddhism, of which artists using traditional animation techniques have been accused, there is something psychologically satisfying in both making and experiencing a film which has a significant handmade component. This argument rests upon some observations made by animation scholars, but can be more easily inferred by the wide range of writings on film, architecture, and other visual arts. The idea of the haptic cinema, the film that affects senses other than vision, has been introduced from various angles. Here are some first ideas of haptic cinema and their origins. The idea that images are films themselves that touch us can be dated as far back as Roman antiquity according to this passage found in Giuliana Bruno's surface. Giuliana Bruno herself indulges in descriptions of films or film installations that feel like fabric, like architecture and vice versa. It is Laura Marx, however, who established the term haptic visuality, as we have seen in a previous presentation, in her survey of non-commercial films that arguably found a place for the body and the non-visual senses, and by extension a place for the displaced filmmakers, either through the content of their image, or the emphasis on sound, or even experimentations with the celluloid itself. These examples may not pertain directly to the act of crafting each frame individually, but they do underline the importance of tactility, hapticity, and the body in the images that communicate place. Oops. Yeah. Beyond the moving image, the growing need for haptic experiences in the art and museum world is well described and documented in the Museum of the Senses by Constant Klaassen, Concordia professor. And the deduction was clear to me. Sorry. was clear to me. Sorry. And closer to home, Jennifer Barker sees the physicality of stop-motion animation almost as an extension of our own physicality. When animated objects are recognized for what they really are and not what they represent, they still communicate a certain familiarity and hence validate and value the realm of the body and that of physical space. Finally, Mark Patterson, through the research of art historian Rebecca Moholt on late Roman mosaics of bathhouses in Palestrae in North Africa, makes a case for the oculomotor perception triggered in such visual art forms that are meant to be traveled and walked upon. Again, even the illusion of a floor mosaic had to offer a tactile component that situated the body in space, its own space. For me, in my own work, the apparently inanimate city becomes animated, or better still, the anima of the city is revealed, the soul, by the textures and layers that mean to reveal the traces of its people and their lived experiences. It becomes animated by movement and narrative. I now know that whenever the animated objects themselves fail to conceal their true identity, this too sends meaningful information to the viewer about the act of making the image and produces sentiments of comfort and familiarity. For me, research through my practice is threefold. Researching the possibilities of handmade animation techniques as a means for sensing, absorbing, and expressing the built environment and researching ways of disseminating animation how others perceive it. For the moment it is primarily the process of sensing and making in which I venture, which I bring, and this brings me to my collaboration with anthropologist Dr. David Howes at Concordia, and the research project he directs, Exploration and Sensory Design, supported by the Social Sciences Humanities Research Council of Canada, an opportunity to explore specific space types and their sensoriality through my animation techniques. These include the urban park and the shopping mall for the time being, among others. And here are some preparatory photographs that I took as I was researching the idea of the park. Beyond pure visuality and certainly beyond any aim at photographic representation and common ideas of perfection. I'm seeking to link my sensory experience of place to my sensory experience of making animation. How is the transfer done? And in what way is handmade based techniques privileged vis-a-vis a computer screen?" That's the question. In the case of the urban park, thinking about bodily presence, movement, and activity, I thought of the act of the urban park, thinking about bodily presence, movement and activity, I thought of the act of drawing as the bridge between responding physically to a physical place and acting out this physical performance in the studio, transferring the drawing into drawn animation. The act of drawing is also an act of recording and an act of internalizing, acquiring a memory. I suspect it is what Johanni Palasma refers to as mimesis when he compares the practice of hand drawing to that of connecting points on a computer screen. He says, whereas the hand drawing is a mimetic molding of lines, shades, and tones, the computer drawing is a mediated construction. Mimesis, the imitative representation of the real world in art and literature, according to the Oxford Dictionary, may well be a way to embody my outdoor experience, internalize it, and freely reenact it in multiple iterations, from static watercolor drawings to hand drawings of various forms, lending themselves gradually to the moving image. It is a continuum between feeling, sensing, and making, visualizing, which I wish to experiment with and which I hope develops into a method about which I can talk more in the future. For now, just a bit of charcoal animation where there's the hand obviously and the bark and how these interact from a personal point of view. And a bit more. E um pouco mais. Sim. Então, isto representa alguns exercícios usando uma branca. Então, é uma experiência individual para o momento. No caso da sala de compras, não posso não pensar nela experiences. In the case of the shopping mall, I cannot help but think of it as the glossy and unlayered space that goes against all things urban. In an act of wishful thinking, I decided to contrast its artificiality and highly controlled environment with that of an imagined future when malls will fade into ruin. This is already happening, by the way. Again, technique and medium may be able to tell part of the story. Here, the present mall is a section drawn in AutoCAD with photographs processed in Photoshop. Once it disintegrates, the handmade silhouettes and drawings will reveal the city beyond its walls and how the ruins can become an integral and functional part of it. Hence, the exercise is not only that of sensing and perceiving, but also an urban design proposal. the aid of digital technology have not had the last word just yet, as they may be offering a lens through which architects and urban designers view and understand space in ways that pristine CAD drawings and animations do not. And that's just a small excerpt. I don't have sound, but it's okay. Through the cracks, I need to work on the cracks later. And just some final thoughts that I had the other day. In animation, there's always space to be covered. Division, vision, I believe, well, somebody told me there's an etymological connection, and that, of course, division implies distance, and distance implies desire. Movement to reach things. The temporal distance between frames frames or physical separation often, the temporal distance between the instances of their creation, the distance between the beginning and end of a frame's completion, the distance between the body and the object of the image, the distance between the technology of today with what is superseded, the distance between the experience, the conceived, the executed, the conceived, the executed, the process, the projected, the perceived. These distances beg to be covered. They had to be desired and walked, and it is this synthesis, this putting together, this traversing that offers meaning, understanding, and fulfillment to the artist, and consequently it translates into the awe of the spectator. Beyond the typical, how is it done, remark, there is the very significant realization, I can see and feel how this is done. I connect with the time it took, the space it occupied, the hands involved, the mood and temper of the human, the manipulator. The walking, traversing of these distances denote space and place. There is eros, and there is pathos, and there is topos, place. Animation is a perfect paradigm of what we had and what we may get in our lives and our spaces, in our ways of thinking and doing. Thank you. At the Burj Khal next to the part of the Museum of Fine Arts in Montreal, the group Astan and the Chag Neme story. Thank you for your great insight into the great research. It's a very interesting question, how can the aura of architecture of a city can be revealed with animation. You talked about the importance of the creation of the idea. Is this something that is important in your artist practice as well, when you think about the architecture of the city, how this is created? As an architect, you mean the design of the space? Yeah, I have to say also that I have my masters and my professional experience was in conservation of historic buildings. So a lot of it has to do with design, but also about appreciating certain things that are not seen by the general public. So it's also about revealing layers and making urban designers conscious, hopefully, on what there is and what there is to be valued, maybe seen through a different lens which incorporates this bodily aspect that maybe all these glossy posters of developers don't really communicate to people. Are there any questions from the audience? We are all tired already. Okay, Ari Stefani, thank you so much. You are around. If we have a little bit more energy, we will ask you some further questions. I would like to invite you on stage to recap the synesthetic syntax panel. Oops, we have to So unfortunately we've kind of run out of time. It would have been nice to have a big discussion with everybody and involve everybody in the audience but we of run out of time. It would have been nice to have a big discussion with everybody and involve everybody in the audience, but we have run out of time. So I was just going to quickly do a quick recap of some of the topics that we've covered today. I wish that we had a lot more time together to discuss these, but we had this morning performance in many senses. We had performance in terms of live improvisation and also efficiency of execution, yn ymddangos yn llawer o ffyrdd. Roedden ni'n cael perfformiad o ran hyfrydwriaeth byw, a hefyd effeithlonrwydd o gyflawni. Ac fe wnaeth hynny ddweud ymlaen yn ystod y bore heno hefyd. Roedden ni'n siarad am y profiad o ddefnyddio dechnoleg digidol fel un sy'n gallu ddatblygu ein hunain, mae'r ffyrdd yn dysgu ein hunain sut i'w ddefnyddio. A rydyn ni'n siarad am y bwyllgorau neu'r biasau sy'n cael eu adeiladu mewn aparatusau o gwaith, software a systemau sy'n cael eu cyflawni. about the automatisms and or biases built into apparatuses of hardware software and networked surveilled systems and this afternoon we um sort of carried on some of these themes and but what came to talk about the haptic so we were interested in the haptic in um in the sense of evoking memory evoking nostalgia absence and loss but also interesting the idea of pleasure and the oxytocin, the happiness hormone that you get through touch was very interesting with different devices. And also coming through the whole day was the use of touch in crafting, in thinking through making, and the knowledge that's held in the body rather than consciously accessed in the head or through drop-down menus, as Andy said this morning. But I think that in all of this discussion, it's not that we're being anti-digital. It's just that perhaps, as Yoichi suggested, we could think of new interfaces. Or Yanni suggested we could think of new ways to interface with the world of computing that might involve gesture, that might involve touch and new ways of involving the body in working with this material. So very sorry that we can't discuss this further but we have kind of run out of time haven't we? But it's a good reason to continue next year with the topic. And a short recap from my side. We started three days ago with a short journey in the past as we started ten years ago with the symposium and we tackled a lot of topics animation and a lot of expansions in terms of yeah senses but also in terms of new technology artificial intelligence was a topic yesterday all these intersections in terms of games, animation, playful environments. We had experts from the industry talking about big screens and new ways of animating characters in feature films. I cannot mention all the topics, but we have recorded everything and it's online at least right now, so we still have the possibility to listen to experts again and again as there is so much content discussed and presented in the last three days. Thank you so much. I will not mention the team again. But if you are interested in Ars Electronica and expanded animation, go to our website and see all the material that we provide for you. Thank you. So yes, so a big thank you to Jürgen. And the rest of the team behind the event, there's lots of people working on it. There's students. And as Jürgen said, everything is archived, and it's on YouTube, and it's freely available for people to watch again, and there's an archive of previous events. And also, as regards performance, there's a very interesting performance happening outside the back of this building right about now, if anyone wants to come. Thank you. Thank you to come. Thank you. Thank you very much. Thank you. Thank you. you you you you you