How do modern day science understand and reconstruct shikohin?
In this series, “Understanding Shikohin and Its Experiences Through Science,” we will talk to scientists of various fields such as cognitive science, brain science, psychology, etc.
………
In Part 1 of this article we discussed the possibility of various experiences through human augmentation in VR, such as overcoming fear of heights by becoming a bird avatar, and subjective meditation through seeing your own aura.
How will physical avatars influence our relationships in society and expand on our view of life?
In Part 2 of this article we will continue to explore these questions with Professor Kouta Minamizawa of Keio University Graduate School of Media Design (KMD) who explores the study of human augmentation.
Shikohin
Shikohin is a food or beverage that is not nutritionally necessary for the body but gives pleasure to one’s senses such as taste, touch, smell and sight.
Examples include tea, coffee, tobacco, liquor, pickles, soft drinks and ice. They often contain organic acids, caffeine, tannic acid, cocaine, alcohol, bitter substances, or essential oils. In a broad definition, it may also include sweets and confectioneries.
Encyclopedia Britannica Academic Japan
The Avatar Robot Cafe where people with disabilities work remotely
―― In Part 1 we talked about your work with avatars within VR, but you also research physical avatars in the real world.
So far I have introduced various experiences in VR, but in our ongoing Cybernetic Being Project we are also conducting research on physical avatars, where robots are handled as a part of your physical body.
In this project Cybernetic Avatars are used to improve the wellbeing of more diverse populations, such as those with physical disabilities and the social issues they face.
We define human augmentation that expands the physical body’s potential through robots and VR as Cybernetic Avatars. The new way we view the human body and daily activities that are made possible through these technologies are called Cybernetic Being. We are studying this through various angles including technology, the human brain, psychology, social systems, and ethics.
A more specific example of our Cybernetic Being project is the avatar robot named OriHime, a joint research project with OryLaboratory and Kentaro Yoshifuji, who operates the Avatar Robot Cafe DAWN 2021.
ーー That is a cafe in Nihonbashi, Tokyo. How is it operated exactly?
A pilot that works at the Avatar Robot Cafe may be a person who is physically immobile due to SMA (spinal muscular atrophy). Such persons control the OriHime robot using only their eyes through a device that picks up eye movement.
The biggest social value this project realizes is that the Avatar Robot allows people to connect and work in society even if they are physically unable to move. However, there are still limitations to the movement that one individual can achieve, so actions such as cooking food are difficult to achieve.
For such complex actions, if another OriHime pilot can log into the same robot at the same time, multiple people can work together to achieve a task. We are conducting our experiments in order to realize such a future.
ーー Two people controlling the same OriHime robot to work in the service industry?
At the Avatar Robot Cafes, we have seen multiple examples of how various pilots develop their skills in order to achieve various tasks and merge them with the avatars.
Drawing on these findings on how to merge human skills with robots, we are researching how diversity can be utilized and how technology can expand and exceed an individual’s capabilities.
―― What characterizes the Cybernetic Avatar and how is it different from VR avatars?
The biggest characteristic is that it allows the user to interact with people around them while expanding beyond the inherent skills of the individual.
For example, one experiment we held in collaboration with Avatar Robot Cafe was the Multiple Avatar Robot Experiment (*4). In this experiment a single pilot operates up to five OriHime robots.
This allows the pilot to speak to the customers from the beginning, using different robots to wave the customer toward a seat, take their orders and deliver their food.
To achieve this the pilot simply looks at various video images from multiple OriHime robots to choose which one to operate. From the customer’s perspective, they get to experience their time at the cafe through the guidance of one pilot moving through various bodies and they get the sense that they are being waited on by multiple OriHime robots at the same time.
In this way, digital avatars make interactions that would not be possible with only one human body possible. The relationships that are formed in these interactions are the biggest characteristic of Cybernetic Avatars.
Is it possible to create a concert that excites everyone?
―― What other research projects are you working on?
One signature research subject of the Cybernetic Being Project is researching how to share experiences.
Have you ever been to a concert and experienced a rush of excitement in your body?
―― The thrilling feeling we get when we are excited?
Yes, like getting goosebumps. In English, goosebumps is also known as Frisson and we are researching how we can generate this physical response in a group in order to create a shared experience.
Frisson Waves (*5) is a research project led by a master’s program student at KMD and she developed a Frisson Generating Neckband to deliberately give people the frisson response.
So how do you deliberately generate the frisson response? In fact, it is very simple and all you have to do is give the person a chill.
The Frisson Generating Neckband contains a module that stimulates the skin by making it feel cold. If you have all the people attending a concert put on this neckband and generate a frisson response during a highlight of a concert, they will all feel the physical thrill at the same time.
Although it is simply that they are getting goosebumps from the chill on their skin, because this physical response happens in the contexts of being in a concert, the theory is that the audience will get the illusion that the goosebumps came from an emotional rush and this will create a shared experience of a heightened thrill among all of them.
The future of this research leads to the development of generating emotional responses through online experiences and content.
It is rare to get goosebumps by watching a live concert on your screen, and most people want to experience live concerts in person.
However, technology such as Frisson Waves may provide a way for us to have more realistic and physical experiences through our screens.
We are also working on other research projects, such as using robots to allow people to experience the sensations of the super refined skills of professionals.
The ultimate shikohin experience in the model of Umwelt
――What kind of experiences do you think people will have once the technologies being developed at the Cybernetic Being Project gets implemented in the real world?
I think that experiences that are generated through avatars or shared via human augmentation will become integrated in our lives and become a part of who we are.
Cybernetic Avatars will allow diverse possibilities that were not possible through our prior understanding of the physical being. As our understanding changes, I believe that the possibilities in life itself will expand and grow for each individual.
Changes in our fundamental understandings can bring about shock and surprise, but if you look back on human history we can say that it has been a continuation of cognitive transformations.
Even in our current world, smartphones make it possible to meet up with an unfamiliar person in an unfamiliar town, without prior research. This is something that was not possible for people to experience in the past.
―― As our understanding changes, what kind of new shikohin do you think will emerge?
Defining shikohin is complex in itself, but one thing that can be said is that it is something that is not necessary to sustain life.
In other words, it is something that exists to expand your world.
By this definition, shikohin is an experience that expands your understanding of yourself.
These experiences allow you to gain new perspectives and values that you were not aware of before. As researchers, we discover new perspectives and values as we develop new technology and it changes even our perception of what it means to be human.
One ultimate form of this change in cognition is about discovering a new model of Umwelt.
ーー What is the model of Umwelt?
The model of Umwelt was developed by German biologist Jacob Von Uexküll. We used to understand our environment, or our world, as something that has existed objectively or independently.
However, the model of Umwelt defines the environment as something that exists subjectively according to the entities in it.
For example, the world through the eyes of a bee is different from the world through the eyes of a human, even though we share the same world.
In a world of digital transformation (DX), where the expansion and fusion of skills and the sharing of experiences will be realized through digital technology, we will come to see and create things in a new world that is different from the physical world that we have always lived in, which will be based on a new understanding of our physical body.
Digital technology such as VR and avatars will expand our understanding of the model of Umwelt and I believe it will lead to the ultimate shikohin experience one step at a time.
※4 https://dl.acm.org/doi/abs/10.1145/3544548.3581124
https://www.youtube.com/watch?v=e5qZNkiMYOg
※5 https://dl.acm.org/doi/abs/10.1145/3550324
Follow us! → Instagram / Twitter
Photo: Kaori Nishikawa
Translation: Sophia Swanson
Akihico Mori writes for WIRED Japanese Edition, MIT Technology Review, and other academic publications. He holds an M.A. in Media and Communication from the University of the Arts London.
Editor and creator of the future through words. Former associate editor of Huffington Post Japan. Became independent after working for a publishing company and overseas news media. Assists in communications for corporates and various projects. Born in Gifu, loves cats.