How do modern day science understand and reconstruct shikohin?
In this series, “Understanding Shikohin and Its Experiences Through Science,” we will talk to scientists of various fields such as cognitive science, brain science, psychology, etc.
………
Common belief tells us that our everyday experiences and life itself is shaped through our physical bodies.
However, this belief is about to drastically change.
For this article, we visited Professor Kouta Minamizawa of Keio University Graduate School of Media Design (KMD).
Professor Minamizawa is a researcher of human augmentation, or interactive digital technologies which creates partial or whole extensions of the human body. It is said that this technology has the potential to expand human experiences, thoughts, and even abilities and may add new value to our lives.
Through an alternate body, we may even discover new shikohin experiences.
Shikohin
Shikohin is a food or beverage that is not nutritionally necessary for the body but gives pleasure to one’s senses such as taste, touch, smell and sight.
Examples include tea, coffee, tobacco, liquor, pickles, soft drinks and ice. They often contain organic acids, caffeine, tannic acid, cocaine, alcohol, bitter substances, or essential oils. In a broad definition, it may also include sweets and confectioneries.
Encyclopedia Britannica Academic Japan
Designing sense of touch and reconstructing boundaries of the human body
―― What exactly is the research of human augmentation technology?
Human augmentation research is about expanding the possibilities in experiences and movements of the human body. My research team is primarily exploring qualitative augmentation, also referred to as enrichment.
When one hears the term human augmentation, most of us think of quantitative augmentation, also referred to as enlargement.
An example of quantitative augmentation is the development of technology that assists our bodies to achieve certain things, such as to run faster or carry heavier objects. Many such devices are used today thanks to recent advancements.
However, lately more and more researchers are exploring not only quantitative, but also qualitative augmentation. In other words, it is the study of how much more we can enrich our senses and experiences through human augmentation.
Humans obtain various sensory information, such as sight and sound, through their physical body. By connecting these sensory information with digital technology, we can record, share, expand and create new sensory information.
Among the various sensory information we use, I have been studying how to connect our sense of touch with digital technology. It is my belief that the sense of touch is a very important aspect in qualitative augmentation.
―― Why do you think that touch is important for qualitative augmentation?
Although we are not so aware of it, the sense of touch through our skin is what makes us aware of the boundaries between our inner and the outer world.
By designing boundaries through digital technology, such as making a boundary much clearer, it allows people to have a stronger perception of their inner world.
Slime Hand, is a sensory illusion created by Nagoya City University’s Kenrikodaka Laboratory. When one watches the slime being pulled, the participants have a sensation that their finger, which is placed on the other side of a barrier, is being pulled in the same way.
On the other hand, if a barrier becomes very vague, it is possible to share sensations with others and people get a sense that they are becoming one with their surroundings.
We call designing the sense of touch with digital technology haptic design.
ーー What are some examples of haptic design?
One example is FEEL TECH™, a technology that shares the sense of touch through a vibrating device, which KMD is developing in collaboration with NTT DOCOMO and Nagoya Institute of Technology.
As you know, for auditory information we use microphones to pick up sounds and reproduce them through a speaker to share with others. In the same way, this device records tactile information, such as the sensations of a person playing an instrument or touching something, and reproduces those sensations through vibration.
Currently, we are working on a device that allows people to feel the delicate sensations of activities, such as playing the Koto (Japanese harp) or preparing matcha tea, through connecting sensations to video content.
We also have programs where people can experience the world of tactile sensations of visually impaired people.
―― I would think that the sense of touch is something that is easy for us to recognize as an extension of our own actions. It is interesting that we will be able to experience the sensations of touch of other individuals or experts.
Actually, I am studying whether or not we can rethink our understanding of the sense of touch as something that can possibly change our relationship with others.
Touch is usually understood as a way of understanding the texture of something. However, in reality we use our sense of touch to feel our own existence and to build complex relationships of emotion and sentiment.
By using a special kind of rubber material called e-Rubber, the pulse of the person on the right can be felt by the person holding the green balloon on the left. This device was developed by KMD in collaboration with Toyoda Gosei Co., Ltd.
―― How do you think technologies such as FEEL TECH™ will change our daily lives?
We currently use smartphones to share various experiences with one another. However, I believe our current lifestyle in which we only use smartphones will begin to change.
For example, there are newly developed ring-like vibration devices that can be worn at all times and I believe more people will begin using such devices. I believe FEEL TECH™ will become one of such “post-smartphone” devices.
If VR allowed people to have wings like a bird
―― How will human augmentation that includes the sense of touch change our everyday lives?
Experiences that we have through human augmentation influences not only our physical bodies but changes the way we think, even if it is in VR.
Our bodies that are expanded through human augmentation are “another body” that has the potential to change us.
There are already multiple reports that have come out of VR research regarding this.
For example, if you are in the VR world and go up to the top of a high building and look down you still experience fear. It is the same feeling that one would have when they stand on a glass floor in skyscraper buildings at tourist destinations. Even if your mind is aware that you are in the VR world, our bodies react and we experience fear in the same way that we would in the physical world.
Our body in the VR world is “another body” that influences our senses in the same way as our physical bodies do. This is how we augment our bodies into the VR world.
ーーHuman augmentation that expands beyond our physical body is hard to imagine…
As humans we do not normally have wings, but in the VR world it is possible to obtain wings. In other words, it is possible to turn our bodies into that of a bird in the VR world.
―― For humans, who have never experienced being a bird before, is it possible to fly in high places like a bird without feeling fear?
Turning into a creature that can fly like a bird or a dragon gives us the perception that we have the ability to fly. It has been found that this perception mysteriously eases our fear of high places.
―― So when we perceive the world through the eyes of a bird in the VR world, our fear of heights is alleviated.
There is another example in which VR was used as a rehabilitation tool for male perpetrators of domestic violence (DV).
In the experiment within VR, the male perpetrator of DV becomes the female DV victim. In other words, they become a female avatar and experience what DV is like from the victim’s perspective. Through this experience, the perpetrator is allowed to subjectively perceive the viewpoint of the victim and it has been shown to suppress future violent behavior.
In this way, the experience of becoming a “different being from oneself” through an avatar in VR has shown to expand our perceptions of “self.”
Perceptions through a “new self” changes your “real self”
―― How are human augmentation technologies that use the sense of touch developed?
We connect ourselves to our actions through satisfying both body ownership and sense of agency. This is not limited to the sense of touch.
Body ownership refers to the feeling that our body is our own. Sense of agency is the feeling that we are taking actions on our own.
When our sense of sight and sense of touch is in sync, it creates feelings of body ownership and sense of agency. In other words, when we see that our body is moving according to our will, our senses confirm that we are taking those intended actions.
For example, if we see our body bump into something, we feel the impact through our skin. Or if we reach out and grab something, we feel that object in our hand.
By intentionally cutting off the synchronization of sight and touch and replacing it with something else, we are able to create a special sensation.
―― Are you referring to illusions?
An illusion is the trigger. However, the key is that there is an intentionally generated perception of self that is maintained by body ownership and sense of agency.
When a new perception is generated it will feel like it is your own body or new self, whether it is actually your body or self or not.
When you take the experiences you have through these other bodies or other selves and make them your own, you can change yourself in the real world as well. In other words, it becomes possible to treat oneself more like a concept (with no reality).
Seeing the state of your mental health through aura
―― What kind of shikohin experiences do you think will be made possible with human augmentation?
If we define the shikohin experience as “something that changes our ordinary selves through extraordinary experiences” we are in fact already researching such experiences and moving toward making them a reality.
For example, there is the research project called Transdential Avatar (*3) that studies a new type of meditation experience by letting us see our emotions in a subjective way in VR.
A person puts on a head mounted display device that allows you to see yourself through external camera footage. Around your body, you can see your aura.
This aura is the visualization of your mental state, measured by how much your hands are sweating and your heartbeat.
It is difficult to view yourself objectively in your day to day life so seeing a visualized version of yourself based on physiological data may allow you to understand yourself in a different way.
We also work in collaboration with a company called Enhance to create a VR experience that uses only tactile and auditory sensations, without visual sensations, to evoke some kind of mental landscape.
Enhance developed a device called Synasthesia X1-2.44, a reclining chair with 44 vibration points. The user sits in the chair and closes their eyes. They are then immersed in a world of tactile sensations and sounds through vibrations sent to the shoulders, back, feet and all around the body, along with sounds such as footsteps and other background noises.
During this VR experience the participant goes through various psychological experiences. Some participants go on a journey to their past and others experience something out of the ordinary appear in what would be their ordinary life.
ーー So each person experiences something different?
The most interesting thing is that these VR experiences actually result in a change in the participants’ real everyday life.
VR is different from reality in the physical world, but it has the same influential effects of reality. It is an alternate reality. Even though it is an alternate reality, it causes real change in a person’s way of thinking and how they live. Because of this, I believe it can be said that it is a shikohin experience.
※1 https://www.jstage.jst.go.jp/article/tvrsj/25//25_2/_article/-char/ja/
※2 https://www.nature.com/articles/s41598-018-19987-7
※3 https://dl.acm.org/doi/abs/10.1145/3550082.3564210
Photo: kaori Nishida
Translation: Sophia Swanson
Akihico Mori writes for WIRED Japanese Edition, MIT Technology Review, and other academic publications. He holds an M.A. in Media and Communication from the University of the Arts London.
Editor and creator of the future through words. Former associate editor of Huffington Post Japan. Became independent after working for a publishing company and overseas news media. Assists in communications for corporates and various projects. Born in Gifu, loves cats.