Summary: Every day, our brain combines information from multiple senses, such as images and sounds. Through this intricate process, we not only perceive the external world but also create narratives and form new memories. In our project, The Hidden Story, we employ the mediums of film and sound design to deconstruct a popular story into its fundamental sensory components and reconstruct it through distinct auditory and visual channels. As a collective audience, we will experience this narrative through silent moving images, evocative sounds crafted by musicians and foley recordists, and through videos capturing their physical effort in crafting the sounds. Will our brain harness sensory information to rebuild the original (hidden, deconstructed) narrative or will it generate its own new narrative?

Project idea and stages: Our project aims to use the well-known story 'The Emperor's New Clothes' by Hans Christian Anderson, which is the story of an emperor who was so fond of his appearance that he spent all his money on clothes that were made from a wonderful but invisible cloth. We recreated a visual-only version of this story, by leveraging our previous neuroscience findings on how the brain processes sensory information to interpret and create stories, which showed that specific acoustic features (such as louder volume and faster tempo) evoke rich visual scenes in the mind of the listener (e.g., a tall villain, a fast and accelerating train; the complete paper can be found here: https://osf.io/preprints/psyarxiv/aj97z). We scripted out our 'visual-only' scenes in a way that would reflect these research gatherings, thereby changing the syntax in which films are created. We are now in the process of creating the installation components dedicated to sounds and sound making.

Installation: Our installation will incorporate the following audio-video components: One screen will show the body work that the foley recordists produced for the film. Another screen will present the work of the film-music composers put in to enhance the narrative of the emperor's narrative. An accompanying soundtrack created by the above participants (foley recordists and film-composers) will be edited and positioned for the installation. The 'visual-only' scenes of the Emperor's New Clothes will be presented on a CRT monitor for reference or interference. The experience overall may trigger our memory of this classic story or may end-up providing for a new narrative. This layering of the visual story, the visual scene of the sound making process, and the sound itself will be complementary-at times synchronous and at times asynchronous- to simulate the exercise that our brain performs when combining information from multiple sensory inputs to understand old stories and create new stories.