One of the many ways that our brains can astonish us is synesthesia, a neurological phenomenon that is well described as “a condition in which ordinary activities trigger extraordinary experiences.” I can give a concrete example from my own childhood, when without knowing it, I was a synesthete; that is, my brain automatically generated extraordinary cross-sensory experiences with no effort from me. Whenever I saw or heard the name of any day of the week, I immediately saw in my mind’s eye a color, always the same one: Sunday was always deep maroon, Friday green, Saturday a pattern of gray and silver circles, and so on.
A synesthetic interaction can happen between any pair of the five senses. Besides my type, there are synesthetes for whom sound produces colors, words generate tastes or touch induces smells, and other combinations as well. These strange linkages mean that synesthesia was once misdiagnosed as a delusion, mental illness, or a metaphor stretched too far. It is none of these, but as research has shown, a real, typically stable mental state found in about 4 percent of the population. For me it seemed perfectly normal.
Synesthesia runs in families and researchers are tracing its complicated genetic markers. Since the 19th century, it has been associated with creativity and artistic ability. Recent testing confirms that it is more prevalent in people who do creative work. Vladimir Nabokov and Richard Feynman respectively saw colors in letters of the alphabet, and in algebraic symbols; David Hockney and Lady Gaga both turn music into colors. But although synesthesia is now better understood, its cause is still incompletely known. It remains semi-mysterious, and has always been hard to describe to non-synesthetes. Three LaboCine films illustrate what synesthesia is, how it feels, and the surprising connections it generates.
The brief story COLORCONDITION lays out the basics of synesthesia and its human effects. We meet a bearded, casually dressed young man who you might guess is an artist. He speaks to us in a quiet, subdued way: “Some people say what I have is a superpower. Some people think its strange…I just see things differently. I’m not some kind of monster…I didn’t know I had it until I was like 20.” The “it” is one form of synesthesia, which he presents by first explaining that the human brain has evolved to interpret certain wavelengths of light as colors. “My brain,” he goes on, “also interprets sound as colors. My wires got crossed…my brain [makes] more connections,” and adds “Some people compare it to tripping. I guess I’m just tripping all the time.”
Colors are important to this young man, who is indeed a painter, but “tripping all the time” affects his whole life. We see one such moment when he both hears a wailing police siren and sees it as “a thousand stars being born in front of your eyes and then dying seconds later.” Yet as he paints on a canvas and the woman in his life hugs him, he says “persons are the most colorful and sounds are the most colorful;” perhaps it all somehow comes together in his work and his relationship. The film ends with a beautiful visual summary, a changing pattern of pastel colors with another colored image like a sound spectrum overlaid on it.
The documentary An Eyeful of Sound conveys more about the neuroscience of synesthesia, while its colorful animations and voiceovers from synesthetes give an experiential sense of the phenomenon. The film begins with several voices uttering short phrases that appear on a shimmering blue plane of color: “It starts like that…it’s almost like a…there’s more than one color there…,” and others. Then a woman says “I like muted sounds… softer shoes …people walking…I feel it in my mouth and I taste it.” Now we understand that we are hearing three women find words to describe their synesthetic experiences.
Their continuing descriptions, and mantras like “all sounds have color...the alphabet has color” are repeated, juxtaposed and overlapped to give a sense of varied synesthetic interactions. The accompanying music, and animations that are abstract or illustrate synesthetic effects such as dogs barking in color, enhance the effect. Some comments show how a synesthete begins to grasp the confusing nature of a mixed response. “I was made much more aware of this,” one woman relates, “when I heard an orchestra playing…I thought there was some sort of colored quilt. I couldn’t quite make out what was going on.” Then attention shifts to psychologist Jamie Ward, a leading synesthesia researcher at the University of Sussex, UK. His discussion of the science of the effect is interwoven with evocative animations and parallel voiceovers from the women.
In the last minutes of the film, the women explore their experiences among themselves, echoed in imaginative animations of their internal reactions. This builds to a kind of final crescendo of forms and colors in motion. Short of entering the mind of a synesthete, this may be as close as a non-synesthete can get to experiencing the effect.
Despite its name, the surrealistic music video Synesthesia isn’t about that phenomenon in the same way as the other two films but refers to its sensory experiences and their connections. The film begins with silent overhead shots of fantastic arrays of food sensually arranged in kitchen drawers and heaped on a dining table -- sausages, fruits and vegetables, whole fish. As the mother of the family cooks by putting printed words from recipes into the oven, the father, seated in front of a massive stereo wall, eats from a tray that includes a presumably edible book.
Meanwhile a son plugs his headphones into various foods, which turns on the soundtrack and starts a series of events: balls, vegetables, frogs, cats, vines and colored shapes emerge from the stereo speakers while they play hard driving music. As these things pile up on the floor, sirens are heard and the speakers emit smoke, and sparks that set the drapes on fire. But the father keeps calmly eating, the mother serves a turkey made of newsprint, and the son plugs his headphones into his navel, then disappears in a puff of smoke as the film ends. Without directly presenting synesthesia, the film’s rapid-fire allusions – the sense of taste, as represented by food, turning into sound; the tactility of connecting headphones generating both music and colored objects; words becoming food – resemble synesthetic experiences. Besides, Synesthesia is pure fun.
The other two films show the human reactions to synesthesia. Sound-color synesthesia may enhance the artistry of the young man in COLORCONDITION, but he has doubts about its overall impact. The participants in An Eyeful of Sound often find their unusual internal links wondrous, but also find moments when it would be simpler if these were turned off. My own synesthesia vanished at a young age, but in retrospect it seems purely enriching. Maybe if I had lived with it or another type for many years, I too would find my reactions to it complex, like synesthesia itself.
About the author
Sidney Perkowitz, Candler Professor of Physics Emeritus at Emory University, writes often about science in film. His latest books are Physics A Very Short Introduction and Real Scientists Don’t Wear Ties.
