Since I can remember, I perceive music in colors and shapes - I have synesthesia. Synesthesia is a rare but healthy form of perception in which a sensory stimulus (e.g., hearing) triggers a sensation in another sense (e.g., seeing). Here, a long-time chamber music partner and friend who has commissioned a painting from me is asking me questions about my perception and how I translate it into acrylic paintings.
Johannes Brahms - Rhapsodie Nr.2, Op.79, (played by Martha Argerich, Deutsche Grammophon 1960)
Can you explain what we see in this picture, or is it self-explanatory?
That's a great question! Each synesthete has an individual perception, which means for someone else, the same piece could have completely different colors and shapes or even taste like something.
As you can see, the picture is divided into two parts. Since Brahms' Rhapsody No. 2 has strong mood changes, it made sense to represent it that way. Often, my paintings also have multiple overlapping layers, as this one does. In general, I can't represent my perception temporally. Therefore, the paintings are always a mix of an overall impression and individual, characteristic aspects of the music. The colors are usually determined by pitch and harmony and are given in that sense. Shapes, relationships, layout, texture, and color gradients are used differently to artistically represent the whole. This can be the rhythm, instrumentation, timbre, playing style, and so on.
What do we see in the lower part now?
In this case, the lower part is characterized by shapes because it represents the recurring rhythmic figure as follows:
Mostly, deep structures are naturally located at the lower field of vision, so I portray them in the same way in the painting. However, if it doesn't fit aesthetically in the visual context, I take the liberty to adjust the composition.
If I were to paint each note one by one, it would be overly detailed, resulting in many individual colors. In this section of the musical piece, the impression of the figures is created by the dominant pitch and harmony colors. So, there's a lot of reddish-brown (e.g., deep G), dark brown, red, burgundy, and purple due to the harmonic blending with the other notes. The overtones and the timbre of the instrument also play a significant role in this perception.
What do we see in the upper half?
The upper part corresponds to the legato triple figures. In such a passage, I always have to decide whether to paint them as a sound carpet or in finer structures. Because I've already represented the lower part in a more figurative way, I decided to depict the upper area in a more solid manner. You can imagine this as zooming in or out of a photo, or looking at the forest or individual trees.
In this context, the green-blue-yellow area primarily represents this part:
And the red-yellow-orange area represents this part:
So, it's very clear to see the harmonic difference, although rhythmically, it's relatively similar.
And what are those white veils?
In this case, the white veils represent one of those additional layers I mentioned earlier. Several sections in the Rhapsody have a lot of sharps in the key signature. Sharps add a white or grayish element to the written note, making the pitch somewhat veiled. Key signatures with many sharps often result in a veiled or really patterned appearance. This, in turn, depends a lot on the rhythm, playing style, and the instrument. So, these two parts here add those white veils to the overall impression.
It's clear how many possibilities there are to represent the entire musical piece and its temporal progression within the constraints of a 2D image. You could have placed these sections alongside the others as well. Initially, I didn't have them in the picture, but I felt something was missing, and I like it better this way now. It adds more depth to the representation.
I assume you can't influence the structure of your synesthesia. Or does it maybe depend on your daily condition?
Oh yes, it does depend on your daily condition, and in my case as a woman, it also varies with my hormonal cycle. As a neuroscientist with extensive research on music perception in the brain, I am convinced that it cyclically affects the quality (not the quality) of musical hearing in women with a natural, non-hormonally altered cycle.
[At this point, I must note that it cannot be inferred from this that women hear worse than men due to cyclical variations. Physiological functions and performance, including hearing, also fluctuate in men but tend to follow a daily or weekly rhythm rather than a monthly one.]
On some days, everything appears rather dull and veiled, and I can hardly see the colors or perceive them clearly. On other days, it's almost glaring, and sometimes just beautifully radiant. On these days, every single note is like a rainbow, and it's as if I see every individual overtone in the spectrum. Coffee and alcohol can have similar effects, as well as adrenaline on stage. Although coffee and adrenaline both stimulate, coffee has a somewhat damping effect on me, while adrenaline and alcohol (in moderation) intensify the colors.
I would be interested to know... If the bassoon and cello were to play the same (solo) piece in the same range, what would change in your perspective? And do you naturally prefer the cello colors in that case?
No, it's not entirely the same, of course. It would be as if you couldn't tell which instrument was playing. The colors are indeed somewhat similar due to the pitch. However, due to the instrument-specific formants and resonances, some notes might even have completely different colors. But most of them will be very similar, differing only in brightness, texture, warmth, and saturation. It's similar to how colors can vary in sound design on the same instrument depending on the playing technique (and the musician). You can think of it like an Instagram filter for photos.
And what about when two cellists interpret the same piece?
That's very interesting! In principle, the colors and shapes are already inherent in the music, but the results can vary significantly. Particularly, variations in timbre manipulation can add more or fewer overtones and completely alter the texture and structure of the colors. The same note might appear as a bright red in one version, a hazy red in another, and have a yellow core in a third, or perhaps fringed with purple at the top. Many differences in the overall impression are, of course, also influenced by the tempo of the performance.
Since when do you know that you have synesthesia?
Synesthesia is indeed an innate trait. I remember that even in kindergarten, while singing in a group, I perceived the notes in colors. When I was 16, my piano teacher stumbled upon a chapter about synesthesia in the book "Musicophilia" by neurologist Oliver Sacks. She had always been puzzled when I mentioned blue or green tones. At the beginning of my piano lessons, I had trouble associating the piano keys with the same notes in the bass and treble clefs because the same note written in a different clef initially had the same color on paper, but it represented a different pitch due to the clef. I had to learn to "see" the new colors (of the bass clef) in the lower staff. I always explained it to her using colors, and she had simply thought of it as an association until she read the book.
And do you have other forms of synesthesia? And how do you perceive the colors?
Yes, like most individuals with synesthesia, I also have grapheme-color synesthesia, meaning I see written letters and numbers in color. I also experience this for written musical notes (hence the issues with different clefs, which I still have today, but don't tell my teachers ;-) ). The piano keys are also colored. Additionally, I perceive body sensations, pain, movements, and positions (such as the hand and finger positions on the cello) partially in color. Shapes often have colors too; for example, cutlery (knives are red, forks are green...). Numbers, weekdays, and months also appear to me as spatial structures, somewhat like ladders.
In synesthesia, there's a distinction between "associators" and "projectors." Associators predominantly experience the secondary sensory perception in their minds, as if on an internal screen. Projectors "see" it in the external space, or they project it onto objects or the body. I am a projector, which means the colors are in the space for me, like on a second level, on the sheet music, on the instrument, objects, and with sounds actually in the space. Depending on the volume, sounds either expand more within the field of vision or come from the direction from which the sound originates.
Is it more of a nuisance or a feature?
By and large, it's a feature. It has often helped me remember things better, play many piano notes simultaneously from sheet music, or replicate body movements. In lessons, I often observe my teachers closely and think something like, "Ah, the hand is making a yellow movement," "the finger is in a somewhat more blue position," or "a bit more turquoise at the end of the sound." I can also hear overtones well because of the colors; it's like painting a picture when working on timbre.
However, when multiple colors come together, or the colors have different sources, it can quickly become confusing and lead to various errors. For instance, I tend to listen very color-oriented, meaning I often see the colors of overtones rather than the fundamental tones. This leads to confusing similar intervals, such as fifths, octaves, and fourths. Additionally, the written color of the note is often not correct on an instrument like the cello, where the pitch of the notes may vary depending on the context. It's the same core color, for example, yellow, but in a different brightness or saturation. The colors also change in a different tuning or when transposed to another octave. When I have to play the yellow note with a red finger and there's a light green "2" above the note, then the confusion is complete ;-).
You can think of it like seeing the word "red" written in blue ink and having to name the ink color, not read the word. When multiple people are talking at once, I'm reading something, or I need to concentrate, it can quickly become sensory overload for me.
And can one also turn it off if they want to?
I often unconsciously suppress the colors a bit, similar to how one tunes out background noise in a large office space. Otherwise, it would be unbearable. However, this also has the disadvantage that I sometimes do it automatically while playing the cello, especially when I'm focusing on observing my teachers or fellow musicians, or reading the printed music. Actively turning the colors on and off consciously isn't so simple. But I recently made a new observation on how this might work, along with a neuropsychological theory. I believe it has to do with the activation of the left or right hemisphere of the brain. Sometimes we tend to use one hemisphere more than the other, with many people having a dominant hemisphere. But that's a bit beyond the scope of this conversation. However, it's true that synesthesia operates very unconsciously for me. This means that even in my case, if I try to force it, I don't see the color (for example, if someone asks me to name an interval). I can only do it somewhat unconsciously when I'm not actively focusing on it or when I'm doing something else simultaneously.
Is there a correlation between your musical taste and color preference, meaning, do you like pieces in certain colors better than others because of those colors?
No, there are pieces that I find musically excellent, but I wouldn't like to paint them because the colors don't harmonize well. However, there are certain keys that I naturally prefer over others for this reason.
Sources and Reading about Synästhesia and Perception
Sacks, O. (2010). Musicophilia: Tales of music and the brain. Vintage Canada.
Jede Wahrnehmung ist anders (Interview im SRF mit Prof. Lutz Jäncke): https://www.srf.ch/wissen/mensch/hirnforschung-warum-jede-person-die-welt-anders-sieht
360 Grad Simulation by the synesthete, software engineer, neuroscientist and musician Kaitlyn Hova:
コメント