Ivy Tang

Project Statement

When I listen to Chopin, I see a meadow at dusk. When I listen to Sibelius, I see storm clouds rolling over rock. These images arrive unbidden — not as decoration, but as something the music seems to contain.

This is a familiar human experience. Across cultures and centuries, listeners have described music in terms of weather, terrain, light, and atmosphere. Psychologists refer to this as cross-modal perception, or in stronger forms, synesthesia. I think of it more simply: music has always taken me to places.

Listening to Landscapes began with a question: could a machine form environmental imaginations from music in a similar way? Rather than translating sound literally into images, the project explores how musical qualities — tempo, texture, dynamics, brightness, and emotional intensity — might give rise to atmospheres, terrains, and spatial feelings that feel perceptually coherent.

The goal is not to produce a single “correct” image for a piece of music, but to investigate how sound can become environment: how a nocturne may gravitate toward enclosed twilight spaces, or how orchestral force may unfold into turbulent skies and monumental terrain. The resulting landscapes are therefore not illustrations of music, but atmospheric interpretations emerging from relationships between sound, perception, and environmental imagination.

Three Pieces, Three Landscapes

The following studies draw from three distinct musical worlds — Chopin’s intimate nocturne, Mahler’s restrained orchestral intensity, and Sibelius’s storm-like symphonic force. Each piece was interpreted through the same perceptual framework, allowing differences in texture, dynamics, emotional scale, and atmosphere to shape the landscapes that emerged.

Inspired by Frédéric Chopin’s Nocturne Op. 9 No. 3 in B major, this landscape emerged from music that feels intimate, inward, and suspended in time. The piano line unfolds slowly in the upper register while the accompaniment remains soft and restrained beneath it. Rather than translating the piece literally, the system gravitated toward a quieter environmental atmosphere: dim dusk light, enclosed space, still air, and traces of mist. The resulting meadow-like landscape feels withdrawn and private, echoing the nocturne’s fragile emotional interiority rather than illustrating it directly.

This landscape draws from Gustav Mahler’s Symphony No. 5, Adagietto, a work of profound emotional intensity carried almost entirely by strings. Compared to the Chopin study, the orchestral texture here is denser and more expansive, though the tempo remains restrained. The generated environment settles into an uneasy balance: green terrain beneath gathering clouds, suspended between calm and approaching turbulence. What emerges is not a dramatic storm, but a landscape charged with emotional pressure held just beneath the surface — much like Mahler’s music itself.

Generated from Jean Sibelius’s Finlandia, Op. 26, this study responds to one of the most forceful pieces in the series: surging brass, rolling percussion, and extreme dynamic contrast. The system moved toward a harsher and more monumental environmental imagination — rugged terrain, fractured light, turbulent skies. The resulting image, dominated by storm clouds and exposed rock, carries a sense of scale and momentum that mirrors the music’s nationalist intensity. Unlike the quieter ambiguity of the previous landscapes, this one feels openly confrontational, as though the environment itself has become orchestral.

Interactive Experiment

The most interesting question this project raises is not whether the machine got it right, but whether you and the machine agree.

In the gallery, visitors can scan a QR code shown below or click this link, listen to a short musical excerpt on their phones, and choose which landscape feels closest to what they hear. There is no correct answer. Some people will match the music the way the system did. Some will hear something different entirely. The piece becomes a small perceptual experiment — a way of noticing how each of us, including the machine, builds an imagined world from sound.

Your interaction with and engagement with the art piece and study are very much appreciated.

Further documentation and related projects can be found on my personal website.

How It Works

The system listens to a piece of music and gradually forms an imagined environment from what it hears. Rather than directly visualizing sound, it translates musical qualities into atmospheric and spatial tendencies — light, terrain, movement, weather, and scale.

A short audio excerpt is analyzed for acoustic characteristics such as tempo, brightness, energy, textural complexity, and dynamic contrast. These perceptual cues guide the construction of a natural landscape description, which then becomes the basis for image generation through a diffusion model. The process remains intentionally interpretable rather than opaque: each landscape can be traced back to specific musical tendencies and environmental associations.

I deliberately chose SDXL-Turbo for its softer, painterly quality rather than a photorealistic model. Cross-modal perception is rarely precise or one-to-one; it lives instead in suggestion, atmosphere, and emotional resemblance. The generated images are therefore not intended as definitive translations of music, but as spaces for the viewer’s own imagination and perception to enter.