The science of the sound of the heart

Share this on social media:

Topic tags: 

Felix Grant 'sonified' his friends' heartbeats and found that young men respond to popular music's base rhythm, but young women to female vocals

The heart is a prime example of art/science crossover. During the month of February, readers are likely to be surrounded by stylised images of the human heart. For societies derived from West European cultural traditions, the heart is currently (it has not always been so) the symbolic seat of affection. The heart also symbolises courage (Lionheart, Braveheart, and so on). The Tin Woodsman in The Wizard of Oz1 is seeking a heart in both those senses.

There is no contradiction between knowing the heart to be a physical pump, and recognising the image of the heart as an icon for complex psychophysical emotional processes. This sort of crossover is a particular interest of mine, so it piqued my interest when Ray Girvan mentioned a heart-rate variability sonification project2. Penn State University's (PSU) department of music is co-operating with physicists and physiologists to use digital music software to transform the sequence of intervals between consecutive heartbeats into an electro-acoustic soundtrack. It seeks a diagnostically useful aural analogue for ECG irregularities, and simultaneously to feed the resulting 'soundscapes' back into future development of musical form. The project represents a pragmatic manifestation of the art/science interface, which is one aspect of its appeal to me.

  • Upper plot: An aggregate of 'at rest' heartbeats before music began.
    Centre plot: The effect of listening to Bonnie Tyler singing
    Total Eclipse of the Heart.
    Lower plot: Staccato fluttering patterns in response to wordless vocal sounds from Chen Yi.

Linkage between music and heartbeat has a cultural history as old as recorded society: the tendency of heartbeat to modify in response to strong close rhythms has always been exploited in social bonding ceremonies. Sonification as a discipline has a history stretching back a couple of decades, at least, with some of the latest information available from ICAD's 2004 conference proceedings3. An interesting discovery is that the healthy heart is not as regular as we think it is.

The topic also has a direct intellectual appeal to me as a statistician. Sonification concentrates not on the 'kerTHUD kerTHUD' of the heartbeat, but on chromatic sensory representation of variations in its period - on differences and rate of change rather than the phenomenon itself. PSU's Mark Ballora and others2 have used SuperCollider, a software sound synthesis (SWSS) programming language, to 'sonify' such features as interbeat interval (represented as a sinusoidal wave frequency), interval variation (phase modulated annotation of that pitch), moving standard deviation over 300 periods (pulse speed and number of harmonics in an overlaid waveform) and moving means over five and 15 periods (described respectively as 'clarinet-like' and 'glassy' sounds).

Having read the available material, I played with the team's program (freely downloadable for use by anyone with access to a Mac) and it is deeply intriguing. There are controls allowing the user to adjust the degree and manner of sonification of each attribute, and thereby the shape of the resulting soundscape. The future of complex data interpretation is very much in such cross-modal exploitation of human sensory richness - this example came from application of a musical tool to medical purpose; could I use conventional mathematical and statistical software to derive a visual result which was both informative and aesthetically satisfying? Could I learn anything from the attempt? The real comparison would require me to create moving images, but still ones would be a start. I took the measures used by the PSU team, devised a context and experimental setup, then asked a patient group of artists and art students to explore that context with me using a laptop full of mathematical and statistical software (typified by, but not by any means limited to, MuPAD and Genstat, with support from the likes of Autosignal).

ECG is beyond such Blue Peter-style setups, but small microphones taped to the skin recorded the heart beat onto one stereo channel, and a wrist pulse onto the other, as music was played to the victim through small in-ear phones. The resulting sound files can be loaded directly into some analytical and visualisation software, or sampled into any worksheet, but the data used in sonification is usually a second or third generation derivative, so pre-processing was done in generic spreadsheets and mathematical products or signal and peak software.

A number of interesting indications were thrown up, and the visual representation of them was sometimes aesthetically striking. The upper plot in the accompanying illustration shows an aggregate of 'at rest' heartbeats before music began. After listening to Bonnie Tyler singing Total Eclipse of the Heart4 20 times without pause, some volunteers' heartbeats appeared to be responding mostly to boredom and/or irritation; others, however, showed apparent links to particular sound components. Heartbeats of young men showed greatest response to base rhythm, altering gradually away from their pre-recorded norms towards the underlying beat of the song's backing. That of young women seemed more receptive to the vocal component; for example, slowing in sympathy with delivery of the refrain's last line (the centre plot in the illustration). Older volunteers showed a mix of the two responses, without gender differentiation. The sudden appearance of wordless vocal sounds in Chen Yi's Percussion Concerto, II, Prelude to Water Tune5, produced the same staccato fluttering patterns in all listeners (lower plot).

I make no claims to a rigorous scientific study, never mind a medical one, but it was interesting and rewarding. This sort of exploratory tinkering is, after all, how much serious research starts out. Time and opportunity permitting, I'd be interested in exploring further ideas. In the meantime, though, the real work in sonification delivers real results - and exciting possibilities in both scientific and artistic terms.

Bibliographic notes

1. Baum, L.F. The Wizard of Oz. 1900.
2. Ballora, M., et al. Heart Rate Sonification: A New Approach to Cardiopulmonary Diagnosis. Penn State University.
3. Hunt, A. and Hermann, T. The importance of interaction in sonification. In ICAD 04 - Tenth Meeting of the International Conference on Auditory Display. 2004. Sydney, Australia: ICAD (International Community for Auditory Display).
4. Steinman, J. Total Eclipse of the Heart. 1983, Sony Music Entertainment
5. Glennie, E. Oriental Landscapes. 2002, Bis Records AB: Akersberga