TY - JOUR
T1 - Sonification of a 3-D Spider Web and Reconstitution for Musical Composition Using Granular Synthesis
AU - Su, Isabelle
AU - Qin, Zhao
AU - Saraceno, Tomás
AU - Bisshop, Ally
AU - Mühlethaler, Roland
AU - Ziporyn, Evan
AU - Buehler, Markus J.
N1 - Funding Information:
This research was supported by Office of Naval Research grant N00014-16-1-2333; US National Institute for Health grant U01EB014976; and the Center for Art, Science, and Technology (CAST) program at the Massachusetts Institute of Technology, with continuing support from Studio Tomás Sara-ceno. We are grateful for the inspiration to develop this instrument for a public performance in the context of Tomás Saraceno’s exhibition “On Air” at the Palais de Tokyo, Paris, in November 2018, supported by Festival d’Automne à Paris. The authors of this article contributed to the work in the following ways: Buehler, Ziporyn, and Su designed the research and carried out the analysis. Su developed the visualization techniques and videos; Buehler, Su, and Ziporyn developed the web sonification method, with guidance from Qin and Saraceno; Buehler developed and carried out the granular synthesis and created the musical scores as well as the associated audio processing, and prepared the associated audio files; Su, Ziporyn, and Buehler interpreted the data and wrote the text, with assistance from Bisshop, Saraceno, and Mühlethaler.
Funding Information:
This research was supported by Office of Naval Research grant N00014-16-1-2333; US National Institute for Health grant U01EB014976; and the Center for Art, Science, and Technology (CAST) program at the Massachusetts Institute of Technology, with continuing support from Studio Tomás Saraceno. We are grateful for the inspiration to develop this instrument for a public performance in the context of Tomás Saraceno’s exhibition “On Air” at the Palais de Tokyo, Paris, in November 2018, supported by Festival d’Automne à Paris. The authors of this article contributed to the work in the following ways: Buehler, Ziporyn, and Su designed the research and carried out the analysis. Su developed the visualization techniques and videos; Buehler, Su, and Ziporyn developed the web sonification method, with guidance from Qin and Saraceno; Buehler developed and carried out the granular synthesis and created the musical scores as well as the associated audio processing, and prepared the associated audio files; Su, Ziporyn, and Buehler interpreted the data and wrote the text, with assistance from Bisshop, Saraceno, and Mühlethaler.
Publisher Copyright:
© 2021 Massachusetts Institute of Technology.
PY - 2020/12/1
Y1 - 2020/12/1
N2 - Three-dimensional spider webs feature highly intricate fiber architectures, which can be represented via 3-D scanning and modeling. To allow novel interpretations of the key features of a 3-D Cyrtophora citricola spider web, we translate complex 3-D data from the original web model into music, using data sonification. We map the spider web data to audio parameters such as pitch, amplitude, and envelope. Paired with a visual representation, the resulting audio allows a unique and holistic immersion into the web that can describe features of the 3-D architecture (fiber distance, lengths, connectivity, and overall porosity of the structure) as a function of spatial location in the web. Using granular synthesis, we further develop a method to extract musical building blocks from the sonified web, transforming the original representation of the web data into new musical compositions. We build a new virtual, interactive musical instrument in which the physical 3-D web data are used to generate new variations in sound through exploration of different spatial locations and grain-processing parameters. The transformation of sound from grains to musical arrangements (variations of melody, rhythm, harmony, chords, etc.) is analogous to the natural bottom–up processing of proteins, resembling the design of sequence and higher-level hierarchical protein material organization from elementary chemical building blocks. The tools documented here open possibilities for creating virtual instruments based on spider webs for live performances and art installations, suggesting new possibilities for immersion into spider web data, and for exploring similarities between protein folding, on the one hand, and assembly and musical expression, on the other.
AB - Three-dimensional spider webs feature highly intricate fiber architectures, which can be represented via 3-D scanning and modeling. To allow novel interpretations of the key features of a 3-D Cyrtophora citricola spider web, we translate complex 3-D data from the original web model into music, using data sonification. We map the spider web data to audio parameters such as pitch, amplitude, and envelope. Paired with a visual representation, the resulting audio allows a unique and holistic immersion into the web that can describe features of the 3-D architecture (fiber distance, lengths, connectivity, and overall porosity of the structure) as a function of spatial location in the web. Using granular synthesis, we further develop a method to extract musical building blocks from the sonified web, transforming the original representation of the web data into new musical compositions. We build a new virtual, interactive musical instrument in which the physical 3-D web data are used to generate new variations in sound through exploration of different spatial locations and grain-processing parameters. The transformation of sound from grains to musical arrangements (variations of melody, rhythm, harmony, chords, etc.) is analogous to the natural bottom–up processing of proteins, resembling the design of sequence and higher-level hierarchical protein material organization from elementary chemical building blocks. The tools documented here open possibilities for creating virtual instruments based on spider webs for live performances and art installations, suggesting new possibilities for immersion into spider web data, and for exploring similarities between protein folding, on the one hand, and assembly and musical expression, on the other.
UR - http://www.scopus.com/inward/record.url?scp=85095454224&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85095454224&partnerID=8YFLogxK
U2 - 10.1162/COMJ_a_00580
DO - 10.1162/COMJ_a_00580
M3 - Article
AN - SCOPUS:85095454224
SN - 0148-9267
VL - 44
SP - 43
EP - 59
JO - Computer Music Journal
JF - Computer Music Journal
IS - 4
ER -