Abstract
This thesis presents the challenges and current state of the art related to soundscapes modeling and design. The concept of soundscape comprises various fields of knowledge and craft: from scientific to technical and artistical. Moreover, current scenarios demand new digital content creation (DCC) paradigms focused on user generated content (UGC) platforms and on-line repositories of sound assets, like Freesound. The work carried out focuses on DSP methodologies that can be used to support the creation and design of soundscapes. We stress their relationship with the nature of interactive and immersive environments.
One application area is explained for the use-case of 'sound concepts' that need to be modeled using material from various recordings, carried using different setups or sound design techniques. An homogenization algorithm is presented with the aim to reach a certain homogenization across groups of samples by means of auditory-based features. Then, a study about the transformations that can be applied using this model are presented as equalization methods. To support the development and evaluation of the techniques presented, a prototype in Matlab and the integration of a SuperCollider server with the game engine Unity3d have been carried. Additionally, some applications and use-cases are also mentioned within the contexts of interactive (non-linear) and linear sound design.
Keywords:
Audio, DSP, Sound Design, Games, Simulation, Soundscapes, Analysis, Synthesis, Interaction, Content-based audio transformations, Cognition, Perception.
Sound Experiments
Homogenization
ORIGINAL Footsteps over ice
HOMOGENIZED Footsteps over ice (MEL-scaled filter bank, 3 iterations)
ORIGINAL Footsteps over concrete and ice
HOMOGENIZED Footsteps over concrete and ice (MEL-scaled filter bank, 8 iterations)
ORIGINAL Siren and alarm sounds
HOMOGENIZED Siren and alarm sounds (Gammatone filter bank, 2 iterations)
ORIGINAL Speech
HOMOGENIZED Speech (MEL-scaled filter bank)
HOMOGENIZED Speech (Gammatone filter bank, 2 iterations)
Source to target transformations
Cepstral envelopes (computed using LPC analysis) comparison.
SOURCE Slow water stream
TARGET Medium water stream
TRANSFORMED SOURCE TO TARGET Slow to medium water stream
TRANSFORMED SOURCE TO TARGET Slow to medium water stream (8 iterations)
Timbre variations
ORIGINAL Snare sound
VARIATIONS Snare sound (MEL-spaced filter bank)
ORIGINAL Footstep over ice
VARIATIONS Footstep over ice (MEL-spaced filter bank)
ORIGINAL Keychains shake
VARIATIONS Keychains shake (MEL-spaced filter bank)
MFCC computation for 8 timbre variations (red)
across 13 coefficients of an original step over ice sound (blue).
Downloads
Thesis report [PDF ~4 MB] Alternate download from the MTG-UPF
Presentation slides [PDF ~1.7 MB]
6th Audio Mostly paper [PDF ~0.5 MB] ACM Digital library link
Sound experiments [ZIP Various MP3 192kbps@44100Hz ~5.2 MB]
UnityOSC, an Open Sound Control implementation for Unity3d
Unity3d scripts for the Soundscapes server [ZIP ~36 KB]
Matlab source code and test sounds [ZIP ~7 MB]
Demo binaries (Win, Mac) and Unity3d project [ZIP ~176 MB]
Citations
Thesis
Garcia, J. (2011). Samples Homogenization for Interactive Soundscapes. MSc thesis in Sound and Music Computing. Universitat Pompeu Fabra, Spain.
@mastersthesis { Garcia2011,
title = {Samples Homogenization for Interactive Soundscapes},
author = {Garcia, J.},
school = {Universitat Pompeu Fabra},
year = {2011}
}
Conference articles
Garcia, J.; Kersten, S. & Janer, J. (2011). Towards equalization of environmental sounds using auditory-based features. AudioMostly conference, Coimbra (Portugal). Published by ACM, New York (US).
@inproceedings{ GarciaKerstenJaner2011,
title = {Towards equalization of environmental sounds using auditory-based features},
author = {Garcia, J.;Kersten, S.;Janer, J.},
booktitle = {AudioMostly conference, Coimbra (Portugal)},
year = {2011}
}