Imagine you are sitting in the darkness of a mixing theatre. You’re attending a spotting session for a television series. The current episode features a gun battle. The director stops the playback and shares notes for a World War II flashback-style sequence. He is imagining a sound design tableau of explosions.
As the director explains, it begins to dawn on you what he wants: a dramatic swell of hundreds of explosions that surround the listener in a 5.1 soundscape.
You begin planning the edit in your mind. You’ll need to find and cut each explosion, spread them out on dozens of tracks, then pray there’s enough time in the premix to place them around the soundstage.
You glance at the calendar. You feel your stomach drop. The episode is due in two days. You just won’t have enough time.
Thankfully, there’s software that can solve this problem simply and creatively, in only a few minutes. Today I’ll share details about the software and tips for using it, as well as interview with its creator.
About the Sound Particles Software
An interesting email arrived in my inbox a few weekends ago. It was a conversation from the Yahoo Sound Design Group. The poster, professor Nuno Fonseca, announced a new sound design tool called Sound Particles (Mac OS X 10.8+, $299 USD, free for educators and non-commercial projects), and invited everyone to try it. Comments about the software also appeared in a Gearslutz thread.
So, what does Sound Particles do? In short, the software assigns a clip (called a “particle”) to a location in a three-dimensional environment, then moves it around this space. This is all done automatically.
So, you could take a gunshot, and make it fire repeatedly in this soundscape, moving it however you wish. This can allow for The Matrix-like bullet time panning not only in stereo, but overhead, behind the listener, and more.
That’s pretty impressive on its own. However, Sound Particles goes further. It is able to assign an unlimited amount of particles around this virtual space. Now your single gun can fire in, say, 10,000 places. It’s now become a full-pitched sci-fi battle that surrounds the listener. You can also add multiple gunshot clips, and Sound Particles will play them randomly and naturally throughout the soundscape.
Inspired by CGI
In website videos, professor Fonseca explains that his idea was born in the realm of computer graphics. Those video pros use “particle systems” for creating natural-looking rain, flame, and smoke. Sound Particles adopts this technique for audio. So, instead of using a virtual camera in this space like a CGI artist would, Sound Particles uses a virtual microphone instead.
The results are impressive. Beyond saving the time of recreating such soundscapes manually, the particle system technology sounds better and more natural than using synthesis; it uses original sound clips instead.
Sound Particles is available in three versions, free, education, and full. The free software is limited to non-commercial projects, 50 particles, and 2 channels. The $299.99 USD version is completely unlimited, and can be downloaded from the Sound Particles website. The educational version appears to be the same as the full version, and is free to applicants with education credentials.
How Sound Particles Works
How do you create a Sound Particles environment?
Here’s a rough workflow:
- Choose the sound clips you want to use. These are the template for the soundscape particles.
- Choose how many particles you want. How many times you want the clips to appear in the soundscape? This can range from one to millions of particles.
- Choose a space for the particles. This is the size and shape of the soundscape (sphere, box, width in meters, etc).
- Design how the particles will move in the soundscape (Doppler, over-under, left-right, explosion from centre, etc).
- If you wish, apply modifiers and random alterations (gain, delay, EQ, time and pitch, etc.).
- Design how the sounds will be perceived or captured. This is pretty much like placing “microphones” or audio channels in the virtual space. There are presets for stereo pairs (X/Y, M/S, ORTF, Decca Tree, etc), multichannel environments (5.1, Dolby Atmos, etc.), Ambisonics, and so on.
For example, let’s say you choose 6 gunshots sound files as your source. Then you choose 100 particles so that a random selection of these clips will be firing 100 times. You place them in a spherical environment, where the clips will move left and right within it. Maybe you specify that once in a while they change pitch just a bit. You want all this to be captured in surround, so you change the settings to 5.1.
At this point, you’re pretty much ready to go. If you like, you can layer a second soundscape on top of the first. Impressively, Sound Particles allows you to stack additional groups of particles (say, a separate howitzer soundscape layered on top of the first gun soundscape), each with additional microphones. You can even add a video, if you’d like to see how your sound design work lines up with picture.
That’s a lot to absorb all at once of, course. Not to worry: you can begin quickly by choosing a template with pre-set preferences instead (explosion in a sphere, Doppler by left to right, jumping, etc).
Then what? Well, a cool 3D view allows you to test how the sound particles move in the space. When you’re satisfied, you then render the sound. Depending on the amount of particles and other settings, this time will vary. You may then play back the result, and export it for use in another editing app.
Of course, the software is quite sophisticated. I’ve only scratched the surface of its potential in my tests.
To get full details about how it works, visit the Sound Particles Learning page. There are many videos that explain how to use the software.
Here’s the first YouTube video which provides an overview of the software:
And here are some sound samples created by the software:
Creating a Battle Composition
I’ve been playing around with the demo software. I found it well-designed and easy to use. Just for fun, I tried recreating the soundscape I mentioned in the story above.
Here are my settings:
- I chose the suggested “Doppler explosion (lite)” setting.
- I selected 13 source howitzer files. The second set sourced from 8 gunshot clips.
- I was limited to using 50 particles in the demo version. However, I found that was too overwhelming for the booming, reverberant type of howitzer clips I was using. I cut this back to 25 particles and had better results.
- I changed the starting zone to a variety of surfaces, which you can see named in the sound playlist below.
- I added two audio modifiers, a random delay (custom: 5 seconds) which spread things out a bit, and a random pitch (default settings) to vary things.
And here are the sounds:
Notice in particular how the shape of the surfaces affect the soundstage and composition of the final clip.
It’s easy to try out new ideas. Those clips took no more than five minutes to set up and create.
You’re welcome to download those tracks to test them in your own sound editing apps (click the small arrow beneath and to the left of the waveform in the player above).
Interview with Sound Particles Creator Nuno Fonseca
I was fascinated after reading the thread on the Yahoo Sound Design Group, and quite intrigued after trying the software myself. I was curious to hear how the software was created. I reached out to Nuno Fonseca and asked if he would like to share info about the app with us. He kindly agreed.
Creative Field Recording: Can you tell readers a bit about yourself, and how you began to work in sound?
I always loved technology and sound. I recall doing a vocation test when I was a kid, and I got the exact same score for “engineering” and “music.” I ended up doing a degree in electronic engineering and then a masters and a PhD in computer engineering.
I worked a few years in the IT area, but I decided to teach to be able to do research and my “crazy” projects, like creating “WordBuilder” for EASTWEST Symphonic Choirs.
Currently I teach at two universities: computer engineering and game development in one (ESTG/IPLeiria), and music technology on the other (ESML/IPLisboa).
Creating Sound Particles
CFR: You explain Sound Particles by comparing it CGI 3D rendering software with examples like creating flame, smoke, rain and so on. The example makes me wonder how you came up with the idea of Sound Particles. Can you share with readers how Sound Particles began?
Probably 10 years ago I noticed that the most interesting visual effects (VFX) on movies were using particle systems to create things like fire, smoke, stardust, etc. And at that time I thought “Wouldn’t be nice to be able to use particle systems for audio?” But that was just an idea and time passed by.
In 2012, I finished my PhD, and I noticed that no one was using particle systems for audio, so I decided to start creating a particle system software for sound applications. And now it’s out…
CFR: Can you share your experience actually coding the software itself? Did you code it yourself, and were others involved in testing?
Yes, the software was entirely created by me, from the code to the graphical user interface. Currently, the software has nearly 100,000 lines of code, spread over 400 files, so it’s a relatively big project for a single developer.
In the end of 2014, I started showing it to the Hollywood studios. For instance, Skywalker was very interested in starting testing it as soon as possible.
Fortunately, I had the help of fantastic testers, some from major studios, others completely independent, that helped detecting most bugs and giving important feedback.
CFR: What was the most technically challenging aspect of creating the software?
Software like Sound Particles uses many different concepts internally that I needed to fully understand to implement them: physics of movement, audio interpolation, random number generation, audio filters, immersive audio, ambisonics, convex hull, OpenGL, CoreAudio, code optimization, etc. As such, the first challenging aspect was the diversity of subjects that I needed to cover. Some of those things I already knew them, others I needed to learn more, and some subjects required learning everything from scratch.
Currently, the most challenging aspect is scale – Sound Particles is so vast in terms of code, that in any change I must be sure I’m not ruining anything else.
CFR: Are there any future plans you have for the software that you are working on right now?
The priority is to release a AAX version for Pro Tools in the second half of the year (first AAX native and later on AAX DSP). Other platforms, like Windows or other plug-in architectures are also possible, depending on the commercial success of the software, to allow hiring additional programmers.
Also, I have a list of around 100 additional features that I would like to implement in Sound Particles, so the work never stops.
Using Sound Particles
CFR: Can you share some practical examples of how a sound designer could use your software?
Sound Particles can be used for many applications, some of them not even obvious to me when creating the program.
Of course, it’s great to create ambiances, with many sound events happening on the background, like a calm forest or a crazy sci-fi city.
Several sound designers are really linking it for Doppler, because not only it has great sound due to the real modelling of sound propagation, but also allows you to think on a real 3D world (velocity, actual movement, mic position) instead of low level parameters (volume, pan, pitch-shift).
Then, there are the granular sounds, like ice breaking or fire.
Eventually, a less obvious application would be immersive signal processing – imagine a multi-tap delay in a 3D space to increase a sense of space, or splitting the sound in bands to have a kind of spectral movement (different harmonics moving in different directions), or simply trying to create you own version of THX Deep Note.
CFR: You mentioned that some productions (such as Black Sails) are using Sound Particles. Can you share examples of how productions are using the software?
The current standalone application is more suited to sound designers, and it’s being used essentially as a sound design tool – sometimes you want to create the sound of many ghosts around you (remake of Poltergeist), sometimes you want the sound of a storm on the high seas (Black Sails), etc.
And then, when the sound designer finishes getting that sound, it can simply export it in any format (e.g. 5.1 or Dolby Atmos 9.1 bed) as a group of WAV files to be imported on the main mix session.
CFR: Can you share some of your favorite features of the software?
What I really love about Sound Particles is its ability to sparkle the imagination of people. In the moment you understand what it does, you cannot stop thinking on crazy things that you never imagined before, like 1000 flying violins or a huge immersive evil fire.
CFR: Do you have any tips for new users to the software?
The software is different from any other existing software, and as such, you will need to understand it to get the most out of it. I would suggest to watch the video tutorials (you can always laugh with my English).
From a more practical point of view, your main concern is to prevent long render periods, especially when you are in the middle of something. And to do that, you can use the granular modifier to use only a few seconds of audio in each particle (instead of a long original audio file), start with a single mono microphone, work on a small time project before making it grow into a longer audio scene.
And, of course, be creative…
Thank you to professor Nuno Fonseca for taking the time to share his experiences and his thoughts about his software.
Read More
- Visit the Sound Particles website.
To stay in touch, receive free updates by email newsletter or RSS feed. | Follow on Twitter, Facebook, YouTube, or SoundCloud.