virtual reality music making Part I

Let’s remind ourselves of the paradigm shift that’s moved the fundament from beneath our feet: we now live in the Virtual Reality age of creating music.

The entire last paragraph makes me shudder with the revulsion of knowing that this post will be replete with trendy terms overused by corporatese-speaking wannabes. Suffice it to say that a zeitgeist can be picked up and shared by anyone and in that democratic spirit that is no less true of myself.

I’ve wanted to blog (there we go again) about my reaction to Panayiotis Kokoras’ Towards a Holophonic Musical Texture and Morphopoiesis: a general procedure for structuring form for quite a while now but I want to discuss the newfound music-creating environment we find ourselves in nowadays first. I’m still startled at how few people “get it” - which I’ll get to in a bit here - even the people who ostensibly use the same tools as I do to create music with computers.

The tools I use are VST plug-ins. They are both my instruments and effects and effects have always played an inordinate role in my music - I was a “sound designer” long, long before I’d ever heard the term.

Less fundamentally I use FL Studio and Usine, one to compose with and one to play computer live, respectively.

FL Studio has these particular benefits: rapid song composition workflow, free lifetime updates and the ability to be hosted as a VST plug-in.

Usine’s most important features for me are: full modulatiry, rapid updates based both on user’s suggestions and visionary new features, the ability to route any sort of data and transform it however you’d like thus one can use whatever hardware midi interface you like and the underlying software will change the virtual instruments, effects and midi routing automatically as it automates the part of the song you’re playing, or switching to the next song which can be done via song timeline event automation or at the push of a qwerty key or a hardware midi control, plus the ability to be hosted as a VST plug-in itself.

There’s much more to Usine but for the current discussion these are the important points.

So it all boils down to modulatity and plug-ins. Why plug-ins?

Unlike in the hardware-only days in software the data is liberated from restrictions. One needn’t memorize a General Midi drum template, worry about how to route 16 different midi channels, figure out how to send program change messages to advance to the next synthesizer patch or effect processor preset. That alone is a major step that should not be overlooked.

Now one’s performance options are limited almost solely by the lack of vision of hardware manufacturers, your imagination and your patience. The hardware limitations can be worked around with modularity, though. For instance I’ve got a EWI midi horn with a USB output instead of a midi cable. With Usine I can create keyboard splits that allow me to, say, trigger 5 different VST instruments with 5 different played octaves. Or I could set up one octave of notes to represent 12 different routing schemes where the pitch bend and breath control parameters automate different VST effect parameters, say the note middle C instantiates “setting 1″ where PB controls filter cutoff and BC controls filter resonance of a VST bandpass filter, and then C# causes the EWI control a buffer override effect’s buffer size and buffer divisor… the possibilities are mind-boggling.

As if these features didn’t expand our sound creation palette sufficiently consider that the VST plug-in standard, as well as VST-creation software tools (SynthEdit, Synthmaker and until recently Pluggo), non-VST DSP applications (Pd, Max/MSP, et al.), as well as many of the plug-in hosts themselves (notably Usine and Reaper) distribute SDKs (software development kits).

I do not know how to code software. But online there’s an improbable amount of people who do, who’ve tackled the task of creating innovative DSP applications and freely distribute them.

So in essence what’ll keep me using a computer, and specifically WinXP, is crowdsourcing and the concommitant Baldwinian evolution. No one person really has the time or resources to code all of the fun audio toys I’d like to utilize but by and large a planet full of people will churn out enough people who are interested in sharing freeware audio applications that the result for me is having hundreds and hundreds and hundreds of VST plug-ins. All I have to do is learn how they each work and dream up a use for them in my particular idiom.

Granted, that last part is the sticking point for much of the digital music world. Very few seemed to have been hipped to this pofound paradigm shift. It is complex but not intractable and I never tire of trying to help people realize that with a fairly cheap computer (I’ve got the third most powerful intel processor, an i7 920, stuck in a box with a power supply and the newfangled mobo and ram that it requires for $700, overclocked to have four 4.0 Ghz cores which provides even me, the worst-case scenario in realtime processing requirements, a sufficient amount of oomph) and an internet connection and a couple of cheap VST hosts you can leap straight into the virtual reality age of music creation.

It is a lot of work. The learning curve is steeper than learning to bang out some chords on a guitar. But the benefit is that you become the entire band, the composer, the sound designer, the producer, the creative studio engineer artist, the live audio reinforcement mixer, and can even pull it all off live on stage as the sole performer, or inporporate other musicians all hooked up to one PC as the nerve center whether they be midi-instrument players or utilizing analog inputs (mic’ed singing or acoustic instrument playing, electric guitar or what have you) all meeting in the automating mixing DAW that it seems one could only dream about 10 years ago.

You get nearly infinite power to create music at a cost of much less than most single musician’s gear would cost with few drawbacks - and even most of those can be worked around with some forethought and perseverance. Luckily the internet supplies plenty of free expert advice.

This particular music composition paradigm allows you to freely experiment and rapidly prototype new DSP. Websites will inform you when new free plug-ins have been released and where to download them. No more looping expensive tape all over a room, requiring numerous people’s hands to steady it all just to create a dub echo. No more multi-person hands-on sessions of riding the faders for each and every mixdown. Simply drop the VST .dll into your plug-in folder, start your host software, throw an effect on a mixer channel or it’s equivalent and hit play. If the presets don’t work for you (they rarely do for me) here’s the fastest way to experimentally find new sounds:

Hook up many of the plug-ins parameters to automation in the host. Draw a ton of random automation squiggles for each automation channel and hit play. When the sound is something you’d like to repeat later hit pause, go to the plug-in interface and “save as a preset” - it’s that simple.

Of course, what I prefer is not simple at all. There are many levels of delicious complexity beyond this trick. For the automation can be scores in a composition just as the notes can be. This means that it’s the equivalent of having dozens and dozens of trained human participants in the studio moving the knobs and dials of your synthesizers, mixers, and effects processors exaclty where you want them to be as the song progresses.

And when you twiddle the knobs oftentimes that’s when the most amazing, heretofore unheard, sound occur. Imagine a dub delay who’s tape you can freely move across the reading head like you play an instrument, playing it in reverse or at whatever speed you desire and can even instantly leap to the position you want it to play from with inhuman accuracy, while the filter parameters are similarly under realtime control, and always exactly repeatable.

Imagine a guitar with a wah-wah pedal and talkbox-type effect where the wah isn’t simply jiggled up and down but performs according to a rhythmic pattern of 2 eight notes followed by a qarter note triplet or whatever else you can think of, while synced to the vowel that the talkbox’s formant filter is set to mimic.

This is impinging nearly on the promises of virtual reality in audio, though the visual component lags far behind. As my main interest for over 15 years has been using modular DSP units with a VR interface to make multichannel 3D audio compositions in a way just barely more efficient than this (using goggles and gloves and icon-based software manipulation) this comes as no particular surprise to me except inasmuch as it’s cheap and already available off-the-shelf.

So let us delve deeper into this paradigm shift.

When you add the incredible power of crowdsource-evolved freeware DSP plug-ins to modularity and multichannel audio what can happen?

As a simple example let us take one sound source, say a human voice. Mic it and set the input of your analog-digital converter to your modular plug-in host. Now use a 3-band frequency splitter with three outputs. Route each of the resulting inputs to a different “track” - one for bass, one for mids and one for high. Beatbox into the mic and set the frequency bands so that the “kick,” “snare,” and “cymbals” noises each reliably get routed to the appropriate output. Now you can add DSP plug-ins to each of those sounds in isolation. You could add an eq boost to the bass, even a subharmonic enhancement plug-in, a compressor and any other tool that you feel might enhance your “kick” vocalizations. For the mids your snare can be gated, you can add a de-esser and some filtering to reduce plosives without affecting the “cymbal” noise. And so on.

Now think beyond that. Say you want a dub delay on the “snare” vocalization. Make 2 outputs for the snare. One will be the clean version and the other will have the dub delay. Perhaps you’d like to modulate the dub delay’s tape speed via both the kick and cymbal audio inputs. In Usine you can insert a zero-crossing counter onto the kick track (which won’t affect the sound of that track) and a envelope-tracking module onto the cymbal track. Insert both of those into a math module and mix the signals however you’d like (probably with a smaller ratio of the kick’s zero-crossing counter, as there’s be very rapid modulation from such an audio trigger) and then route that to the dub delay’s tape speed VST parameter inlet, and add a data module to gently reset the tape speed automation back to zero when not receiving a signal from the audio-based modulation. Can you hear in your head what will occur to the dub delay?

Since there’s 2 outputs on the snare you can blend the wet/dry sound to taste. I’d use an expression pedal to do this, but song-timeline-based automation could work for a hands-off solution, bring in the echo only at the end of phrases…

That’s enough ranting for now. More to come….