When we set out to make our "clicky keyboard" music video, I had no idea how to gather all the samples, line them up into a song, and cut together a totally synced video to represent the track. I knew this kind of thing had been done before, but I found surprisingly little about how to actually pull it off. So here I am, after figuring out a kind of long, annoying process, to share that process with you. Hopefully this helps someone down the line.
The first step, obviously, was shooting the keyboards. We rolled on a pair of shotgun mics and two camera angles. For each keyboard, we got a variety of takes and key presses at different volumes and styles. Basically, we gathered a pallette of samples.
Next, I used Pluraleyes to sync the two angles and the stereo recording, bringing all that into Premiere. In Premiere, I identified all the samples I would potentially want to use in the song. I wound up with about 40 of them, and placed numbered markers on a timeline of these samples, with names like 01_IBM_space, in order to keep them identifiable later. I exported each clip on that timeline separately as AIF audio files, keeping the filenames the same as the markers.
The next stop was Logic. In there, I created a new EXS24 instance (that's Logic's built-in sampler instrument). The plugin lets you drag in a group of files and assign them to successive notes.
At this point, I had a MIDI instrument loaded up with our keyboard samples, so I could create the track with the standard piano roll. For the purposes of looping, I wound up using a handful of different instrument tracks, even though they were all copies of the same EXS24 instrument.
After the track and the mix were locked, I collapsed all the MIDI events across all the tracks into a single MIDI region. Then, I used separated MIDI events by note pitch (found in Edit > Separate MIDI Events > By Pitch). This gave me a new group of tracks, where each one was every instance of a single sample. Annoyingly, I had to manually name each track to its sample name, so that I could match them up with the Premiere clips later. Once that process was done, I bounced each track into its own .aif file (Export > All Tracks as Audio Files).
Back in Premiere, I added all of these clips to a new sequence. And here comes the most annoying part: for every audio track, I had to paste its matching video clip onto the timeline for every hit of that sample. I used the waveforms to find each hit visually, and checked sync along the way. At the end of this tedious process, I had an enormous timeline of 50 video tracks, and replaced all of the Logic stems with the final mixdown of the song.
From here, the editing process was simply deciding which clip(s) to make visible at any moment in time. For the areas of the piece where the rhythms were hocketed (one sound at a time), I used straight cuts; during heavily polyphonic areas I split-screened clips.
The thing that strikes me about this process is how programmatic it is, but how there is no software (to my knowledge) that could have automated this specific rig. VJ software wouldn't have been quite right, because I didn't want to just trigger video clips; I wanted to have handles to the left of each clip so that it could start earlier than the audible sample hit. I'm by no means claiming that this method was the "right way." But it worked for me, and if you know of any less manual way this could have been pulled off, please let me know!
Loading comments...