I’ve been collecting my thoughts about the long and very interesting process of making my latest short film, The Dream. Here’s the whole story, from conception to completion.
1. The idea
After finishing my first short film/music video, Aria for Strings, last year I knew right away that I wanted to take on another project. A bigger, better, more complicated and more challenging one. Aria for Strings was a dry run and seeing it turn out well gave me the confidence to dive further into uncharted waters.
As I let my imagination run free I came up with two core concepts: electronic music mixed with live solo musicians and surreal/dreamlike imagery. In addition to that I decided that my overarching mission would be to be as creative as possible, to try new things, and to experiment. I know from experience that as long as you work with dedication and intelligence you can make even the wildest dreams become reality. This was the chance to put that philosophy into practice.
2. The music
Step one was writing the music. I wanted to allow it to take shape organically and let the finished composition inspire the direction of the visuals. I wrote the entire thing in MIDI using Sibelius and REAPER, with the electronic instruments provided by Miroslav Philharmonik, Steinberg Retrologue 2 and Pianoteq.
In Aria for Strings I used orchestral samples in a very traditional way, imitating the sound of a classical string ensemble. That’s fine: digital instruments do a fairly good job mimicking the real thing. But you can do so much more. Any sound you can imagine is possible, so why not use the medium to its full potential? I decided to have the core textures consist of synth sounds (Retrologue), supported by more traditional piano, string and percussion parts.
I also wanted to showcase solo, live musicians throughout. The main problem with digital instruments is that they completely lack the capacity to express subtle emotion. Sure, you can massage them a bit to fake a human touch, but it will never be quite good enough. Digital excels for backing material, but solo parts require a talented, trained musician.
I spent the first months of 2016 composing off and on and everything came together quickly. I wrote parts for singer and violin, first alternating solos, then coming together for a duet at the end. I decided to write with a sort of smokey, jazzy feel for the voice, leading into an up-tempo neo-Baroque violin “concerto.”
I was very happy with how it was all taking shape (and was having a blast with the synth stuff), so once I finished the next step would be to find some soloists and book a recording session. Around that time I also started meeting with my model, Brianna, to test visual concepts and potential camera setups.
3. Camera tests
In the interest of being as high-tech as possible, I had the idea to shoot the entire production in 4K resolution video. For those who don’t know, a 4K frame covers about 4 typical widescreen computer monitors. I used a simple Canon DSLR for Aria for Strings, but that camera only shoots in HD resolution.
Most 4K cinema-quality cameras cost as much as a small car and I knew that would be serious overkill. There are a few DSLRs that do 4K, but I would have had to buy new lenses and still spend a lot of money on what would in essence be a compromise between a still camera and a video camera. I looked seriously at the Panasonic GH4. One of my absolute favorite art films, Upstream Color, was shot on a camera from that line and it looks incredible. I wasn’t totally sold though, and then I happened upon Blackmagic Design’s URSA Mini. 4K, 60fps for slow motion, 1/10th the price of a true cinema camera and I could use my Canon lenses. I decided to rent one first to give it a try.
I met up with Brianna at the nearby lake where we shot Aria for Strings and compared my old Canon with the rented URSA. I was blown away. The URSA’s image quality was beyond perfect and it had a fantastic weightiness that made it very easy to shoot steady images while on the move. That was it: I would be either buying or renting one of these for the actual shoot.
4. The recording session
In the late spring I started sending out word that I was looking for a mezzo-soprano and a violinist. After some failed leads I connected with Kathy Andrew and Lindsey Warren, thanks to my former composition teacher in Vermont, Erik Nielsen. Both Kathy and Lindsey were open to working on an experimental project, so I sent them the parts and a description of generally how the process would unfold. I also contacted a few recording studios in NH and VT. I heard back from one just an hour north of where I live and made an appointment to visit. I was sold the minute I stepped in the door. CedarHouse Sound is located on a huge and lovely parcel of land in rural NH. The facilities are state of the art and the owner, Gerry Putnam, is a great guy and very experienced recording all different kinds of music.
I coordinated with Gerry, Kathy and Lindsey to find a mutually agreeable time to record and we settled on June 17th. The session itself was going to be challenging for a few reasons: we’d all be meeting each other for the first time, there was no rehearsal, and this whole project was a little non-traditional in that the musicians would be performing along with an electronic backing track. All in all it was a stimulating challenge (which on my end included preparing thoroughly and anticipating any potential pitfalls in advance so as to make it all go as smoothly as possible). Everyone rose beautifully to occasion and we were able to finish everything in about 4 hours. If you’ve ever spent time in a recording studio you know that’s record time! What you think will take an hour usually takes 5. At the end of the day I left with a USB drive filled with different takes that I could pick and choose the best from.
My next task was to go through everything, edit it together with the synth tracks, balance levels and add reverb. That took another few weeks of working off and on, but the high quality of everyone’s work combined with the relative simplicity of the musical textures made it a pretty smooth process.
With that, the music was finally done!
5. The video takes shape
You’d think after about 6 months of planning and work we’d be nearing completion on this whole endeavor, now code-named Project Combustion. Not quite! The biggest challenges by far were to come: coming up with the storyline and visual concept, costumes, filming, editing and special effects. You see, going back to my original idea about having surreal images, I decided that a great way to accomplish that look would be to include digital visual effects. I had never done anything of the sort before, but hey, I love a challenge and I have an unhealthy amount of confidence in some areas. I always figure that if I can imagine it I can do it. It just might take a while.
I started by listening to the music, closing my eyes and allowing images to come to me. The idea of fire kept popping up. The color red and dreams too, images from video games and animation, and a simple story arc about a woman leaving work, going home, setting up candles and entering a dream world full of bizarre goings-on. Brianna and I had been sending visual ideas back and forth for months on Pinterest and had a good idea of where we wanted to go with the various looks. After a ton of brainstorming and many, many revisions I finally settled on a set of maybe 30 scenes that could be pieced together into a coherent whole.
The only thing that was escaping me was the ending. I love a good twist and I just couldn’t think of anything. The ending is the most important individual part of any creative project. It’s the thing that people remember and it can make or break everything that came before. I left it on the back burner, knowing that it would come to me eventually. You can’t force creativity! It happens at its own pace. The artist just needs to keep the faith.
Brianna and I had begun meeting more frequently to rehearse and plan. We also were finalizing the different characters she would play in the dream sequence. After trying a few different things we settled on Demon form and Fire form.
Demons have horns (it’s a fact, look it up!), but after fruitlessly searching for the perfect pair to buy I just decided to make my own. They’re made of aluminum foil and spray painted and turned out fantastic. Brianna showed me a bunch of different outfits and accessories and we decided on a couple of different looks. It was all coming together. By this time I had come up with my twist ending as well (the Demon crosses over from the dream to the real world… or does it???).
Around this time I had also purchased my very own URSA Mini 4K. That was a whole new drama: the new 4.6K version was coming out and all the 4Ks were out of stock literally everywhere. After emailing something like 20 random camera dealers all over the country I found what was quite possibly the last remaining one in existence at a small company in Florida.
I was using a lot of test footage I shot with Brianna to begin to experiment with VFX. For the Fire form I knew I wanted to have the character be on fire (Project Combustion, remember?) but of course actually setting anything (or anybody!) on fire for real was out of the question. Slowly but surely I got the hang of After Effects and before too long I had a basic grasp of how it worked and what I needed to do to make my vision a reality. After Effects is not a simple or intuitive program to use, but thankfully there are a wealth of free tutorials online that cover every topic under the sun.
6. The shoot
Being the perfectionist that I am I had planned out in exhaustive detail how every shot would look and how the shoot would unfold. Brianna and I scheduled two sessions in August. I put together an extensive shot list and we discussed the best order in which to do things. She would have to change outfits and makeup multiple times, including one very complex look involving the horns and face paint. Brianna sent me some ideas over email of makeup looks she tried out on herself and I was thrilled with how perfect her Demon concept was. The success of the look of that character is due entirely to her amazing talents. For the Fire form she came up with a great red dress and wig and also made a super cool and creepy white mask.
Our first session took 5 hours and it was roughly a billion degrees outside while we worked. Sunny and humid too. The very first thing we shot ended up being the final scene, where the main character revisits a location from the dream and has to look cold and shaken. Brianna was a champ wearing a sweatshirt in the heat and pulled off some convincing shivering. For my part I was lugging around a nearly 10 pound camera rig all day. Every time we’d take a short break I’d realize how hot it was and how physically exhausted I was getting. Not only was it tiring but I had to stay mentally sharp to make sure we didn’t make any mistakes. This was pretty much our only chance to get most of these shots. No pressure!
At the end of the day we got everything done and it was time to adjourn for a few weeks before the second, final session. From what I could tell the takes came out great (although I couldn’t actually play back 4K footage at full speed on my computer at the time).
Session 2 was the fully costumed one. We had two outdoor locations, one at a lake about a mile from my place and the other a small forested area just a block or two away. We did the lake scene first, which was Fire form. Everything went great and we certainly attracted some interest from people out enjoying a Sunday afternoon by the water. From there we went back to my place and Brianna undertook the transformation to Demon form. We got her decked out in a black dress and accessories, plus the horns, and headed off to the forest.
Day 2 was another 5 hours, but we were finally done! The end was in sight. All I needed to do was pick my favorite takes, finish the visual effects (VFX) in After Effects and edit together a final cut in Premiere Pro.
7. The final stage (warning: massive wall of highly technical text incoming!)
That sounds simple enough on paper, but I didn’t know at the time what a job I had ahead of me. You see, when I was practicing my VFX skills over the summer I didn’t really try to make things perfect. I just wanted to get the general idea and leave the detail work for when it actually mattered. As it happens though, the detail work is incredibly time-consuming and complicated.
The first thing that became abundantly clear was that my older computer wasn’t up to the task of handling an After Effects project with 20+ different layers. Calculating a single new frame would take 30 seconds or more. I had to upgrade to 32GB of RAM, a new Xeon workstation processor and Windows 10. That made a huge difference and I was able to work at a reasonable speed going forward, with complex frames taking only a few seconds to render.
I had a checklist of the effects I needed to create, so I knew that, while I had a long road ahead, all I had to do was take it one step at a time and eventually I would be done. The first thing I needed to do was mask Brianna away from the background for a teleportation effect I wanted to have while she was in Demon form. At our second shoot I filmed a separate background “plate” of the forest that I could drop her on top of. Ordinarily you’d want to do this kind of thing with a green screen, which I considered, but it became apparent that for what I wanted to do we’d need a warehouse-sized green screen that covered the ground as well. Not a possibility. So, I had to use the Mocha rotoscoping tool included with After Effects to cut Brianna out, frame by frame. At first progress was glacially slow, but I gradually started learning tricks to work faster and got it down to 4 hours per 100 frames of footage. I had about 1000 frames to get through, so I was looking at 40 hours total for just the roto work on this one shot. Brutal, but that’s just the way it is.
Every day I would put on headphones, listen to music and just mindlessly plug away at it. Lo and behold, one fine day I finally drew the last mask and it was time to move on. I had a ton of other effects to accomplish: lens flares/light rays, eye glows, flames and flame ignition, black smoke, the distortion that happens with each teleport jump, sky replacement, and the final Demon overlay for the twist ending. The lens flares were pretty easy with the amazingly comprehensive and easy to use Optical Flares plugin from Video Copilot. Black smoke and fire were taken care of using Rampant Design’s high quality stock footage. They provide tons of fire and smoke clips shot against black backgrounds. You drop the clips into your project and key out the black so that your original footage shows through. The smoke is originally white, but using the Invert feature in After Effects it was easy to make it into a sinister black.
I had worked out the distortion effect a few months before and it came together pretty easily. I created a layer with random fractal noise and used that as the source for a displacement map on the footage I masked out of Brianna walking towards the camera. It looked super cool. I had been watching the TV show Fringe around the same time and they had a few characters teleporting around. Honestly, I liked the teleport effect I came up with better!
The sky replacement proved to be a bigger challenge. I knew how to do it, but making it look good was another story. The sky in the original Fire form zoom shot was totally washed out because it was such a sunny day when we filmed. The good news was that made it easy to key out because it was all the same greyish-white color. The bad news was that I needed to zoom the sky image in tandem with the camera zoom. After Effects includes a motion tracker which generally works extremely well. You select two high contract points in one frame and let the software go frame by frame and analyze how the two points move relative to each other. You then paste those data points into a null object and parent it to whatever layer you want to move along with the camera. However, there are always a few hiccups and it never gets 100% perfect tracking. The sky replacement image will track amazingly well for hundreds of frames, but then suddenly it will pop out of sync and look painfully amateurish. What I ended up doing was going through frame by frame and paying attention to any discrepancies, no matter how small. Most of the time the solution was as simple as deleting that keyframe in the null object and letting After Effects “guess” at the position of the sky instead. After deleting maybe 50 keyframes that way I got an extremely smooth sky animation.
That brings us to what ended up being the most difficult part of the whole process: the rain. It was key to the story to have rain come down and extinguish the fire, but getting actual rain was impossible. Even if we somehow managed to film on a rainy day the problem is that real life rain just doesn’t show up on camera very well. You might see a streak here and there, but in order to get a theatrical downpour you’d need to have just an absurd amount of water coming down.
After Effects includes a vector rain simulation which looks okay, but where it falls flat is that it doesn’t provide any depth. In the real world some rain drops would be closer to the camera and would be larger. You also have a lot of motion blur, so that drops turn into long streaks with slightly varied coloration thanks to refraction and reflection. I did a ton of research and eventually discovered a plugin called Trapcode Particular that mathematically simulates all different types of particles. In VFX particles can be used to create all sorts of natural phenomena: fire, smoke, and rain.
I watched a few tutorials and eventually ended up using an actual picture of a single water drop that I set After Effects to use as the baseline texture for each raindrop. I applied motion blur and set up two different layers of rain. The second layer of smaller drops in the distance was there so I could mask off Brianna and the railing she was leaning against so that it looked like the rain was falling behind them. I tried it without the mask but it looked totally fake to have the small drops falling in front of the foreground. Moving them to a background layer made all the difference. It looked great, but the catch was that exporting this particular shot took over 4 hours. I needed to render a finished, low-compression 4K clip of each VFX scene for the final Premiere cut, so around this time I started leaving my computer on all night to process stuff. I’d wake up in the morning to 4-6 finished VFX shots, hot off the presses.
Of course the problem was that often I’d want to fix something later, so I’d have to go back to After Effects, make the changes and then spend another 10+ hours exporting. Well, that’s just the nature of the beast! It all worked out in the end. Somehow…
After 2 months of pretty much constant work the VFX were done. Then it was time to piece everything together in Premiere, do color grading and time everything in sync with the music. That all went relatively smoothly and for the first time I actually got to see if this all was really going to work or not. To be honest, I had my doubts from time to time. I had never done anything of this scope before and I was a complete novice when it came to VFX. I was pretty sure I was on to something good, but you really never know, despite your best intentions.
Imagine my relief when I played back the first rough cut and it WORKED. I went on to spend another week or so tweaking details but for the most part I didn’t have to change much. The only final challenge was when the Demon crosses over to the real world (or does it???), which involved Photoshopping out the makeup from a profile shot I took of Brianna, rotating and scaling it, and using that as a mask over a layer of black smoke with a slight distortion effect.
Then, two days before my release deadline I had a sudden flash of inspiration. I wanted this to be surreal, right? Well, what better thing to do than instead of just a normal blue sky in the Fire zoom shot, plop a giant planet in there? I spent some time Photoshopping a high-resolution picture of Mars onto my sky image and eventually came up with something I liked. That was the final piece of the puzzle and made the scene really pop. Before that it was honestly a bit boring to watch.
8. Conclusion
So, there you have it: after almost a year in the making, The Dream is now a reality! From beginning to end this was an enormously enjoyable process. Brianna is always a delight to work with and without her talents and creative input the whole thing would just be a shadow of what it is now. Kathy, Lindsey and Gerry did incredible work and brought the music to life better than I ever could have hoped. And of course all the tools I ended up using were essential, especially the incredible Blackmagic Design URSA Mini 4K camera.
I’m taking a little vacation now after pretty much burning myself out from overwork these last few weeks. But yes, I’m already planning the next project. It’s going to be even bigger and more complex than this one. I can’t wait!
Buy albums on Amazon & iTunes!
Leave a Reply