QLab 2: Custom Rendering with Quartz
If you have never used custom rendering before, get ready to have your abilities with QLab increase dramatically. If you have done any research you know that Quartz Composer is a pretty nifty little beast. In any case here are some sample patches for you to try and use in your next workspace.
Download nifty patch. Save somewhere useful on your machine. Open QLab. Add a video cue, and link to a valid video or image file. Go to composer tab, check "Custom Rendering with Quartz Composer File...". Input the appropriate file path for downloaded patch. Press 'GO'. Commence "Ooh-Ahh" sequence.
Mask Patch v1.0
I guess I had something to do with the creation of this repository, so I had better do something to keep it from being bare. I created this patch to accept incoming video from QLab and use a B/W image as a filter in order to control the output of the projector onto the set. See attached photo on right. Notice that there is video black in the doorway areas so that imagery does not end up on actors, unless they wave a hand in front of the jamb. (Thank you maid on SR....)
This patch has been tested on OS X 10.5.8 and 10.6.7. I am going to go thru this as plainly as I can, sorry to advanced users for the verbosity.
Make the Mask
If you're ready to begin, fire up Photoshop or some equivalent. Open a new document and specify that the dimensions of the document match the output resolution of the projector. If you don't have a preference, using native resolution wherever possible is best, IMHO. Whenever possible, I find it's best for the QLabber to do any scaling and not the built in algorithms of Projector Manufacturer Name Goes Here. That said, luckydave would tell you to scale all of your images in an external editor so that QLab does as little work as possible and thus does not hiccup as much during playback of multiple video cues. See this post.
If you know how to use QLab to output video, you probably know how to get your newly created photoshop document onto your second screen, i.e. the projector. Here's the tricky bit, be sure to zoom the Photoshop document to 100% (located in the bottom left corner of the document window), and press the 'f' key twice, or select View > Screen Mode > Full Screen Mode.
Now, use the paint bucket tool and paint a new layer with plain ol' barn door red. I think you'll find it makes fine tuning your filter easier when you can easily see the arrows and handles. To do this, first press 'd', which will set the default colors to Foreground Color = Black, and the Background Color = White. Click on the black box under the tools, and it will open up the color picker dialog. Choose the reddest red and that will become your foreground color. Next, choose the paint bucket tool by pressing 'g' or clicking on the icon. Click anywhere on your document, and ta-dah, one easy as pie red background.
Now, if things are going well the objects your projector is pointing at should be bathed in red. Return Photoshop to default colors, by pressing 'd' again, and then press 'x' to swap the foreground and background colors. This should result in the foreground = white and background = black. Choose an element in the raster area, such as the doors in the example photo to make into a mask filter. Once you have selected your target, choose the pen tool by pressing 'p', or by clicking the icon. Click around the perimeter of the object and photoshop should create a shape object with an outline matching you physical object. If you want to get fancy, you can drag the handles of your shape to help follow curvy objects, but that's a Photoshop lesson, not QLab + Quartz.
Hang in there we are almost to the fun part.
Now, you should have a red raster with a white blob in it somewhere. Turn off the visibility of the red layer by clicking on the eye next to the layer name. Now you should have a gray/white checked background with a white blob still hovering on your object. Go to File > Save As... and call your blob something useful, like door_shape. Don't hit save yet! Once you name your blob, change the format to "PNG" or Portable Network Graphic. PNG's retain RGB plus K information, which is great for us. Now you may hit save.
Editing and Enabling the Patch
Open up the Quartz Composer patch attached to the bottom of this post. You will see 4 patches in the main window. The patch in the top left reports to the other patches how big the projector's output is and thus how big to make the sprites, ostensibly matching pixel for pixel. The patch in the middle is a "box" for two patches within, if you double click it, it will open up and you'll see those patches, a sprite and a clear layer. Clear layer ensures we start with a clean slate each time. The Sprite patch receives input from QLab. Once QLab sends a video file to this patch, this sprite will be the one on which the image resides. It may help to think of each output patch as a layer of glass.
When you want to go back to the main patch, click the edit parent button. The large patch to the right is yet another sprite, which performs an nifty function. Since we have a sprite buried in the middle patch, we can manipulate the image on it independently of this outer sprite. That's great because once we build the necessary white blob images in Photoshop, we don't want them moving around, just the image that our QLab video cue pipes in. This way, all of the publishable qualities, or parameters of QLab video cues: x, y, scale, rotation and opacity, will function as intended, but only on the source file in the QLab workspace. The fourth and final patch is the small one just left of the main-level sprite. This is an image patch and it's output is connected to the mask image input of the outer Sprite.
So, drag your "door_shape.png" file into Quartz. Hook up the image output of your PNG file to the "Mask Image" input of the outer Sprite. You can delete the original PNG at this point. Save the file as something useful, like "door_shape_mask.qtz".
Open up QLab, and drag in an image or video file. Click on the Composer tab and select "Custom rendering with Quartz Composer file:" radio button, then double click in the gray bar to the right. Direct QLab to "door_shape_mask.qtz". When you are ready hit GO. You should have an image, visible only through the area you outlined in white in Photoshop. Parts of the original image may be cut-off, but that's a good thing. This way you only get as much of the image as you need to cover the object, and the rest is "hidden" under video black.
Start combining shape layers for complex output, and making lots of individual ones too. Each time you make a new shape layer, save it as a PNG, then drag it into Quartz and update the patch. Save as many versions of the patch as you need for the show. For the show in the picture above, we had a pile of patches including: SR Door, SL Door, SR Screen, C Screen, SL Screen, All Screen, Both Door, All Surface etc. You can use the individual layers in Photoshop to create combos in any way you see fit.
Thanks and Good Luck!
Questions and feed back to: munk at utdallas dot edu
Update for 2013
Well, it's been quite a week here at my theater home. We have been pushing the envelope again and got ourselves a few papercuts! Thanks again to luckydave for guiding me to enlightenment in the sometimes ethereal world of video. Did you know that even though your output screen(s) isn't displaying the pixels outside the bounds of your raster area, it IS still trying to account for and render them? It seems obvious, but, consider how often you move things on and off screen using QLab. Consider how often you scale things to crazy big dimensions just because it looks cool. Well, I have been forced to learn the hard way, and I do mean the hard way, that QLab, while a robust and usually happy-go-lucky show buddy, will in fact become moody and quite intractable when you ask it to do the impossible. Especially when you ask repeatedly, in quick succession, like 700 times with 100 or so different video files. Shame on me.
Soooo, to the point. I wrote the patch above many moons ago. For the show we just opened, on opening night, I was re-rendering our videos to Photo-JPEG and had to hold house while QLab finished bundling to the SSD. Night two, I had to hold house after cropping all the videos I had scaled the night before back to raster sized output while QLab bundled to the SSD. In light of that, I felt it prudent to revisit the Mask Patch. I have learned the importance of only asking QLab (and your hardware) to manipulate the pixels you NEED, not all the pixels you HAVE in your video assets. An important distinction! In this era of Terabytes and Gigaflops, we forget that we only really need the pixels we are showing to the audience. In the end, we often render useless (pardon the pun), lots of the pixels we have created and stored. This waste manifests itself as glitchy playback and crashes due to overtaxed system resources.
I have added, for those of you who hadn't already, a crop patch. I put it in between the "Render In Image" patch and the final output "Sprite". This allows you and QLab to do whatever you like with your images, and *I think* dynamically remove anything you aren't using i.e. pixels that fall outside of the raster area. It's really hard to see the results of extraneous pixel output that don't exist anywhere except inside the mind of your graphics card. The point is you can't see what it's doing, you and I will just have to trust that it is working as intended. It should help give your system a fighting chance to keep up with your creativity. By cropping images you are sending via QLab to within the bounds of your raster area, your graphics card doesn't have to worry about things no one can see. I don't know if it will result in any noticeable increase in speed as far as frame-rates and "animation smoothness" are concerned, but after this week, I definitely feel like it should go under the heading of "Best Practices."
Projector Blending using Modded SEB Patch v1.0
What's better than one projector? Two, of course. Which can lead to wondering: How might an operator get two projectors to behave as one again? Why bother? Many good questions, dear reader.
First answer: If you have ever searched 2 projector blending, there is some fancy math to describe and even fancier mathematical solutions to the problem. The condensed answer is, in order to get two projectors to appear as one seamless output, a user must compensate for what would be a doubling of light in the blend zone. Translation: The blend zone, relative to the rest of the image, will appear overly bright because there are two projectors projecting the same thing in the same place. More details (and elegance...) here. At the top of the article is a ZIP file with the original SEB patch which I mangled into what is available below. Special thanks go to Mattias Oostrik for giving me permission not only to use it and post it, but also to mangle it.
Second answer(s): You need HD resolution but can't afford an appropriate PJ. You'd like to wrap an object with pixels. You'd like to do something very wide, with respect to aspect ratio. Or, as in the case that led to the development of this patch, I had a huge still image, two HD PJ's and wanted to show as much of the image as possible, while also layering logos and some video timers over the "background".
Set-Up the Projectors
The only method I know works is: one Super-Fancy-Hot-Rod video card feeding a Matrox TripleHead or DualHead. In most cases, the ability to run multiple super-fancy Hot-Rod video cards does not exist. Therefore, I use the Hot-Rod card to feed the PJ(s) and a dinky video card for the operator monitor since that screen need not be able to do fancy transforms, on multiple layers, in front of a live audience, smoothly. This chews up some motherboard bandwidth, but I think it is best to separate the activities of the QLab from the Operator monitor activities. Let your Super-Fancy-Hot-Rod card focus on the important task. Then you need to go about setting up your outputs to match the "Native Resolution" of each projector. This is one place you really don't want on-board (read: uncontrollable) scaling to mess with your output. It's hard enough to get the pixels and optics to behave without the projector(s) arbitrarily moving pixels about. BIG CAVEAT: if you have two flavors of projector, all hope abandon ye who enter here! Especially in the case of trying to blend LCD with DLP, the difference in technology will be incredibly apparent. If the lens curvatures are different, matching the difference in the rate of expansion of the light-cones proves difficult. Quite frankly, it's difficult even to blend two PJ's with different numbers of lamp and/or service hours. Just check out the images below! In short, you need twin projectors, not cousin projectors and definitely not one Montague and one Capulet projector. Once you have your PJ's hooked up, detected and pointed in a similar direction, It's time to open the patch.
This takes a little doing, so get comfy. First open the patch in Quartz and connect the test pattern to the image input of the "Render in Image" patch at the top level. This should turn on a white pattern with black numbered grids and some circles. If you full screen the viewer in Quartz on your projector output, you should get two PJ's spewing grids and numbers. At this point, begin having lots of fun dialing in your projectors. Start by leveling them. It's easiest to run zooms to one stop or the other and then do your best to focus the image, and then start moving around the projectors and their feet. The more you try to fix the image with the optics of the projector, the harder it will be to get good alignment, because the optics (theoretically) need to match exactly. It's relatively easy for human eyes to detect straight horizons and vertical bars, as opposed to apparent differences in geometry of a lens surface. It's also a good point of reference if the top and bottom of the image match at the blend zone. It should be an indicator as to which is out of square and which way it needs to go, e.g. the edges meet like / \ or \ / or | \ or / |, etc.
After getting level both horizontally and vertically, you can fiddle with matching the blend zone edges where you see the center circle and grids 6, 18, 10, 42, 54, 66, 78, 90 and 102, although this will shift according to your blend width. I have found it most effective to move the projector left and right while maintaining a right angle to the projection surface, rather than to angle it. Angling the light-cone just leads to more 3D trouble. Tip: this is one of those places where "Good Enough For Gov't Work" is an excellent starting place. You can adjust the size of the blend zone by tinkering with the math patch inside the Matrox patch. 2^7 is 128, 2^8 is 256, etc. The patch will adjust to whatever amount you decide to blend, just be aware that for every pixel of overlap, you lose 1/2 pixel of overall resolution.
Here are three ugly photos of the result after alignment. It is clear that one lamp is much brighter than the other, these two PJ's also suffered from burn spots on their LCD panels, but, it worked for developing the patch. There was only room for improvement, and I didn't have to worry about how long I left them on! I'm not sure if some of the differences could be corrected using a device like a Spyder3 as far as color and temperature, but that may be one thing to try if your output ends up looking like this.
Once your projectors are satisfactorily aligned, you can exit full-screen mode in Quartz. Re-publish the "Input" port or close without saving(if appropriate) to keep the patch working once you assign it to videos in QLab. If you look at your output surface now, you will clearly see the overlapped section, because it is brighter. If you were to put an image/document/window in the blend zone it will look funny because Quartz is not altering the output of the projectors anymore. This is a good sign.
Questions and feed back to: munk at utdallas dot edu