As I prepare for an upcoming S16 transfer, I have the option of getting an HD telecine or a UHD film scan. In the end, I'd want a ProRes file for editing in Premiere Pro, but if I go the scan route, I'd also have the DPX sequence along with a ProRes file rendered from the individual DPX frames. As I'm familiar with the concept of individual DPX frames from film scans, I'm not well versed when it comes to working with them, so hopefully someone can help with some of my questions.
I don't have an exceptionally high powered editing system (I'm editing on a MacBook Pro with Premiere Pro CS6), so is there much of an advantage to me in having the individual DPX sequence files, and is there much I can do with the DPX sequence in Premiere Pro? My understanding is that the individual DPX frames are around 10 MB each, so I'd imagine that you would need a fairly powerful system to be able to edit directly or perform color grading with the individual DPX frames. I think using the ProRes file is much more in line with my system's capabilities, so I'm just curious what, if anything I could do in Premiere Pro with the DPX files. Even if I can't really use the DPX sequence for real-time editing, is it still possible to perform color correction on a DPX sequence in Premiere Pro, or is DPX color grading typically performed elsewhere?
Also, since DPX files are essentially a "flat" scan of the original camera negative, should I expect a DPX sequence to be pretty flat in terms of color saturation and low in contrast, even if properly exposed, with the expectation that some type of color correction will be performed on the resulting image sequence? And, if that's the case, would a one-light ProRes file rendered from the individual frames essentially have the same flat-ish color and contrast as the original DPX sequence?
I have a lot to learn about the DPX workflow, so any help would be appreciated. Thanks!
First of all, I recommend getting the UHD transfer as you call it. Will it really be UHD or 2k for S16 though? Usually S16 is scanned at 2k.
Your editing system does not have to be "high-powered" to handle DPX. You do need a fast data connection to play back at 24fps though. Don't get hung up on this, 24fps playback of your scan is likely unnecessary.
When you get the scans back, they will be flat. Low contrast and low saturation. Premiere can interpret DPX image sequences as a singular movie. You'll want to bring the DPX into a timeline to encode a ProRes editing proxy. Given your computer setup, I would recommend making 2 prores files - a 2k "online" ProRes4444 file that you'll color correct later, and a 1080 or 720 422 editing proxy. Don't touch the color of the 4444 file, but add some contrast and saturation to the low quality editing proxy so you can tell what you're looking at.
When you're done editing, you'll replace the editing files in your timeline and color correct the 2k version. I'd recommend color correcting in software like DaVinci Resolve, which is free, albeit with a bit of a learning curve.
But here's why you should bother going through the trouble of the DPX. The scanner they use for the DPX sequence is almost certainly a different scanner than their telecine setup. And telecine scanners can't interpret the entire tonal range captured on your negative. DPX will preserve more information (even when encoded into prores) and give you more creative freedom with color after you edit instead of making these choices before. And, down the road, if you decide to revisit this project and have access to a more robust setup, you can return to your DPX master and have all the information possible to play with.
Thanks for the reply, Dan. You've answered many of my questions and provided some excellent information for a DPX newbie like me
I do have a couple of follow-up questions. If I receive a DPX sequence, I'm assuming it will be a collection of many individual .dpx files...one for each frame of film. How do I go about importing this into Premiere in preparation for generating a 2K ProRes4444 file? I'm assuming (incorrectly, perhaps) that I'll have to drag the frames onto the timeline, but perhaps there's another method of doing this.
I didn't realize that DaVinci Resolve is available as a free download, so I'll definitely check it out. It looks like, at least for Mac, that only the "Lite" version is free, however I'd imagine that it should be more than sufficient for someone like me just getting started with color grading.
You did mention color correcting the ProRes4444 file in Resolve. Would there be enough data / tonal range from the original DPX sequence preserved in the albeit high resolution, but more compressed ProRes4444 file that would make the ProRes file suitable for color correction?
Also, this is more of a general question about DPX files and something one probably needs to grasp in order to understand the significance of a DPX sequence or a negative scan in general, but what is inherent or integrated into a DPX file that makes it so suitable for color correction? I've seen a few stills of flat DPX scans, and I was surprised at the lack of contrast and color saturation, almost as if they had been over-exposed. That said, what is it about a DPX file that allows you to pull good contrast, saturation, and an overall good image from an image that, on the surface, looks so lousy?
There is nothing intrinsically different about DPX other than that it is uncompressed (and it supports various film-industry metadata, but that may not be that relevant to you). In practice they tend to be used for images stored with higher precision than something like a PNG file you'd find on the internet - storing an image a 10 bits per pixel, for instance, is common, and means that each of the RGB channels is stored with 1024 levels of brightness, rather than the 256 of an 8-bit scan.
This makes it practical to store much lower-contrast images. You'll put contrast back into them in the grade, but because there's finer encoding of luminance, you don't get ugly artifacts. Of course this can become a circular issue, since you only had the low-con scan done because you had 10 bits available in the first place so... anyway. Ultimately, you get more information to do more things with in the grade.
The very low-con, odd-looking scans you describe may well have been logarithmically encoded. This is (usually) a luminance encoding that ensures each f-stop of exposure information occupies something like the same number of digital values. This is not how most displays expect the information to be supplied so the image looks extremely flat and washed-out. Again, this is mainly useful with 10-bit encoding, otherwise the large contrast corrections required to make it look normal on a normal display become problematic. Resolve supports various log encodings; liaise with the scanner people to ensure they're giving you something you can handle properly (in the worst case scenario, you can often fix it by hand, but better not to).
The difference between high-res prores and DPX is likely to be vanishingly insignificant, especially if you're shooting 16mm to begin with. Prores is compressed, very much like a JPEG. DPX isn't, very much like a BMP or PNG. But the prores compression is fairly light.
If you want to get a bit more techy, you can convert the DPX frames to prores quicktimes en masse using the free tool ffmpeg, and a commandline such as:
This will convert DPX files with names such as inputfile.00001.dpx (and subsequent files) to prores.mov. Alter the filenames to suit. Alter the %05d to howevermany digits you have in your DPX file names. Adjust -r to a suitable framerate. Adjust -qscale up to maybe 13 if you need to save a bit of space. This will give you Prores HQ. I think you can add -pix_fmt yuv4444p10le to the command line if you want 4444, but I haven't needed it personally.
ffmpeg is a great tool, though if you're intimidated by that command line stuff, Premiere will display an image sequence as a single file in the import dialogue. Or sometimes you just select the first image and check a box that says "image sequence" and it will bring the whole sequence and display it as one entry in your media browser. I have CS5 so I imagine CS6 is even easier to deal with.
DaVinci Resolve Lite has only 2 limitations: No noise reduction and no 3D tools. Every other feature is available, including UHD output now. Just don't expect any customer support if you have trouble with it - they don't care if you're struggling with something you got for free.
Last time I scanned S16 it was to FullHD (2011), which was more than enough resolution given the lenses used maxed out at around 1200-1600 pixels in sharpness (superspeeds used around T1.3-5.6). It was shot on Vision 3, T200 and D250 so with better lenses UHD might have been worth it, but then it cost like 3-4 times more to do 4K, and we would struggle in post even with 2K which was twice the cost from FullHD and only 10% more resolution.
I took the DPXs into after effects and made ProRes files for editing, in FullHD. But you can just as well use premiere nowdays.
Are you scanning and storing them in Log or Linear? I did Log and then used the cineon-effect in After Effects (premiere has it too) and once finished editing just re-linked and graded in AE.
You are looking at around 1TB per hour of footage. When I scanned last time I was moving between MACs and PCs, and USB3 wasn't really out yet, so I had to stick to USB2, which was very slow and the bottleneck for production. Just transferring the scanned footage took a 24 hours (and then it isn't backed up)... With USB3, that should be around 3-4 times faster, since my disks stopped around 20MB/sec, and nowdays they go about 60-80MB/sec on USB3 or Thunderbolt. So expect 1-3 days of just transfer/backups including rendering out proxies.
Edited by Karl Eklund, 14 December 2014 - 04:23 AM.