Jump to content




Photo

16mm 4K scan edit workflow for digital and film out 23.98 or 24p?


  • Please log in to reply
21 replies to this topic

#1 Brian Rose

Brian Rose
  • Basic Members
  • PipPipPipPip
  • 893 posts
  • Student
  • Kansas City area

Posted 23 April 2016 - 09:27 PM

Hey all,

 

I'm getting closer to beginning shooting and editing of my new documentary!  I'm shooting Super 16 BW and am getting a 4K scan.  I'm having a little trouble nailing down my workflow, having done some earlier tests with transcodes and relinking to DPX files, and the results were rather wonky, with cut points changing when I relink, etc, etc.  I suspect I made a mistake with the frame rate, and wanted to pose this general question to  you all about workflow.

 

My goal is to have a 4K master, for theaters and for future resilience as UHD blu-ray and UHD streaming gains further traction.  I'd also like, if I successfully raise the money, to produce at least one 35mm print from the 4K DI so I can have that option as well.  

 

My plan is to get 4K DPX files, and for editing cut using 4K prores proxy transcodes.  What I'm uncertain about is, just what framerate do I edit with and transcode to, given the stated goals for distribution?  Do I use 24p or 23.976?

 

Your help and expertise will be deeply appreciated!

 

BR


  • 0




#2 Tyler Purcell

Tyler Purcell
  • Sustaining Members
  • 2363 posts
  • Other
  • Los Angeles

Posted 23 April 2016 - 09:50 PM

Why not get Pro Res XQ? Its a full 12 bit RGB format and most transfer houses can deliver it. The great thing about XQ is that Avid, Premiere and Final Cut X use it natively. So you don't need to do any conversion, drag, drop and edit.

You'd edit at 24fps for theatrical and going back to print.

Are you shooting B&W negative or reversal?

Sounds like a cool project, I'd like to know more about it.
  • 0

#3 Brian Rose

Brian Rose
  • Basic Members
  • PipPipPipPip
  • 893 posts
  • Student
  • Kansas City area

Posted 23 April 2016 - 11:40 PM

Why not get Pro Res XQ? Its a full 12 bit RGB format and most transfer houses can deliver it. The great thing about XQ is that Avid, Premiere and Final Cut X use it natively. So you don't need to do any conversion, drag, drop and edit.

You'd edit at 24fps for theatrical and going back to print.

Are you shooting B&W negative or reversal?

Sounds like a cool project, I'd like to know more about it.

Thanks!  I may give that Prores a try.  I like to get DPX from the lab because it's nice to have the film is the "rawest, most uncompressed" state possible, and because the lab charges extra to do the conversion.  But on your advice I may try converting the DPX to prores XQ.

 

I'm shooting BW negative - Orwo 400 since I'll be shooting in some tough locations, with low lighting, plus some night shooting.  Had a lot of great results pushing up to 2 stops with that stock.  

 

The story itself is an incredible one.  It's the culmination of three years of research.  It's about a student who, while on a class trip, vanished without a trace.  No clues, no indication if even a crime took place.  If I can pull it off, it will be like In Cold Blood...minus the solution.


  • 0

#4 Tyler Purcell

Tyler Purcell
  • Sustaining Members
  • 2363 posts
  • Other
  • Los Angeles

Posted 23 April 2016 - 11:48 PM

Very cool!

DPX is 10 bit BTW and it's a very old codec. Pro Res XQ is 12 bit and pretty much lossless.

With B&W, the bit depth becomes more important and honestly, if you do 12 bit, I'd go for it.

Come monday, you'll get your ear talked off by the "scanner" guys on here. :D
  • 0

#5 Dirk DeJonghe

Dirk DeJonghe
  • Basic Members
  • PipPipPipPip
  • 543 posts
  • Industry Rep
  • Kortrijk,Belgium

Posted 24 April 2016 - 01:10 AM

There is nothing to prevent you from working in DPX 16 bit.  If you need to change one single shot in the middle of a 2 hour program, DPX suddenly becomes much more easy to work with than any Quicktme codec. Just rerender the shot and you are done. That said, you need a fairly heavy RAID to playback DPX in real time without proxies.


  • 0

#6 Bruce Greene

Bruce Greene
  • Basic Members
  • PipPipPipPip
  • 393 posts
  • Cinematographer
  • Los Angeles

Posted 24 April 2016 - 01:12 PM

Hey all,

 

I'm getting closer to beginning shooting and editing of my new documentary!  I'm shooting Super 16 BW and am getting a 4K scan.  I'm having a little trouble nailing down my workflow, having done some earlier tests with transcodes and relinking to DPX files, and the results were rather wonky, with cut points changing when I relink, etc, etc.  I suspect I made a mistake with the frame rate, and wanted to pose this general question to  you all about workflow.

 

My goal is to have a 4K master, for theaters and for future resilience as UHD blu-ray and UHD streaming gains further traction.  I'd also like, if I successfully raise the money, to produce at least one 35mm print from the 4K DI so I can have that option as well.  

 

My plan is to get 4K DPX files, and for editing cut using 4K prores proxy transcodes.  What I'm uncertain about is, just what framerate do I edit with and transcode to, given the stated goals for distribution?  Do I use 24p or 23.976?

 

Your help and expertise will be deeply appreciated!

 

BR

What frame rate did you shoot in the camera?  If 24fps, stay with that.  If 23.98, stay with that.  If your timeline doesn't match your camera FPS, you will loose sound sync every 501 frames I believe.  And your sound recording should also match your camera FPS as well...

 

After you edit and mix the sound masters, you can change the frame rate for playback with a tiny shift in running time.  DCP for theaters will be 24fps.  Home video, in the USA, will be 23.98.  I usually shoot 23.98, edit, 23.98, sound mix at 23.98 and convert to 24fps only for the DCP for theaters.  Our post production sound mixer strongly prefers 23.98 for his workflow.


  • 0

#7 Robert Houllahan

Robert Houllahan
  • Sustaining Members
  • 1511 posts
  • Industry Rep
  • Providence R.I.

Posted 24 April 2016 - 11:19 PM

DPX is not a Codec, it is an uncompressed 32bit or 50bit RGB file format which is open.


  • 0

#8 Dirk DeJonghe

Dirk DeJonghe
  • Basic Members
  • PipPipPipPip
  • 543 posts
  • Industry Rep
  • Kortrijk,Belgium

Posted 25 April 2016 - 12:02 AM

On my scanner I can do 8, 10, 12 and 16 bit per channel DPX, but most applications I have won't recognize 12bit version. It is indeed open format, SMPTE approved, which means you are not vendor-locked into one particular workflow. Try combining Windows or Linux with a full Quicktime workflow.


  • 0

#9 Perry Paolantonio

Perry Paolantonio
  • Basic Members
  • PipPipPipPip
  • 347 posts
  • Other
  • Boston, MA

Posted 25 April 2016 - 07:18 AM

Very cool!

DPX is 10 bit BTW and it's a very old codec. Pro Res XQ is 12 bit and pretty much lossless.

 

Putting on my pedantic hat here: DPX isn't a "codec" in the sense ProRes is. [Edit: Gah! Rob beat me to it!]

 

That said, If the film is color negative, there is basically no difference between 10bit DPX Log files and 16bit linear. That's just the nature of log-encoded data. You don't really gain anything from the extra bits unless you're working with linear-encoded files. ProRes is generally linear (it can be log though), so if you're comparing 10bit linear DPX to 12 bit linear ProRes, then yes, you're getting a bit more with a 12 bit file, but not that much.

 

To the OP: we scan 16mm directly to ProRes or DPX, no charge for conversion because it's output by the scanner directly to either (or both) formats. There's definitely a lot more convenience with ProRes, but if I was planning on a filmout, I'd stick with DPX to avoid the compression applied by the ProRes codec. It's minimal (really minimal), but it's there. Hard drives are cheap, so there's really no reason not to scan to an uncompressed format like DPX if you want that.

 

It's easy enough to make a set of proxy files in ProRes at a lower res, to go along with a DPX sequence for  editing, and then simply relink with the DPX sequence later in the editing or grading tool of your choice. Best of both worlds..


Edited by Perry Paolantonio, 25 April 2016 - 07:20 AM.

  • 0

#10 Anthony Schilling

Anthony Schilling
  • Basic Members
  • PipPipPipPip
  • 992 posts
  • Camera Operator
  • Portland, OR

Posted 25 April 2016 - 03:30 PM

DPX is not a Codec, it is an uncompressed 32bit or 50bit RGB file format which is open.

I feel like working with DPX, TIFF or any codec free sequence file to be a step forward in my work flow over any codec. Maybe because i'm a Windows only guy, tired of Apple based codecs dominating the scene, tired of the codec boondoggle all together. My one experience with ProRes was too noisy, and I find codec free files to be a much smoother work flow. DPX and TIFF drop into resolve with no fuss.


  • 0

#11 Perry Paolantonio

Perry Paolantonio
  • Basic Members
  • PipPipPipPip
  • 347 posts
  • Other
  • Boston, MA

Posted 25 April 2016 - 03:33 PM

I feel like working with DPX, TIFF or any codec free sequence file to be a step forward in my work flow over any codec. Maybe because i'm a Windows only guy, tired of Apple based codecs dominating the scene, tired of the codec boondoggle all together. My one experience with ProRes was too noisy, and I find codec free files to be a much smoother work flow. DPX and TIFF drop into resolve with no fuss.

 

Can you define "noise?" -- there's no noise in the analog sense (random high frequency information) in a ProRes file. I seriously doubt that ProRes compression was visible unless you were using a lower quality (more compressed) flavor like LT or Proxy. ProRes HQ or higher is indistinguishable from uncompressed, including DPX. 


  • 0

#12 Tyler Purcell

Tyler Purcell
  • Sustaining Members
  • 2363 posts
  • Other
  • Los Angeles

Posted 25 April 2016 - 05:41 PM

Yep, there was a recent test done where the filmmakers went out with an Alexa and shot both RAW and Pro Res XQ. The difference was indistinguishable. They pushed it, pulled it, did all sorts of funky things to the image in order to break it up and nothing happened, both were rock solid.

Now I heard the Quicktime Pro Res decoder on PC's wasn't quite to the level on the macs. I'm not a windows guy and I do post production for a living. The WHOLE INDUSTRY is Pro Res and now that more and more cameras are shooting in that codec, it's become more popular as a complete workflow codec then it ever was before.
  • 0

#13 Robert Houllahan

Robert Houllahan
  • Sustaining Members
  • 1511 posts
  • Industry Rep
  • Providence R.I.

Posted 25 April 2016 - 09:15 PM

Yeah ProRes444 is very good and XQ is even better, that said you cannot decode 444XQ on a PC as far as I know. IMO it will be a long time before the film and tv industry moves away from ProRes it's just too bad that Apple is making junk computers these days.


  • 0

#14 Dirk DeJonghe

Dirk DeJonghe
  • Basic Members
  • PipPipPipPip
  • 543 posts
  • Industry Rep
  • Kortrijk,Belgium

Posted 26 April 2016 - 12:35 AM

It would be better for everybody if Apple would make the ProRes codec 'open' like Avid did with DNxHD.


  • 0

#15 Perry Paolantonio

Perry Paolantonio
  • Basic Members
  • PipPipPipPip
  • 347 posts
  • Other
  • Boston, MA

Posted 26 April 2016 - 08:53 AM

Now I heard the Quicktime Pro Res decoder on PC's wasn't quite to the level on the macs.

 

It's the same as in Quicktime 7 on the mac. Quicktime X is 64bit so the playback performance of higher resolution files is better there, but that's not available on Windows. QT7 is 32bit, so it's got some limitations. It's the same engine as in Quicktime 7 on the mac, less the ability to make ProRes files.


  • 0

#16 Perry Paolantonio

Perry Paolantonio
  • Basic Members
  • PipPipPipPip
  • 347 posts
  • Other
  • Boston, MA

Posted 26 April 2016 - 08:55 AM

Yeah ProRes444 is very good and XQ is even better, that said you cannot decode 444XQ on a PC as far as I know. IMO it will be a long time before the film and tv industry moves away from ProRes it's just too bad that Apple is making junk computers these days.

 

Scratch Play can do it and that's free (and 64bit). Apparently, it's just a matter of changing some metadata (the fourcc code) in the file, and then you can do it in Quicktime player as well. But it's a little kludgy and kind of a pain. There doesn't seem to be any technical reason why it won't work, but Apple didn't update Quicktime 7 player to support it without this little hack. 


  • 0

#17 Anthony Schilling

Anthony Schilling
  • Basic Members
  • PipPipPipPip
  • 992 posts
  • Camera Operator
  • Portland, OR

Posted 26 April 2016 - 01:02 PM

 

Can you define "noise?" -- there's no noise in the analog sense (random high frequency information) in a ProRes file. I seriously doubt that ProRes compression was visible unless you were using a lower quality (more compressed) flavor like LT or Proxy. ProRes HQ or higher is indistinguishable from uncompressed, including DPX. 

Interstitial plasma? on a few bright shots.. but what it was inherent to is hard to pinpoint. it was a shadow scan of some S8 so maybe more due to the colorist than the file? But wasn't really my point. My point is I found ProRes  a bit more cumbersome to deal with. I guess I just don't get why an Apple based codec is becoming the current standard? There are already enough headaches on various compatibility issues between Windows, Mac, codecs, hardware, media players, edit software ect... and I feel like image sequence files cut through a lot of workflow headaches for once. At least that's my perspective as a basement novice, between incoming and outgoing files. 


  • 0

#18 Perry Paolantonio

Perry Paolantonio
  • Basic Members
  • PipPipPipPip
  • 347 posts
  • Other
  • Boston, MA

Posted 26 April 2016 - 01:28 PM

What you're describing as noise definitely isn't something that's inherent to the ProRes format, which frankly is one of the best available codecs. That had to have been in the transfer somewhere.

 

I've been using Quicktime almost since it was released, and for years before ProRes came out, we had to use Uncompressed 10bit files, which were massive for HD. For 2k and 4k, they're just totally impractical. ProRes has been around for at least 10 years, I think, and in its HQ form, it's equivalent to 10bit uncompressed files, at roughly 1/10 the size. It became (past tense) the standard because of this and because playing it on both Mac and Windows has been possible for some time. There are indeed issues with some applications and how they handle things, but those are mostly a thing of the past. We have been using ProRes for years as our primary containerized media format, and it has vastly simplified and sped up work around here, without sacrificing quality. We are about 90% Windows here. 

 

I can see what you mean in that removing media layers from the equation makes the signal path simpler. But I have to disagree about the workflow argument. We do a ton of work with DPX. Our ScanStation can scan to this format directly, and our Northlight can only make DPX files. We use DPX in Resolve as our main media format, and we use DPX in our restoration system as well. But it's an utter nightmare to work with, especially with long-form projects. 

 

Just to give you an idea:

 

We are currently wrapping up a feature film project. We scanned it on the northlight at 4k, we graded it, we are doing the restoration right now. Deliverables to the client are the original log scans (DPX), the graded files (DPX), the graded & restored files (DPX), as well as about a dozen variations in Quicktime (HD, 4k, 23.98, 24, 25fps, a couple different aspect ratio mattes, etc). 

 

Each version of the film in DPX is just under 5TB. To copy one reel of the film to a hard drive (7200RPM SATA 6gbps disk in a dock) takes 2-3 hours. We then remove that drive and move it to the next workstation (say, from resolve to the restoration system). That then gets copied to the RAID in the restoration system. 2-3 more hours. Now we've spent an entire work day moving the files from one machine to another, before we get any work done. To do the entire film is a couple days of just copying files, renders aside. 

 

When we're done, one reels of film (as DPX) will fit on one LTO5 tape. I'm pretty sure we'll be able to fit all of the ProRes versions (10-12 files) on one, maybe two LTOs, depending on what they decide to order.

 

DPX is a wildly impractical format. The only way it works smoothly is if you have a massive and incredibly fast centralized storage system that uses something like 10-40GB Ethernet, Infiniband or FibreChannel. That's expensive stuff (though we're working on a cheap 20-40Gbps Infiniband SAN that I hope to get wrapped up in June). It's also clunky, because you really can't do this over standard networking protocols like SMB, which would be the easiest way to go. Instead you have to use iSCSI, which isn't complicated, but it means only one workstation can access the files at any given time, unless you've spent tens of thousands of dollars on an enterprise-level SAN with proper management software.

 

 

 


  • 0

#19 Anthony Schilling

Anthony Schilling
  • Basic Members
  • PipPipPipPip
  • 992 posts
  • Camera Operator
  • Portland, OR

Posted 26 April 2016 - 04:57 PM

Interesting info Perry- I didn't realize ProRes came in smaller packages. The file I got was 130GB for 400ft S8 uncompressed I presume, it was 4 years ago. But yes, large files are an issue for me, even though I have never gone above a TIFF. Im grading and editing 250ft S8 i scanned for someone now, 100GB proxy as a TIFF. Most of the files I deliver are from small volumes of film but the files are usually 60-100GB. But I was curious as how pro facilities handle transferring large volumes... which like you said, involves more expensive gear. 


  • 0

#20 Perry Paolantonio

Perry Paolantonio
  • Basic Members
  • PipPipPipPip
  • 347 posts
  • Other
  • Boston, MA

Posted 26 April 2016 - 06:23 PM

Interesting info Perry- I didn't realize ProRes came in smaller packages. The file I got was 130GB for 400ft S8 uncompressed I presume, it was 4 years ago. But yes, large files are an issue for me, even though I have never gone above a TIFF. Im grading and editing 250ft S8 i scanned for someone now, 100GB proxy as a TIFF. Most of the files I deliver are from small volumes of film but the files are usually 60-100GB. But I was curious as how pro facilities handle transferring large volumes... which like you said, involves more expensive gear. 

 

ProRes has several flavors, each with different levels of compression. They're all compressed to some degree. but then again, so are most (if not all) of the HD and SD tape formats, even the ones most people consider to be "uncompressed." 

 

TIFF is a nightmare to work with and I'm not aware of anyone using it regularly for editing or even grading at the professional level. It's always linear, so to get the full color bandwidth you have to work with 16bit TIFFs. There is no 10bit TIFF standard, only 8bit, which would be inadvisable because the vast majority of the color data would be missing. DPX is at least more widely supported by some editing systems, but even then you still have to have massive disk arrays to hold and move that much data. It's not a common format at all for editing, just for grading and restoration, and for VFX work. Few edit systems really handle it well. 

 

On the other hand, containerized formats like Quicktime, make it simple to keep audio and video together and in sync, and to work with files that are much more reasonable in size. 


  • 0


CineTape

Tai Audio

Ritter Battery

Paralinx LLC

Abel Cine

Broadcast Solutions Inc

Aerial Filmworks

The Slider

Zylight

Glidecam

Visual Products

Rig Wheels Passport

CineLab

Technodolly

Pro 8mm

Willys Widgets

rebotnix Technologies

rebotnix Technologies

Glidecam

Pro 8mm

Abel Cine

The Slider

CineLab

Rig Wheels Passport

CineTape

Willys Widgets

Paralinx LLC

Tai Audio

Zylight

Aerial Filmworks

Ritter Battery

Visual Products

Broadcast Solutions Inc

Technodolly