Jump to content




Photo

Data Management - Backing Up Cards n Set

data 4k DTI AC

  • Please log in to reply
7 replies to this topic

#1 Ziggy Mueller

Ziggy Mueller

    New

  • Basic Members
  • Pip
  • 1 posts
  • Other
  • New Jersey

Posted 05 August 2016 - 09:35 AM

As an AC or DTI what is the best way you manage your captured data? Are there any standards or protocols? Is there a key device that is must have for backing up data?

I guess this bridges into on set workflow which I'm sure differs from set to set, but in general how does everyone handle this? It seems that the growing size of memory cards (and their falling prices) make shooting on fewer cards easier, but does any one run backups while on set?

 

The last film shoot I did was on film, the last digital shoot I did was still in the land of 7D and 1080p which was decently manageable. I've been out of the loop a bit and I'm curious going forward how I can prepare for a more modern workflow.

 

Thanks!,
Zig


Edited by Ziggy Mueller, 05 August 2016 - 09:38 AM.

  • 0




#2 aapo lettinen

aapo lettinen
  • Basic Members
  • PipPipPipPip
  • 699 posts
  • Other
  • Finland

Posted 05 August 2016 - 10:30 AM

depends on the production but normally on feature sets there is a DIT handling the copies and also helping with the monitoring LUTs, making dailies and offline versions on the fly, sometimes also making LTO backups, etc. 

It varies a lot depending on the specific movie but the very basic setup is a laptop, couple of external drives, card readers, UPS backup for all the setup, and a backup software which is able to do checksum verified file transfers, preferably one which can also automatically create good enough reports for prod department and post production. 

 

If making only the file transfers+dailies etc. and not on the fly LUT+live grading+color management work the DIT can usually arrive to the set later than the rest of the crew, in our productions usually about 3-4 hours so that he/she already has something to transfer when the DIT set is up and running. The DIT should always have a separate car and power supply so that it is possible to transfer+transcode the last card when the rest of the crew is wrapping, with mediocre setups this may take so long that the last car out of the set has to wait the DIT to finish so it is not ideal at all and the DIT needs to set up on another location every day after the shoot (hotel room etc.) 

 

One thing to note is that a screw up may cost anything from 20K upwards on pro sets (reshooting is expensive) so the DIT should be very careful and always triple check everything (probably will never be hired again if screwing up the material even one single time, even if it's not your fault) 

 

So a very basic setup: 

-laptop, usually macbookpro, or a "trash bin" MacPro with at least Thunderbolt and USB3 . some setups may need a separate thunderbolt-->PCIe box and add on cards like a SAS card 

-necessary software, for example Shotput Pro or Silverstack . for smaller setups, Resolve may be enough for quick grading tests and for making dailies and offlines, though it needs lots of GPU power and is a bit unreliable program. 

-separate hard drives for 3 physical copies (will be stored on different locations after the shooting day)

-card readers, preferably two or even three so that you can swap them if having problems or use two at the same time if having lots of material like high speed, multiple cameras, etc.

-fast RAID array /arrays if needed to reduce transfer times, especially for the first copy 

-extra monitor if needed 

-very good UPS to protect the setup from (always very frequent) power problems like over/under voltage, power failure, enables you to shut down the genny for couple of minutes for refill without affecting transfers, etc. 

-good small and silent generator +lots of gasoline (always have your own genny if doing time consuming transfers) 

-backup genny 

-a separate car for the DIT only so that you can have your own wrap time and don't have to hurry when everyone else is wrapping 

-backup laptop and one card reader +hard drive(s) (don't have to be your own, just make sure that there is another laptop existing on set at all times even if someone else is using it normally for other work)

so that if your setup fries, gets stolen etc. you can still transfer simple copies using for example simple Finder drag->drop 


  • 0

#3 Stuart Brereton

Stuart Brereton
  • Basic Members
  • PipPipPipPip
  • 2574 posts
  • Cinematographer
  • Los Angeles

Posted 05 August 2016 - 11:28 AM

Ultra low budget productions may have what's called a Data Wrangler, rather than a DIT. A wrangler's job is purely to copy and back-up cards. They don't do any color-timing or transcoding. Their setup can be as simple as a laptop, and back up drives. This was particularly common a few years back when transcoding was a long and laborious process, and Resolve was not yet available for free. As soft and hardware has become more powerful, more and more work is done on set, so the distinction has been blurred a little.


  • 0

#4 Phil Rhodes

Phil Rhodes
  • Sustaining Members
  • 11224 posts
  • Other

Posted 05 August 2016 - 11:50 AM

Transcoding can still be a long and laborious process, if someone decides it's necessary to have every resolution from 4K to QCIF with every LUT option under the sun...

 

Which is why it isn't a very good idea to do this stuff on Macs, unless there's absolutely no other option, softwarew-wise.


  • 0

#5 Stuart Brereton

Stuart Brereton
  • Basic Members
  • PipPipPipPip
  • 2574 posts
  • Cinematographer
  • Los Angeles

Posted 05 August 2016 - 12:02 PM

Transcoding can still be a long and laborious process, if someone decides it's necessary to have every resolution from 4K to QCIF with every LUT option under the sun...

In my experience, we simply transcode to Pro Res 422 HD, with either a 709 LUT and minor adjustments, or a custom LUT. The DIT is usually finished transcoding 30 minutes after wrap.


  • 0

#6 Phil Rhodes

Phil Rhodes
  • Sustaining Members
  • 11224 posts
  • Other

Posted 05 August 2016 - 12:05 PM

Ha, we're lucky if we're shooting ProRes 422!


  • 0

#7 Mark Kenfield

Mark Kenfield
  • Basic Members
  • PipPipPipPip
  • 712 posts
  • Cinematographer
  • Melbourne, Australia

Posted 05 August 2016 - 02:08 PM

This is why I miss shooting on my Sony F5/R5 package - gives you 4k 16-bit linear raw, with simultaneous recording of XAVC proxies with a burnt in LUT. Takes so much fuss and bother out of the process - just backup your cards and they're ready to go to editorial.

 

It's a really nice workflow.


  • 0

#8 Robin R Probyn

Robin R Probyn
  • Basic Members
  • PipPipPipPip
  • 1035 posts
  • Cinematographer
  • Tokyo

Posted 06 August 2016 - 05:15 AM

Hi Ziggy

 

Apart from obviously a decent laptop and decent HDD,s.. Shotput pro is the pretty much the standard software used.. its $99 well spent.. I believe a new version is coming out soon.. but you given a license number when you buy and can up grade for a small sum.. most productions now will insist on a hash check down load and a down load log for insurance purposes.. drag and drop is a big no no .. and in my experience .. usually 3 separate backups are made..  shotput is handy as you can download to multiple drives at the same time.. although it slows down a bit ofcourse.. 


  • 0



CineLab

Broadcast Solutions Inc

CineTape

Tai Audio

Paralinx LLC

Ritter Battery

Technodolly

Pro 8mm

Rig Wheels Passport

rebotnix Technologies

Glidecam

Willys Widgets

The Slider

Visual Products

Aerial Filmworks

Abel Cine

Media Blackout - Custom Cables and AKS

Zylight

CineTape

CineLab

Media Blackout - Custom Cables and AKS

Pro 8mm

Technodolly

Tai Audio

rebotnix Technologies

Zylight

Aerial Filmworks

Abel Cine

The Slider

Visual Products

Glidecam

Willys Widgets

Paralinx LLC

Rig Wheels Passport

Broadcast Solutions Inc

Ritter Battery