Jump to content


Photo

Storage in a digital climate


  • Please log in to reply
8 replies to this topic

#1 Marco King

Marco King
  • Basic Members
  • PipPip
  • 11 posts
  • 1st Assistant Camera
  • South Africa

Posted 24 May 2011 - 09:20 AM

I recently encountered a problem with a certain hard drive onto which I was offloading footage for a series production. The drive, a Lacie Rugged, was supplied by the production company for offloading and storage during the post-production process and beyond. Now I knew about the reputation of these drives and was very skeptical about long term storage on them. However, this is what I was given and budget restraints didn't allow for anything more reliable such as a mirrored drive system with a dedicated power source.

I am an AC and relatively new at it. Turns out the drive crashed, well almost and luckily I managed to retrieve the data because I had picked up a problem with it mounting to a computer before it really did crash. So my ass was saved but still too close a call. Why didn't I have a second back up you might ask? Well I did, also onto a Lacie, but by some foul turn of fate that drive also crashed, just days after the first one, leaving me with one back-up and no other drives for another. I also used a data transfer program, Shotput Pro, as an assurance because it creates a text file of the offloading process that can be used to verify that nothing went wrong during that process.

The question that I would like to pose to everyone is; Is there now an industry standard method for storing data over long term periods? For instance a new series in production here in South Africa, The Wild, stores all of their media on the cards they originally shot on as well as onto dedicated drives. However with budget in mind it may prove to be too costly for most to purchase hundreds of p2, CF or whichever cards to be used as tapes originally would as storage mediums. Then there is the question of space. When an entire series, shot in HD, is being stored it means a LOT of space and if the drives are mirrored, even more cost. Are there any solutions to this.

I feel this is a relatively important discussion to have because with the few production companies I have worked with thus far none have allocated sufficient budget for long term or even reliable storage that will span the whole production process. Who does this cost fall upon, the production company, post production company? The client?

Any input would be great.
  • 0

#2 Adrian Sierkowski

Adrian Sierkowski
  • Sustaining Members
  • 7118 posts
  • Cinematographer
  • Los Angeles, Ca

Posted 24 May 2011 - 11:12 AM

For Long term, LTO tape drives are useful, and I'd say pretty standard; if expensive. Another option would be archiving to BluRay dual layer (50GB) and making multiple copies. I just started doing this for some footage for clients. Not perfect, but better than a hard drive IMHO. Sony also make proprietary Blu-Ray discs for their XDCam cameras which you can use for archiving.
For my own personal stuff, I have multiple drives @ home and I also have a web-plan with unlimited server space; then it's on them to keep it backed up. Problem is, it takes a long time to upload thing....
  • 0

#3 Brian Drysdale

Brian Drysdale
  • Basic Members
  • PipPipPipPip
  • 5070 posts
  • Cinematographer

Posted 24 May 2011 - 12:36 PM

It's usual to keep 3 copies of the digital material. Even on a low budget job they should be making multiple copies before deleting any camera memory cards for reuse.

Here's the Academy report on the subject.

http://www.oscars.or...digitaldilemma/
  • 0

#4 John Sprung

John Sprung
  • Sustaining Members
  • 4635 posts
  • Other

Posted 24 May 2011 - 06:42 PM

What has worked well for us is to send the cards straight from the camera to the post house. They dump them over to their SAN, and then back up on LTO. They take responsibility for having a verified backup before the cards are returned to production.

Just like in the film days, you get it in the can, the lab does the rest.




-- J.S.
  • 0

#5 Will Montgomery

Will Montgomery
  • Sustaining Members
  • 2030 posts
  • Producer
  • Dallas, TX

Posted 25 May 2011 - 08:44 AM

I make 35mm film outs of everything. Just kidding. :rolleyes:

That really is the big question these days. With film you have that negative to go back to. It's almost like digital is a negative that you have no control over how it's stored; like it's dumb luck whether or not it will be there when you need it.

For any paying projects I like to make three external hard drive copies. One for the client and two backups that I hold on to. Plus if it is a current project and it's on my system, I use Time Machine to constantly backup everything to a 6 gig external drive. I've just been burned before on dead drives and don't want to relive it. If you think you're going overboard on redundancy, do more.

As a new AC you have the right idea about staying on top of that... one good save and you'll have a career for life.
  • 0

#6 Brian Drysdale

Brian Drysdale
  • Basic Members
  • PipPipPipPip
  • 5070 posts
  • Cinematographer

Posted 25 May 2011 - 10:22 AM

Good old tape is really useful on those one day jobs with a fly in producer and/or director when you can just hand over the rushes and they make a mad dash to the airport. You don't always have time to download and make back up copies. It's a weakness of the tapeless workflow, unless the producers bring their own memory cards.
  • 0

#7 Marco King

Marco King
  • Basic Members
  • PipPip
  • 11 posts
  • 1st Assistant Camera
  • South Africa

Posted 27 May 2011 - 06:07 AM

So I guess it all depends on what has been decided beforehand;

Cards straight to the edit suite can work but I think in that case it would still be wise to have one backup onto a drive as well. Obviously this option isn't always viable, in remote locations and so on.

Otherwise three backups seems to be the standard if you are storing on hard drive; One for the client and two that the production keeps. On that side of things, is it generally accepted that you charge for the storage of digital media? How does that get worked into the budget?


Brian. I'll be reading that academy report when I get time to see what conclusions they came to about that.

Also, When storing to drives, is there a reliable standard format that the footage should be stored as? I know there is great debate about standardizing digital media so that is does not become redundant but have there been any conclusions to this? Does one simply store the raw footage (tons of space req.) or a codec?
  • 0

#8 Will Montgomery

Will Montgomery
  • Sustaining Members
  • 2030 posts
  • Producer
  • Dallas, TX

Posted 27 May 2011 - 08:56 AM

Does one simply store the raw footage (tons of space req.) or a codec?

I like to think of the original file from a digital camera as a negative and treat it with the same respect.

More and more I'm seeing the ProRes 422 HQ codec as a standard for production I'm seeing. This is of course completely situationally dependent. At the very least it would be a safe codec to store footage in. ProRes 444 is giant and unnecessary for the work I do.

Film transfers generally come to me as ProRes 422 HQ, 1920x1080, 23.976 FPS and I like to work in that format unless the project dictates something else.
  • 0

#9 Marco King

Marco King
  • Basic Members
  • PipPip
  • 11 posts
  • 1st Assistant Camera
  • South Africa

Posted 02 June 2011 - 05:48 AM

Thanks for all the inputs. I think I have a better understanding of the process.

marco King
  • 0


Metropolis Post

Paralinx LLC

Technodolly

Glidecam

Broadcast Solutions Inc

rebotnix Technologies

Aerial Filmworks

CineTape

Opal

Media Blackout - Custom Cables and AKS

Rig Wheels Passport

Wooden Camera

Tai Audio

Willys Widgets

CineLab

Ritter Battery

Abel Cine

FJS International, LLC

Gamma Ray Digital Inc

The Slider

Visual Products

rebotnix Technologies

Paralinx LLC

Broadcast Solutions Inc

Ritter Battery

Aerial Filmworks

Visual Products

Metropolis Post

Media Blackout - Custom Cables and AKS

Glidecam

Wooden Camera

Abel Cine

FJS International, LLC

Willys Widgets

Gamma Ray Digital Inc

The Slider

Technodolly

CineTape

CineLab

Tai Audio

Rig Wheels Passport

Opal