{"id":86279,"date":"2018-11-08T07:00:33","date_gmt":"2018-11-08T15:00:33","guid":{"rendered":"https:\/\/www.backblaze.com\/blog\/?p=86279"},"modified":"2022-04-04T12:23:59","modified_gmt":"2022-04-04T19:23:59","slug":"cloud-based-video-production-workflows","status":"publish","type":"post","link":"https:\/\/www.backblaze.com\/blog\/cloud-based-video-production-workflows\/","title":{"rendered":"Modern Storage Workflows in the Age of Cloud"},"content":{"rendered":"

\"Video<\/p>\n

This is part one of a two part series. You can find part two at Cloud-based Tools and AI Make Workflows More Powerful and Increase Content Value<\/a>.<\/div>\n

Not too long ago, hardware storage vendors held an iron grip on what kinds of storage underpinned your creative, film, and broadcast workflows. This storage took many complex forms \u2014 RAIDs, JBODs, SANs, NAS systems, tape robots, and more. All of it was expensive, deeply complex, and carried fat vendor margins and high support costs.<\/p>\n

But when you\u2019re considering storage in today\u2019s technology environment — whether it’s cloud, on-site storage, or a USB stick — the guiding principle in choosing storage for your creative production should simply be to choose the storage that best fits each workflow step.<\/strong><\/p>\n

Production Storage Maxim: Choose the storage that best fits each workflow step<\/em><\/strong><\/p>\n

Doing your best creative work is what builds your customer base, boosts your reputation, and earns you revenue and royalties. So any time sunk into legacy storage solutions, wrestling with complexity, unneeded production steps, refereeing competing vendors, and overpaying for, well, everything, just gets in the way of what you really want to do, create.<\/p>\n

The right answer for your specific production needs is a function of the size of your production team and the complexity of your operating environment. Whatever that answer is, it should be as frictionless an environment as possible that helps you get your work done more efficiently and gives you the most flexibility.<\/p>\n

An independent filmmaker can follow this production storage evaluation process for each stage of their workflow and decide to make do with a small deskside RAID system for primary production storage, and depend on the cloud for everything else.<\/p>\n

A large, global production team will probably need multiple SANs in each production office and a complex series of cloud and dedicated playout applications and systems. If your environment falls somewhere between those two extremes, then your ideal solution mix does as well.<\/p>\n

\"Traditional<\/a><\/p>\n

The traditional content production workflow is thought of as a linear process. Content is ingested<\/strong> as raw camera files pulled into a shared work-in-process<\/strong> storage for editors, the final cut is then delivered<\/strong> to the client, and when the project is finished all files are saved off to an archive<\/strong>.<\/em><\/p>\n

Simplified Production Workflow Steps<\/h2>\n

Let\u2019s look at what the storage requirements and needs are for each of the common steps in a production workflow and where cloud can add value. Along the way, we\u2019ll call out concrete examples of cloud capabilities at each stage with B2 cloud storage<\/a>.<\/p>\n

\"Ingest<\/a><\/p>\n

The Ingest Stage<\/h3>\n

Media copied in the ingest phase typically needs to get off of camera carts and flash drives as quickly and safely as possible and transported to the editing environment. Since those camera carts need to be used again for the next shot, pressure to get files copied over quickly (but safely) is intense.<\/p>\n

Any time that critical content exists only in one place is dangerous. At this stage, lost or corrupted files mean a reshoot, which may not be practical or even possible.<\/p>\n

Storage Needs for Ingest<\/h4>\n

Storage at the ingest stage can be very rudimentary and is often satisfied by just copying files from camera<\/a> carts to an external drive, then to another drive as a safety, or by putting a RAID system on a crash cart on-set. Every team tends to come up with a different solution.<\/p>\n

Where Cloud Can Add Value to Ingest<\/h4>\n

But even if your data wranglers aren\u2019t ready to give up external hard drives here, one way cloud can help in the ingest stage is to help combine your ingest and archive for safety steps.<\/p>\n

Instead of carrying carts from the shoot location to the production environment and copying them over to production storage, you could immediately start uploading content via the internet to your cloud storage, simultaneously copying over those files safely, and making them available to your entire team immediately.<\/p>\n

When you restructure your workflow like this, you\u2019ll get better than RAID-level protection for your content in the cloud.<\/a> And by checking content into your archive first, your asset manager tools can immediately start processing those files by adding tags and generating lighter weight proxies. As soon as the files hit cloud storage, your entire team can start working on them. They can immediately begin tagging and reviewing files, and even mark edit points before handing off to editors, thereby speeding up production dramatically.<\/p>\n

Some creatives have hit a roadblock in trying to take advantage of the cloud. Data transfer has historically been gated by the available upload bandwidth at your given location, but our customers have solved this in some interesting ways.<\/p>\n

Producers, editors, and reporters are finding that even cellular 4G internet connections make it feasible to immediately start uploading raw shots to their cloud storage. Others make it routine to stop off at a data center or affiliate with excellent upload speeds on their way in from the field.<\/p>\n

Either way, even novice shooters and freelancers can safely get content into your system quickly in a system that can be as simple as an upload bucket in your B2 account and making sure that your media or project manager tools are configured to watch those upload points.<\/p>\n

\n

Cloud Capability Example — Use a Backblaze Fireball to Rapidly Ingest Content<\/p>\n

Backblaze offers a Rapid Ingest Service<\/a> to help get large amounts of your content into your Backblaze account quickly. Backblaze ships you a 70TB storage system that you connect to your network and copy content to. When the system is shipped back to Backblaze, it is quickly moved directly into your B2 account, dramatically reducing ingest times.<\/p>\n<\/div>\n

\n

 <\/p>\n<\/div>\n

\n

Cloud Capability Example — Share Files Directly From Cloud<\/p>\n

\"Archive.zip<\/a><\/p>\n

An example of navigating to a file-review bucket in the B2 web interface to copy the direct sharing link to send to a reviewer<\/em><\/p>\n

In addition to the archive on ingest technique, many customers share files for approval review or dailies directly from their Backblaze B2 account\u2019s web interface.<\/p>\n

If your B2 bucket for finished files is public, you can get a direct share link from the Backblaze account management website and simply send that to your customer, thereby eliminating a copy step.<\/p>\n

You can even snapshot a folder of your content in B2<\/a>, and have Backblaze ship it directly<\/a> to your customer.<\/p>\n<\/div>\n

\"Work<\/a><\/p>\n

The Work-In-Process Stage<\/h3>\n

Work-in-process or primary production storage is the main storage used to support collaborative editing and production of content. The bulk of what\u2019s thought of as collaborative editing happens in this stage.<\/p>\n

For simplicity we\u2019re combining several steps under the umbrella of work-in-process such as craft editing, voiceover, sound, ADR, special effects, and even color grading and finish etc. under a far simpler work-in-process step.<\/p>\n

As audio, color grading and SFX steps get more complex, they sometimes need to be broken out into separate, extremely high performance storage such as more exotic (and expensive) flash-based storage that then feeds the result back to WIP storage.<\/em><\/strong><\/p>\n

Work-in-Process Stage Storage Needs<\/h4>\n

Storage performance requirements in this stage are extremely hard to meet, demanding the ability to serve multiple editors, each pulling multiple, extremely large streams of video files as they edit raw shots into a complex, visual story. Meeting this requirement usually requires either equipment intensive SAN, or a NAS that scales to eye-watering size and price.<\/p>\n

Many production environments have gotten in the habit of keeping older projects and media assets on the shared production environment alongside current production files, knowing that if those files are needed they can be retrieved quickly. But this also means that production storage fills up quickly, and it’s tempting to let more and more users not involved in primary production have access to those files as well, both of which can slow down production storage and creation of your content.<\/p>\n

Having to make a rush purchase to expand or add to your SAN is not fun, especially in the middle of a project, so regularly moving any files not needed for current production to your content archive is a great strategy to keep your production storage as light and small as possible so that it can last over several seasons.<\/p>\n

Where Cloud Can Add Value to Work-in-Process<\/h4>\n

By regularly moving content from your production storage you keep it light, fast, and simpler to manage. But that content still needs to be readily available. Cloud is an excellent choice here as content is both immediately available and stored on highly resilient object storage. In effect, you\u2019re lightening the burden on your primary storage, and using cloud as an always ready, expanding store for all of your content. We\u2019ll explore this concept more in the archive stage.<\/p>\n

\"Deliver<\/a><\/p>\n

The Deliver Stage<\/h3>\n

The deliver stage, where your finished work is handed off to your customer, varies depending on what type of creative you are. Broadcast customers will almost always need dedicated playout server appliances, and others will simply copy files to where they\u2019re needed by downstream customers, or upstream to a parent organization for distribution. But, at some level, we all have to deliver our work when it\u2019s done.<\/p>\n

Deliver Stage Storage Needs<\/h4>\n

Files for delivery should be moved off of your primary production storage and delivered in a separate workflow available to dedicated workflow or playout tools. Whatever the workflow, this storage needs to be extremely reliable and available for your customers whenever it is needed.<\/p>\n

Where Cloud Can Add Value to Deliver<\/h4>\n

Whether content delivery in your workflow is met by copying files to a playout server or giving a finished file to a customer, cloud can help cut down on the number of steps to get the content to its final destination while giving you extreme reliability.<\/p>\n

\n

Cloud Capability Example — Serve Time-Limited Links to Content<\/p>\n

Many customers use the Backblaze B2 API to add expiration limits that can last from seconds to a week to shared links:<\/p>\n

\"B2<\/a><\/p>\n

An example of using the B2 command-line tool<\/a> to generate time-expiring tokens for content sharing and delivery<\/em><\/p>\n

If your team is comfortable writing scripts to automate your workflow, this can be a powerful way to directly share files simply and quickly with tools provided by Backblaze.<\/p>\n

For more information see this B2 Article: Get Download Authorization<\/a><\/p>\n<\/div>\n

\n

 <\/p>\n<\/div>\n

\n

Cloud Capability Example — Move Content Directly to Your Delivery and Distribution Servers<\/p>\n

Serving your content to a wide audience via your website, content channel, or app is an increasingly popular way to deliver content. And thanks to our recent Cloudflare agreement<\/a>, you can now move content from your B2 storage<\/a> over to Cloudflare\u2019s content delivery network at zero transfer cost for your content application or website.For more information see this B2 article: How to Allow Cloudflare to Fetch Backblaze B2 Content<\/a><\/p>\n<\/div>\n

\"Archive<\/a><\/p>\n

The Archive Stage<\/h3>\n

At last, we come to the archive stage of content creation, traditionally thought of as the end of the traditional content creation chain, the source of the most frustration for creatives, and the hardest storage to size properly.<\/p>\n

Traditionally, when a project or season of a show is finished, all of the files used to create the content are moved off of expensive primary production storage and stored on separate, highly reliable storage in case they are needed again.<\/p>\n

Archive Stage Storage Needs<\/h4>\n

Archive storage<\/a> needs to be a safe repository for all of the content that you\u2019ve created. It should scale well at a sustainable price<\/a>, and make all archived content available immediately when requested by your users and workflow tools like asset managers.<\/p>\n

Tape was often chosen to store these archive files because it was cheaper than disk-based storage and offered good reliability. But choosing tape required a large investment in specialized tape systems, tape media, and the associated support contracts and maintenance.<\/p>\n

Tape based archiving strategies usually rely on compressing content as it’s written to tape to hit the advertised storage capacity of tape media. But video content is already stored in a compressed container, so compressing those files as they\u2019re written and retrieved from tape offers no advantage and only slows the process down.<\/p>\n

Here we find the chief drawback of tape based content archives for many customers: the time required to retrieve content from those tape systems. As the pace of production has increased, many customers find they can no longer wait for tape systems to return archive sets or unarchive files.<\/p>\n

Where Cloud Can Add Value to Archive<\/h4>\n

The archive stage is where cloud has the most impact on your entire workflow. The benefits of cloud itself are familiar: the ability to scale up or down instantly as your needs change, paying only for the storage you actually use, extremely high object storage file reliability, and availability anywhere there is a network connection.<\/p>\n

\"Modern<\/a><\/p>\n

\"\u2794<\/a><\/span>