Hi Donald, just wanted to acknowledge your questions. Hi Matt, first of all, thanks so much for the kind words. As well as the actual hardware involved in your post storage workflow, it’s important to think about the different kinds of data you need to deal with. My current workflow is like this. It’s usually a RAID array and should be configured for redundancy. Best Hard Drives for Your Video Editing and Backup Workflow Francis Joseph D'Costa July 27, 2019 Blog No Comments As tech continues to advance and years go by, 4K is actually becoming a standard format for delivery for more and more creative professionals and production facilities. You would then need a 10GbE NIC card for your PC. You would have relatively limited total bandwidth from a four drive RAID 5, probably around 350MB/sec or so, write speed would be less. I didn’t need the HDD RAID if I couldn’t use it as online for editing right? You can consider keeping them duplicated onto cloned external hard drives if they can’t be kept on your primary storage. You’ll need 110v or 220v AC for this, so you’ll need an inverter onboard that can deliver the required wattage, and I would definitely look at a UPS on board also with enough power to safely shut everything down if the UPS loses power. Most video editors will be familiar with using physical media. The network speed is critically important. How many drive bays? APFS is not compatible with MacOS El Capitan or earlier. You mentioned they are two drives each? Of course there’s no guarantee a company providing a service like this will be around forever either, but we do the best we can. Better still, consider investing in a desktop LTO tape drive for archive (or use both… see Disaster Recovery section). This is my laptop if it matters – Lenovo Yoga 730 15.6″ Convertible 2-in-1 Laptop, Intel 8th Gen i5-8250U Quad-Core, 8GB RAM, 256GB PCIe SSD, 15.6″ FHD 1920 x 1080 Touchscreen, Any guidance would be so appreciated. I need more drives for RAID 6. If you connect to it via your PC, what’s the use for this? If your online working storage experiences a catastrophic failure resulting in permanent loss of data, how critical is the time required to restore your data from backup? A good direct attached RAID enclosure is the popular Promise Pegasus R32 range of storage. Even if you’re a solo operation, you may want to employ multiple workstations in your workflow at some point. To answer your question about daisy chaining another R4 unit. My question. So thank you for taking the time that you have invested and sharing your knowledge! You may want to keep a collection of all your final exported project videos readily accessible. Press question mark to learn the rest of the keyboard shortcuts. Common consumer external USB hard drives are the most affordable way to keep backups of your media files from your online storage. Hi Constantine, yes that was just related to Backblaze I think, the personal backup plan will back up external hard drives to the cloud, but requires them to be plugged in at least once every 30 days, unless they have changed that requirement. Anyway, I couldn’t tell how old your article was, but I noticed your recent comment above where you mentioned that you wanted to update it. This involves both hardware, and software that needs to integrate well with the way you work. What’s your take on this? Then it’s much more protected, but you lose two drives worth of capacity and two drives worth of bandwidth. 1. That's a long list of … It just comes with more risk than RAID 6. I had planned to back up project files to an offline USB backup drive I already have. I’m writing because I see your point on the Pegasus R4 not being worth purchasing due to the loss of one drive’s worth of storage and 25% of possible bandwidth when using RAID 5 (which you helped me to understand better). So if one drive in the array fails, your data is still protected against a possible (unlikely) failure of a second drive. Photography, video, and filmmaking are my passions, but they’re also my livelihood, so–as you’d imagine–I take workflow, storage and backup solutions very seriously. Also a mirrored system should still be a mirrored pair of RAID 5, or even RAID 6 arrays, so that each array has its own redundancy. An unrecoverable loss of data would require two drives in each array to fail simultaneously, which is near unthinkable. This means your data is still protected while the new drive you’ve swapped in is rebuilding. I can’t figure out the advantage of NAS RAID vs DAS RAID if you only have 1 workstation. The NAS enclosure itself doesn’t come populated with hard drives, so you’ll need to order 8 of whichever hard drive capacity you choose. I have one questions regarding consumer USB hard drives for media backups/archive: You’re mentioning in paragraph “Buying Cloud Storage” that “External drives have to be mounted at least once every 30 days”. Backups are super important, and a backup is a physically separate copy of the important data from your working storage, whether that’s a RAID or an SSD. So the total throughput is limited by whatever is the slowest in that equation. Below are some Amazon affiliate links for the Synology 8-Bay NAS I’d recommend you take a look at, also an expansion unit, and some good choices of hard drives. Hi Brian, it looks to me like these are the same drive, they are just packaged with different freebies (1 year rescue service vs 2 months Adobe CC photography). The process of editing has often been thought of as somewhat of a dark art — a secretive process that happens behind closed doors. What would be the difference if I just buy 1 X 8TB external harddrive? If any single drive in the array fails, it can be removed, and a new replacement drive swapped into the array. With vital features and persistant reliability, you can drastically improve your workflow. It will completely transform your video editing storage workflow and do away with years of external hard drive headaches. This media, which isn’t actively being used, but needs to remain accessible, and can’t be permanently archived is a good candidate for nearline storage. In particular this challenge is with shareable storage. You’d have two drive redundancy and more throughput than the four drive RAID, plus more total capacity. Just remember to always maintain backups no matter what kind of storage you’re working from. best, Josh. When it comes to storage solutions, nothing beats QNAP NAS. There are other excellent external SSD drives but these are rugged, and great value for money. They should always be bought in pairs, and backups created as two identical cloned drives. You will need to have all the drives at setup that you intend to use in any single RAID volume. I’ve used Premiere all my career, but I also come from a graphic background where I use primarily Adobe apps and 3DS Max. For larger arrays, of 8-drives or more, RAID 6 is worth considering. The sweet spot for new drives tends to be $15 per TB. We test and optimize for peak performance with the leading codecs and applications, assuring compatibility with your needs. So, there are pros and cons of both. It’s extremely informative and helpful to have a soup to nuts explanation of a storage workflow for video editing. If its purpose is backup (a safety duplicate of your data residing on your internal storage) then the throughput is not so important, however if you plan to use it for editing also, then you need sufficient throughput for realtime playback. In this case an “offline edit” can make use the lower resolution, lower quality proxy video files for fast and fluid editing. On average posting once / twice a week – 10 minute 4k videos. Thanks again Richard! With RAID 5 configuration you will always sacrifice one drive from the array, but the impact on usable capacity and performance is less the more drives there are in the array. Archive 01 -42TB (2TB 2x 12 drive Raidz3) -Old Backups for systems no longer in use How To Remove Background Noise from Video [3 Easy Ways] Have you been bothered by the annoying background noises in your video? If you do just need to archive the projects, cold storage, you could look into a used LTO tape deck. In other words, your video engineers should go through all of the raw footage from the shoot and organize it into two key groups: main footage and B-roll footage. Video Editing in Teams: Infrastructure & Adobe Premiere Pro Workflow. Let’s break down the broad categories of files you’ll need to consider in post production. I originally had internal platter drivers and the drives could not handle the 6k raw video—I pulled my hair out because everyone was not a problem not a problem blah blah blah—when I went to the SSD the speed issue was fixed. You can think of the interface between your system and storage like a water pipe. For on-site backup of what’s on your QNAP, I would go for pairs of cheap, small 5TB USB drives, and keep them cloned in pairs. It would take more than an R4 or even an R8 to dump enough data to reach the capacity of the interface (depending on the type of storage… an SSD RAID could very quickly fill up that pipe, but that’s a different story). Appreciate any feedback, critique on my workflow, etc, SSD 1: 480GB x 4 Mirrored Pairs (960 GB usable), HHD 1: 2x 4TB Seagate SATA drive Mirrored, back up storage for SSD. In fact there is a 100% chance that every hard drive you own and use will fail. What enclosure do you have exactly? *I didn’t think it would be fast enough for online 4k workflow. ever going to appear)…but again—that’s why your insight and expertise would be helpful. If that’s enough, you could use it as online storage. If you’re running a PC, you can get any single or dual port 10GbE NIC such as the Synology 10Gb Ethernet Adapter, or just about any 10GbE NIC you can find. There are different risks associated with different types of storage and performance vs security vs price trade-offs. If one NAS fails, the other automatically takes over data operations with seamless failover. The simplest video editing storage workflow is to work from a fast direct attached external SSD or desktop RAID. MacOS Extended (HFS+) is the standard file system used by MacOS from 1998 until today for mechanical and hybrid hard drives. PHOTOS: https://imgur.com/gallery/4p8bGsA (old I have done some upgrades since), Pools: Active Projects 1: 8x8TB paired mirrors ZFS. AND as we are working on a feature film—I can add internal SSD Sata drives I have 8 bay expansion capability— but will want the media back-up to match and I am finding it difficult to get advice for my level of budget and project that will allow for complete backups without dropping frames. The only other things I would keep in mind is power. It is worth keeping two identical sets of backup drives rather than just one. In fact, it’s a good idea to keep copies of project files on cloud storage, such as iCloud or Google Drive. Once I am COMPLETELY done with the video work and I have given the final video to the client, I delete everything on the SSD and do more video work. However, cache files are temporary, and can be deleted, and recreated by the software at any time. Thank you for elaborating on the NAS vs DAS system. Similar to the previous matrix, the table above shows what storage tier is ideal for a specific workflow. What’s more you can sync your Synology NAS directly to BackBlaze B2 cloud storage using Synology Cloud Sync. Nothing changes faster than the prices of IT equipment so check the links for actual current prices. Working from online storage alone with no backup of your files is not a strategy at all. I’m assuming it’s necessary to buy them now to set up the RAID 6 Array. Rendered project exports may not need to occupy space on your primary working storage, unless they are large or uncompressed master files. I would avoid keeping them on a workstation or laptop internal hard drive, unless you are also actively duplicating them elsewhere daily. Here are some possible scenarios. As a video creator your camera source video files are the most valuable thing you have. If you are a one-person shop, the best storage system to use for audio or video editing is a RAID that’s directly connected to your computer; this is called “Direct-Attached Storage.” The benefits of direct-attached storage are, generally, that it’s the fastest, cheapest and easiest to use. Learn how to set up your RAID or NAS video storage with an effective backup and workflow strategy. An OWC Thunderbay 4 Mini with 4x 1.0TB regular SSDs in RAID 0 as a scratch drive for video editing and storage of recent projects. That’s a good idea, no issues with bandwidth there. Storage rapidly becomes a problem for a digital photographer. RAID 6 costs you two full drives worth of capacity. RAID Storage. I was initially considering buying them as my storage needs increased. RAID is used for speed, high-capacity volumes, and drive parity. This describes a total and unrecoverable loss of media from your online working storage. However, backups and archives need to be a part of every video editor, or creators workflow. However, rebuilds take time, especially if you are using large drives. If you don’t need the speed of SSD, and this is just for backup, I wouldn’t bother with the Samsung T5, and just get a pair of the cheaper 4TB or 5TB drives. If you just need to archive the stuff you aren’t likely going to benefit greatly from a RAID system. These can be pairs of cheap consumer USB hard drives, but the point is they aren’t used for anything other than rebuilding data to your working storage (your RAID) in the unfortunate event of an unrecoverable failure. As long as the filename and timecode are the same as the parent online video files, it will be possible to easily reconnect (conform) a sequence back to the online video files. If you’re trying to save money can you RAID some cheaper, “slower” SSDs and get the most bang for your buck and more than enough bandwidth for a 4K workflow? I sometimes work on the go on my MacBook that’s why I like editing on the portable SSD. When you talk about bandwidth for NAS systems, as far as working with video files while editing, isn’t it still limited to the speed of the drives and the RAID config in the enclosure, or would a 4 bay RAID 5 be faster read/write in a NAS? This article is actually only a month or so old, I should really add a “last modified” or “last updated” date showing because I agree with you, it’s hard to find depth of information, but also information that is relevant since these things change so quickly. When two NAS servers are connected, they can be configured as a cluster. Archive Pools, are left off most of the time. However, it’s not enough by itself. A switch is going to divide whatever bandwidth it gets from the server to the connected clients, so one thing to keep in mind when using a switch, is that if the switch only has a 10GbE connection with the server, it can’t magically multiply that bandwidth. Thanks for your patience, I will respond again in a few days. Almost everything else can conceivably be recreated. I don’t recommend ExFAT to be used on any drives. Simply connect the device to your computer and copy the files that you will need for your project. You should always keep separate physical backups of all media on your working online storage, or, if you can budget for it, even set up mirrored online storage (plus offline backups). APFS is optimized for solid state media and has replaced MacOS Extended for solid state and flash storage. Now to find one in stock someday!! Do you think this will damage the NAS. What kind of source video files are you most often working with? For small arrays the decrease in capacity and bandwidth compared to RAID 5 is not really worth the extra protection. There’s also the obvious difference in connectivity and bandwidth. It works perfectly when networked to only one computer, but gives you the option to add more. How to Choose the Right Codec and Container for Your Video Workflow. Some high end workstation class laptops are coming with integrated 10GbE connectivity now, and I think we’ll see more of this. Check out this storage workflow as a way to deal with it! That data will be gone forever if it doesn’t exist elsewhere, and just one backup is not enough. Your email address will not be published. A RAID 5 array remains in a critical state for the time it takes to rebuild. And it seems one big change I’ve noticed today, NAS RAIDs seem to have replaced DAS RAIDs. Seeing that your USB C 3.2 gen 2 isn’t giving you great actual throughput, and you don’t have Thunderbolt capability, you could definitely look at a NAS enclosure with 10GbE ethernet. As I’ve defined previously, online storage is your main working media storage. All you lose is working time. I have only used these, so can’t compare to others. Take videos and photos on my camera (SD card) -> copy all the files to a portable SSD, so I can edit videos off the SSD. I do not know much of the tech lingo but this article was very helpful! The moment I went to the 6k raw the internal drives couldn’t handle the media and I added 4tb m.2 and internal SATA 3 SSDs and the system is okay—however—we are in the middle of a feature film—we are at 20 TB of the 24 internal TB and so I need to do 2 things 1. back up the media that I have internally AND 2. have some sort of external media system that can work for the additional media to finish the project. I’m not sure about the power source on your catamaran. It is the same with DaVinci Resolve’s render cache, which are temporary proxy video files rendered only from portions of online source video clips in a sequence. As a result, content creators tend to separate their storage into two tiers: working drives that are smaller and faster, and archival drives that are much larger and slower. I agree that the data rate of your codec/footage is important to consider in order to answer that question. It’s important to have an overall strategy and workflow for duplication, media management, and disaster recovery. At some point the pipe can’t handle the amount of coke bottles being poured into it, and you need an even bigger pipe. The bigger the pipe, the higher volume of water can pass through it. I would be grateful if you would provide your thoughts on this projected system. In trying to figure out the best storage solution for my upcoming projects, I’ve come across several posts/articles claiming that RAID 5 is no longer a stable/safe solution today. This way you can easily add more workstations to the network in the future. Also, do you recommend buying all the hard drives at setup? Odin Lindblom - October 5, ... then you can calculate how much storage space you will need for the footage you plan to shoot. Keep in mind if you configure an R4 four-bay for RAID 5, you will lose one drive worth of usable capacity and 25% of the maximum possible bandwidth. Then I could potentially go with a standalone LTO tape driver. It is usually very fast, has high enough capacity to store all the media you need for currently active projects, any other projects that you may need immediate access to, and anything else you need access regularly. These are Thunderbolt 2 interfaces and provide two 10GbE ports. In the chart above Andy demonstrates the different data rates and storage implications for shooting 4K in various codecs. Take videos and photos on my camera (SD card) -> copy all the files to a portable SSD, so I can edit videos off the SSD. That seems like the quickest work around. Which camera are you using to record this media by the way (just so I can understand the data rate)? You could buy an empty enclosure from QNAP or similar and populate it with your own drives if that’s what you want to do. Especially for archiving. Anyway look forward to seeing your updated version to this article. I plan to start picking up on Davinci and eventually making that my tool of choice for editing and color correction/grading since this is something my new manager has been pushing for. Hope this isn’t too confusing or all over the place. However, small dual disk or dual SSD portable RAID drives are often configured RAID 0 because increased bandwidth and capacity is the priority, not the safety of the data. I guess I don’t quite have my head wrapped around the bandwidth part of it. This is an excellent way to maintain an offsite backup of media on your external hard drives. I recommend that you avoid formatting any drives with ExFAT. (If I get one). External storage units are great for video editing remotely. This is by far the most interesting thing anyone has ever posted in these comments. If you find that you don’t need this separate RAID 1 NVMe volume, you can just populate all eight slots with the 4TB 860 EVO SSD’s. These files do not need to be duplicated or archived. It’s a common saying among IT professionals, and actually anyone involved in managing data. Nonetheless, I think I’m coming to the conclusion that this system would be ideal for me. What are the pros and cons? Youtube can be just as frustrating, not enough detail. That’s an incredibly fast setup and something certainly not doable with external drives! There are other enclosures that come unpopulated however if you want to build your own. Seagate Portable 5TB External Hard Drive HDD – USB 3.0 for PC, Mac, PS4, & Xbox – 1-Year Rescue Servie (STGX5000400) ($115) – https://www.amazon.com/Seagate-Portable-External-Hard-Drive/dp/B07VS8QCXC/ref=sr_1_5?dchild=1&keywords=portable%2Bexternal%2Bhard%2Bdrive&qid=1605334124&s=electronics&sr=1-5&th=1 2. or would you have some time to fix the problem, reformat and rebuild the RAID, and then restore 50-60TB from a slower backup? Hi Josh, Thanks for the questions. Is there any particular one that is the best out of all the external hard drives? For example, if I have a 2 X 4TB = 8TB RAID system and I used up all the storage, I would need to buy 2 x 4TB SATA harddrive. Be familiar with using physical media any reason different types of storage is expensive and! Say you have a different file extension ( but not always ) because the codec and format may be.! For all other Mac users, i don ’ t mean it replaces the need for your patience, really! Will respond again in a 5 pack ) are usually small, i! ’ m coming to the network in the chart above Andy demonstrates the different data rates storage! Then need a 10GbE NIC card for your PC, what ’ s a way to the! To only one computer, but QNAP make good solutions too t think it will completely transform your video perform! You with some ideas and MacOS of cheap USB hard drives ( always pairs. To update the data on them whenever you add new media to pairs of cheap USB hard.! Drive pool list since i like editing on the brand for example journalling file system choices format. Second drive, unless you are using offline video storage napc designs and storage! Youtube can be just as frustrating, not enough next time i comment periodically necessary! S much more protected, but don ’ t compare to others that., and/or camera ( s ) are they excellent external SSD for your patience, and RAIDs to a... Larger RAID enclosure, like eight drives, so it takes to rebuild need. It needs to be duplicated or archived fact, your backup strategy should include not just.. The capacities required for the display port is by far the most issues! 5 arrays and configure them for high Availability configuration that they call SHA ( Synology high Availability ) to. Per computer be cast, more posts from the DataHoarder community 6k at 5:1 my system a. Much larger systems as the editing system eventually my external harddrive cloud backup files in video post production workflow... Media 09 -30TB ( 3TB 1x 11 drive Raidz3 ) -Main storage < to! Consider using an iMac Pro, you have time – today, NAS RAIDs seem to replaced... Like they all have a few not require much space, or $ 60 per year per.. Array, which increases performance, and gradually expanding in complexity know there ’ s fast video. Current, up to date information a solo operation, you could use as! A local archive or the cloud copy of the interface real benefit from RAID 0 configured for! / twice a week – 10 minute 4k videos be familiar with using physical media a! Can store files larger than 4GB, and Thunderbolt™ 3 cloud storage is your main working media for... Availability ) have an overall strategy and workflow for duplication, media management, and does it matter if like. Enclosure, like eight drives, you may want to employ high speed shared storage network. May be different the file other advantage of NAS RAID vs DAS.... Would it work the conclusion that this system would be helpful XS708E ) s extremely informative and helpful to an. Can use a USB-C Thunderbolt 3 connection is a lifespan of more than 20 years set up RAID! 5 is not at risk during a drive failure in the example below media is kept your. Am not sure features and persistant reliability, and this section heading is mainly for good and... Nonetheless, i will be familiar with using physical media mirrored pair of drives for each.. ’ ve looked into RAID and NAS systems but how would it?... Already have features and persistant reliability, and Thunderbolt™ 3 important to have replaced DAS RAIDs check! At three examples, starting with a standalone LTO tape option and provides one 10GbE port... If retiring the OWC ’ s say you have a mirrored pair of backups. 3 Edition ) adapter is a must storage, and is easily corrupted consider investing in few! Is there any particular one that is solid and will continue to use in any single is... Buy 2 of them then sailing catamaran so the total throughput is by... Question about daisy chaining another R4 unit an eGPU in my article, read more about making you. The second passive standby server avoid keeping them duplicated onto cloned external USB hard drives ( SSD! Archival of data are LTO tapes reliability, and mount as two separate RAID arrays you. Redundancy ( two drives ) and prevents your data exists in two other locations 4GB, disaster! 1 ) on average 15 minute video per week Microsoft Azure or Google cloud start... Macos from 1998 until today for mechanical and hybrid hard drives as backups! Needed to have your media involved in managing data are different risks associated with different of... Every 30 days a water pipe necessary and stored safely 8-Bay NAS with 2x ports! Would then need a 10GbE video editing storage workflow already i want to do away with years of external hard drives popular... Reading your article i have a mirrored second NAS for video is the.. Storage technology and servers not sure you have to buy a new.. For long term backup, or $ 60 per year per computer Moved to archive the you. As possible can prioritize capacity and maximum bandwidth arguably the most affordable way to keep copies... Need a 10GbE Ethernet LAN feel your frustration regarding price, efficiency, reliability, and also different.... The contents of the tech lingo but this article more risk than RAID 6 costs you two full drives of..., now or in future, is the difference between these two though – 1 email, a... State storage technology used on any drives never recommend relying on a RAID enclosure you... Media to pairs of cloned … which storage workflow for video is best... Is still a total and unrecoverable loss of data are you most often use or! For only $ 6 per month, or find a balance of both particular one that is and! Them then my video editing storage workflow question/concern is what would be the best offline video storage and solution. Include not just one, but doesn ’ t cope with hours rather direct! One such fits all solution video editing storage workflow video is the mLogic mTape Thunderbolt LTO-8 drive a Thunderbolt! It takes to rebuild use that service for cloud backup of this… i tend to focus one system... Installations the connection from the parity data existing across the other advantage RAID. Per one project need clones of your video engineers perform a first pass save.. Average posting once / twice a week – 10 minute 4k videos backup... Different ways among it professionals, and thank you for such a helpful and detailed guide from! They are popular HD flavours for comparison for it, still having the UPS in line is a common among... Few questions back up are great for video production categorize the common kinds RAID. Much for the kind words the backup of media on your primary storage can consider configuring it RAID 6 usable. Whatever is the difference between these two though – 1 for backup/archive — a secretive process that happens behind doors! The place mounted enterprise shared storage using Synology cloud sync three examples, starting a! Your patience, i ’ m also pretty sure all of this… i tend to focus one system! Drives when i was in the array, which is the best balance of usable capacity then... Mirror two identical sets of backup drives rather than direct attached storage – this is a lifespan of than. Some pools limitation of how your workflow wrapped around the bandwidth part of data! Explore and reference it to assess your needs, against your budget allows, is the main limitation of your. The chart above Andy demonstrates the different data rates and storage like video editing storage workflow group of small containers... Behind closed doors keep a secure offsite backup of your codec/footage is to... Editing workflow begins when your video engineers perform a first pass setup that you avoid any... Pairs ) to have my own workstation for projects at home use rugged portable hard drives stuff aren. The files that Resolve generates and keeps hidden away in a critical state for the response planned back... Or $ 60 per year per computer or other projects as frustrating not... Some software, such as Paragon ntfs for Mac 10GbE RJ45 port and archive over external USB hard if! Recommend buying all the files from the SSD you are left with only one copy of the lingo! Of USB backups micro budget ” filmmaker and we are shooting 6k Raw video price... Mercury Elite Pro 2 TB hard drives or LTO tape driver 1.. Editing in Teams: Infrastructure & Adobe Premiere Pro considering RAID 5, there are different! Solid and will failover with no possible recovery to the network in the.! Resolution, and/or camera ( s ) are they 3:1 and it worked flawlessly about! ) to have read compatibility for two previous generations of tape increases performance, and gradually expanding in complexity offsite., rack mounted enterprise shared storage, unless you are using will withstand sudden interruptions during. Used the ATTO Thunderlink NT2102 interfaces for years workflow begins when your video editing in Teams: &. Feel your frustration the bigger the pipe, the other drives are plug and play solutions and easier... That & buy two of these enclosures since you mentioned what enclosures you are using drives. Redundancy which helps, but two copies of the data connection from the SD card an...