How To Build A DIY 4K Raid..!

As many of you may already know or should know…4K is here to stay! That’s right…each year the past several we’ve been hearing and seeing more and more 4K buzz at big expos like CES and Nabshow especially. So far we’ve seen the 4K TV and Professional Monitors hit market each year but at a very high buy in price. Though more recently the hottest screen for the money is the the Seiki 4K going for around $1499(or less…I’ve seen them sell for as little as $1k on ebay). This has been one big issue with 4K still taking a long time to be embraced outside of the local Movie Theaters like Regal, AMC, or for me locally Cinetopia have all upgraded screens to 4K. This past Nabshow we also got another curve ball. Blackmagic Design introduced a 4K Cinema Camera to follow last years 2.5K for $3k camera. The exciting part is it’s only $4K for 4K(you can learn more about it on my Blackmagic Booth vid). The other issue have been how do I play it out. Well more recently various companies are offering you options. Nvidia have several offering that will output 4K, Matrox as well with their 4K Mojito Card(also check out my booth vid for Matrox with Dan discussing their new 4K card), and again Blackmagic with their New DeckLink 4K Extreme so as you can see you have options!

Unfortunately I can’t offer you any ideas to build your own 4K i/o card, camera, or monitor…but if you’re like me and are looking forward to a World In 4K…then one other major piece of the puzzle outside of a fairly beefy machine like my HP Z820 which is my Workstation of choice(you can also follow my journey switching from Mac to HP here). As of now the fastest machine for your A/V needs is the the HP Z820…only time will tell how the new MacPro performs next to a pimped out Z820(I would love to put it to the test Apple…you can learn more about the new 2013 MacPro here). So it’s pretty clear all the companies are bringing 4K hardware to market sooner than later-reality is…it’s already here). It really only comes down to money. There really isn’t a compromise. In time all tech gets better, faster, cheaper. I only expect to see more in the coming years!

So what does that all this above have to do with anything. Well in my blog post I like to not just give you one idea…I like to let you in on the variety of stuff you might need. One area I think can be easily forgotten is how you plan to store all this data, play it back, and have a backup. Well when it comes to 4K it’s a ton of Data. I myself own a Red and shoot everything in 4K. I outgrew my 8-bay iStoragePro iT8MIS. Nothing against their bigger 16 bay raids….I just hit a bottleneck in my workflow with my old 3GB/s Atto R380 and iStoragePro solution. I needed bigger, better, faster. I priced it around and iStoragePro had a nice enclosure but at this point in time out of my budget. I believe it sells for around $2k(by itself with no drives). If you don’t have the time to tackle a DIY raid I highly recommend their solutions. Another company that a lot of Red users use is Maxx Digital. They have a sweet soluiton but is pricy. I priced out a 32 TB solution at $12.5K I believe. Not all of us including me can afford that(at least not at this time). So I started looking into a DIY solution. Doing some basic searches I really didn’t find any info on creating such a raid. Since I’ve been up against this challenge and no good documented solution I figured this might be of great interest being that the transition to 4K is getting to be a lot more possible(this 4K do we need it or not debate is not what this post is about-it’s for users who want to or need to and need a bigger, better, faster raid they might be able to build themselves-but I assure you this applies to anyone really. This will work with any format smaller or bigger…it’s just storage…but not all storage can handle 4K so that’s why I’m sharing how you can build your own)!

So after a fair amount of research and some bad information I ended up building my own the hard way buying stuff, trying it, and buying different stuff cause the retailers didn’t exactly give me the correct info(won’t name any companies names). So I’ve taken the guess work out. I can’t give you an exact step by step but I’ll give you the basics. Since I am using an old SCSI enclosure we gutted out giving you an exact step my step to build the exact raid I built would be a bit pointless since it’s an old box and likely impossible to find the exact one. The good part is rack mount enclosures abide to a specific spec to fit the rack. So good news is likely any old rack will work. Just search ebay like I did and you’ll likely find one. All you need to look out for is that it’s tall enough to install the hot-sway boxes, is deep enough to house all the cable and SAS controller, and have a single or dual or even triple power supply like mine..! You can also just pick up a really large extended computer tower case…just be sure you can install all the hot-swap boxes in the front without any issues. Power supplies are fairly cheap and hard drives don’t require much power at all. Since I built a rack style raid I will focus on that scenario but I’m just mentioning it if you can’t find one or don’t want a rack style raid you can just use a computer case and install the same hardware in it.

First off I’m going to provide the Parts List for 4K DIY Raid:
(Note: * Reference pic of DIY 4K Raid – Inside View; pic below for detailed breakdown)!

  1. Qty 1 – Enclosure(I found one on Ebay and gutted it leaving just the Power Supplies)
  2. Qty 1 – Raid Card Areca 1882x external mini-sas hookup(they also have internal cards too)!
  3. Qty 2 – 1m SFF-8088 to SFF-8088 External SAS Cable(two cable for faster throughput) 
  4. Qty 1 – Astek A33606-PCI(past 8 typically need an expander depending on your Raid Card)
  5. Qty 1 – 5.25″ cage for Astek expander enclosure to mount above Astek card
  6. Qty 4 – 1m SFF-8087 (host) to 4x 7 pin SATA (device) Internal SAS/SATA Fanout Cable to connect AMS internal sata/sas ports to the internal 8087 miniSAS Astek ports
  7. Qty 3 – 3.5″ x 5″ AMS Backplane Module Model #DS-3151SSBK
  8. Qty 16 – Western Digital HD(WD1002FAEX 1TB 7200 RPM 64MB SATA 6.0Gb/s 3.5″)
  9. Qty 1 – Optional Kingwin SATA Mobile Rack(to use all 4 ports x 4 cables = 16 drives)

So lets get started..!

My SCSI Enclosure we gut to house all the new 6GB/s HW..!

 

 

First off we gut the SCSI Enclosure…here’s what it looked like(my pops makes a cut for Astek SAS Expander card that will mount into 5.25″ cage expander enclosure to mount card safely into chassis).
Astek cage and card is mounted in Raid Enclosure in below pic.

Internally we had to create a small aluminum riser plate so we had something to mount the Astek card and cage to(you could do this a bit more crude by just finding some small pieces of aluminum at a hardware store and glue it. To cut use a wood saw or get a blade that can cut metal, table saw, or band saw-basically use whatever you can get your hands on. worst case scenario use a some strong glue. Again this will differ greatly depending on what enclosure you find).

Cut small pieces of Aluminum as a spacer to mount the Astek Card/Cage into raid enclosure.
Here is a shot of the AMS Backplane Modules.

Pic below is a shot of the old SCSI backplane bottom plate we remove from old hot-sway modules(note the spacer/riser plate to mount to the new AMS Backplanes Modules(note bought this before Sata III-but they work. If you buy a new one it states Sata III, SAS, etc.)..!

Drilling new holes to mount the old scsi plate below to the new AMS Backplane Module(we used a drill press).
Method above was repeated for all three AMS Backplane Modules and we then drilled, tapped, and screwed them in from bottom and sides directly into the old scsi bottom plates.

 

Notice the single hot swap tray on the end. That is so I could use all 16 of the sata cables fan out cables. The hot swap Backplane Modules holds 5 so they can be installed in the standard bay openings typically in the front of PC box. Note we gutted everything in the old SCSI enclosure since we are drilling and little artifacts will creep into areas you don’t realize. So before evertyhing goes back into the newly housed enclosure it’s clean of all artifacts(you Do Not want something shorting out because you missed some metal flakes getting into sensitive areas of your new enclosure)!

 

 

Once you’ve gotten everything cleaned out, installed all the new hardware…you are now ready to start hooking everything up. Next step we hooked up all the internal wiring and hard drives that we screw into the hot swap trays. Once all the wiring is hooked up and hot swap drive trays populated then you can slide them into the front of your new DIY 4K Raid. Lastly you’ll need to install the Areca 1882x 6GB/s Raid Card. Always be sure to touch metal on the computer case before handling the cards and installing them. If you can get an anti-static mat or I have a wireless strapless anti-static wrist band(you just need to put it on about 15 minutes before you begin handling the hardware).
Now that you’ve got the DIY 4K Raid built, Raid card installed into you computer, and both 8088 Cables attached from the Raid card to your Astek(or Areca)SAS Expander your ready to boot up. Since I’ve switched from Mac to PC(HP Z820)I will be showing you how to configure it all in Windows. If your familiar with Mac OS and have every bought a raid or even a hard drive and had to initialize it in Disk Utility it’s very much the same process once your past the actual raid build setup. What I can’t show you here you can get the manual and just follow instructions. It’s not that difficult. Ok so let’s do this..!
Now it’s time to setup your raid(meaning configure and initial it)!
If everything is good to go then fire that puppy up(um ya this is your computer…but if you have a pup helpin ya out…give em a pat on the back while your at it;).
This will cover windows. It’s somewhat of a similar process on os x but you’ll have to explore that since I’m mainly on PC these days.
If you own an HP Z820 your likely to see the HP Logo and or a mini rom window in the upper right corner. Hit Tab(other pc manufactuers vary so hit the manual and study up on your bios-maybe your a builder and you’ve got it all up in the master mind then rock it).
So now you’ll be watching everything Post. When you see this screen in below pic hit either Tab or F6. This will take you into the ARC-1882 Areca Raid Bios.
You should now see a Bios screen like the one below.
Hit Enter to Select the Raid Controller to Configure.
You can just hit Quick Volume/Raid Setup but I recommend you manually configure your new raid! So hit the arrow key down to Raid Set Function. Now the Raid Set Function window pops up. Hit Create Raid Set. It will then open up some new configurations. I chose Raid 5(it’s I think the best for size, speed, and redundancy. I also chose a 4K block size. I believe I also set it to foreground initialization which should be faster)!
This is what it will look like with all your drives connected to the controller.
Once you see that the raid is building(there should be a percentage update). This time to go have a coffee, watch a movie whatever. Depending on the size of drives you use will determine how long it will take. Again I believe Foreground is faster but figure it’s going to take awhile. I can’t remember exactly how long but figure it’s not going to be an hour likely more like a few…and if you choose the other option and it seems like it’s going to take days then you could probably just delete the raid set by properly powering down of course first then going back into the Raid Configuration and set it to the opposite setting you tried. Once your raid has finished initializing then your ready to boot into windows. Once your past the login screen and see the windows desktop hit the windows key or click on the start menu. Click on the right and then right click on Computer and scroll down to Manage. This should open up Computer Management(you can also right click on computer anywhere you see the icon and get into this menu too btw). Once you’re in the window you should see something like the pic below. Choose Disk Management. You’ll likely see what looks like a black bar. Right click on that Volume(it should match the amount of terabytes all your drives combined). Choose Format. Once your in there the defaults should be fine. Just name your new Drive and typically you want to create a drive letter. I chose V for Video(most will just take whatever is already there and that’s perfectly fine). When your done you may still need to initialize the volume. I chose simple(it should go from a black bar to a blue bar once it’s complete-it will also give you a percentage update). When it’s done it will go from Offline to Online.
Click Start Menu>RT Click>Choose Manage
 You should now be in Computer Management>Click Disk Management
Find your new Raid Volume and Right Click on it. You should see a pop up window with options. Choose Format.
Once you see it online your ready to Rock…Enjoy..!
 
If your feeling ambitious then now it’s time to run a Speed Test. I’ll post a couple of examples that I got. Check it out..!
As you can see AJA is well over 1700 MB/s in Both Read and Write. Not bad..! Note if you are not happy with this speed or need more throughput you can always add more spindles, go SAS hard drives up to 15K rpm, SSD(though still a bit pricy doable if in your budget). If capacity if your issue you can go up to 24 with just the Astek Expander Card alone…it also has a two ports. Instead of running two cables in you just remove one from the card and plug it into another sas expander for more capacity. Pretty cool. If throughput drops with only one cable the solution is getting an Areca SAS Expander because it has three ports. So you can go two in and one out to the next expander.
As a reference only here are some ballpark figures for hardware pricing. Please do your own research to find the best source, price, availability. This should at least give you a good idea if building a DIY 4K Raid is a good choice for you financially.
  1. Enclosure(I bought a used enclosure on ebay with shipping):  $ 100
  2. Areca 1882x                                                                              $ 779
  3. 2-1m SFF-8088 to SFF-8088 External SAS Cable  $47 each  $   94
  4. Astek A33606-PCI SAS Expander Card                                   $ 415
  5. 5.25″ cage for Astek expander to mount above card                $   25
  6. 4-3ware CBL-SFF8087OCF-10M  Fanout Cables $15 each    $   60
  7. 3-AMS Backplane Modules Model #DS-3151SSBK $99 ea     $ 297
  8. 16-Western Digital HD WD1002FAEX 1TB         $99 each      $1594
  9. Kingwin SATA Mobile Rack                                                       $   19
  10. TOTAL…………………………………………………………………………..$3383
So that’s it…now you’ve got a plan to build your own big, fast, rebuildable Raid Storage for 4K Editing and save you some money DIY..!
I have to give thanks to my Pops who is a huge supporter and great help with all my wild ideas. This wouldn’t have been possible without all his expert help building this DIY 4K Raid..!
I also would like to thank my new friend Bill over at HDNLE.com He is a wealth of information, very knowledgeable and just so happens to be living up in the NW now after many years working with the Media and Entertainment Industry in SoCal. He is an expert with A/V Solutions. Anyway he was able to get me an Areca 1882x Raid Card and 6GB/s SAS Cables. I found him on Ebay and just my luck I was able to complete the transaction same day and get product…now that’s service..! If your in the NW I know no one with as much knowledge and experience as Bill in our field. I feel I’m very lucky to have connected with such talent in my area..!
Links to Bill @: HDNLE.com or his Ebay
Enjoy thanks for reading. If you have any suggestions, questions or comments feel free to leave feedback in the comment field below.
Cheers..!
Ryanb. Aka: Filmguy

 

Apple 2013 MacPro..!

So it’s finally here…the new Apple MacPro..! Wow they’ve done it again…a total redesign…it almost looks like a kitchen appliance. But all jokes aside it certainly boast some really interesting specs and concept..!

One thing I have to tip my hat to Apple is taking a new approach with their new MacPro. It’s just like them to change the game once again.
I won’t bore you with all the Specs…just here to offer some food for thought..!
link to apple macpro:
First off. What’s the deal with ATI GPU. I know it’s a constant back and forth between ATI and Nvidia. One year we are sold on Nvidia Cuda and the next it’s all about ATI. Frankly I’ve felt that Nvidia has been the most dominant in recent years and the likely choice. ATI seems a bit distant and little to no information or very much talk about how well it’s working for the Media and Entertainment industry. My guess is Apple is writing their code around these new GPUs. How that will effect the apps you use outside of Apple Pro Apps is something I guess we’ll just have to wait to be able to start benchmarking it..!
On the topic of GPU. This all in one solution is actually pretty interesting. My first gripe is the GPUs though actually. I feel that the bare minimum these days it’s 2. I’d love to see more like 4. So many apps, plugins, you name it is leaning on the GPUs. So why only one option. I feel like FCPX came on to change the game and it’s yet to really move users over to apple. In fact many have moved away from Apple to Avid or Adobe. In fact I just attended the AEPDX.org user group meeting with some of the folks at Adobe like Steve Forde(who also mentioned that Premiere Pro is now the #1 used NLE)and Paul President/CEO from Maxon. Adobe already utilizes the GPUs heavily(Maxon is looking into adding GPU but for now it’s all CPUs). It seems more and more important these days and to make us choose just one is not the best idea in my opinion. If this was redone I’d love to have a box similar to this but housed my GPUs(or mobile ones that could just slide in and out and be updated over time, allowed me to update the ram(with off the shelf options). The lack of hard drives is fine because I’d just have my root drive/sdd(or card of ram)and an external box for my Raid. Remember hard drives take very little power but GPUs on the other hand…a ton..! And they just keep getting bigger, better, faster, cheaper. 
I think something many don’t consider is how much it cost the manufacturer to adopt Thunderbolt..! I’ve overheard somewhere in the neighborhood of $50k. So what happens with TB v2.0? Is it an additional cost. Don’t forget all the new money you’ll need to spend on the new external hardware. I really never liked the idea of having to attach a host card and cubix expander for anywhere from $2-3k. Remember if your base machine already has internal expansion like my Z820 then that is just a cost I have to spend that I wouldn’t normally have to. So consider even a $1k external expansion box to hold the cards that would typically just plug and play. Are you following me. So if everything is now an external option…that means just that. I have to buy a bunch of new hard drives, raids, i/o, Gpu boxes. So take that into consideration. Basic math can probably give you a ball park number on what this new MacPro might cost…then add all the additional Thunderbold v1 & now v2.
As far as all the other amazing specs. I dig it…bring it…we love the speed and convenience…but just looked how iPhone and iDevices changed the game. So much so it’s has put us Media and Entertainment folks in my opinion on the back burner. My guess is this new box will be a welcome update…but there will be so many different ways to use it that v2 of this new design might need more updates to better suite all our needs. 
Again this is a totally new design and concept. I don’t doubt many other manufactures will adopt similar cause just think about it. It really simplies the system. It’s leaves all these 3rd party companies we must rely on now to build hopefully dependable expansion. Like anything this is all new. Time will only tell how it will all play out…will it completely change the game. Is apple doing us all a favor by doing a total redesign or are they opening up a can of worms..?!
All the i/o is great. I can see this lil box becoming the lil power house portable I could have runnin off battery power, generator etc…even edit out the back of my car. There will indeed be many possibilities. I look forward to seeing how many users attempt to hack this box to more suite their needs(if that is even possible…but trust me if it is they will:)!
For me this is not the machine for me..! I do think it’s innovative and cute…but I’m still down for a true workstaion. Shoot my power supply could easily be the size of this whole box..! That is something to be desired. Will this thing get hot, will it be unstable, or is it the revolution in size vs. cooling and so on.
I also love to hear more products being made in the USA(from apple.com):
Designed and Assembled in the USA..!
With the new Mac Pro, we’re assembling the entire product and machining several of its high-precision components in the United States. By leveraging the innovative power of industry-leading companies in Texas, Florida, Illinois, Kentucky, and over a dozen other states across America, we’re able to build a product that’s impeccably constructed and beautiful in every detail. In other words, exactly as it was envisioned by our designers and engineers in California.

I encourage your feedback and would love to here how you plan on using it..!

Thanks for reading..!

Ryanb. Aka Filmguy

 ps: noticed some people searching guess how much it will cost. off the top of my head I’d say in the neighborhood of $3500 and up based on config. if it’s less certainly welcome but they are packing some pretty impressive specs to say the least…that will cost you..!