Question : Problem: Building a Video Server - what RAID to use?

Hi,

We want to build a video storage file server, for a new video project we are facing about 100gb of data.

The goal is to have a unique way of accessing our data (now the files are spread among thousands of internal/external harddrives), so raid technology comes to mind.

the question now is, which raid to choose. we want to have (of course) protection against hardware failures, but the speed of accessing the files is slightly more important. the workload of the workstation should  be distributed to the server, meaning that they use the raid as a network drive, if the speed would be increased through the raid.
we own already 4x 1TB seagate, 7200U/min, 32mb cache, 8.5 ms, plus 1 maxtor 1TB, 7200U/min, 32mb cache.
we want to use debian etch 64bit as operating system. the server will have an amd athlon 64b 2,6ghz with 2gb ram if you need this information..

heres a link to the mainboard: http://tinyurl.com/c64ewk
link to chip: http://tinyurl.com/cqj6j6

information is in german, but you figure out the numbers ;)

could you please elaborate following questions:

- which raid suits us most in this situation
- what speed in reading/writing can we expect when facing 10x 10GB chunks of data, over a 100 eth/lan network (possible to upgrade to 1000mbps)
- is it necessary to use a pci raidcontroller (when using only 4 harddrives) over an onboard controller for increasing the speed
- how does the controller speed play a role in the speed measurement? if the bus of the mainboard is restricted to xyz, does it affect the speed at all?

thanks a lot for your time :)

Answer : Problem: Building a Video Server - what RAID to use?

This is not so simple to answer as it might seem.  One thing that is lacking here is a better explanation of what you are going to do with it and how it will be used.  If the array is only storing 100GB, but you mentioned a 10x10G?  Are 10 machines trying to write 10G each at the same time?  The more detail you can give for the use of the storage and network , the easier it will be to help with an explanation.

max for 100Mbps Ethernet is 12.5MBytes/sec and you can sustain 11+ pretty easily.
max for 1Gbps is 125MBytes/sec and this is more dependant on the source and destination systems, but 40-60MBytes/sec is pretty easy.  Saturating the with network with two machines at 120MBytes/sec can be pretty hard to sustain and *very* expensive if you want 10 machines to be able to do it simultaneously to one server...

Sequential versus random is very key here as well.  Every time the disk heads change tracks you are not able to write any data for 10-12 msec with those SATA drives, so a goal, espcially with SATA drives is to keep the heads from seeking all over.

Also with data sets this large, 1Gbps networking is 2X-5X faster if the files are large.

This is an excellent series covering network storage systems and the clients like XP and Vista performance.
http://www.smallnetbuilder.com/content/blogcategory/51/77/