The server has a Quad Core Xeon CPU E5440 and 4 GB of RAM. The storage needs are covered by a MD1000 disk array connected to a Perc5/E controller. The controller supports different RAID configurations: 0, 1, 10 and 50. I’ve tried most of them plus different combinations of software raid and hardware plus software raid. In the end I’ve settled on RAID-5 on 3 disks.

As the operating system I’ve installed CentOS 5.1 which is basically a free copy of RHEL 5.1. It comes with Xen 3.0.3 and of course I’ve installed that too. I want to convert the existing servers we have at work to Xen guests and have them running on this new machine. This will let me update OS on all the servers to the same version without bringing the whole office to halt. And once the servers are virtualized it will be much easier to deal with maintenance of the physical servers. Maybe I will even set up some cluster to host all these Xen guests. I didn’t have time to fully explore this idea but it sounds like it could offer further advantages.

So far I was able to convert one physical server to a virtual one with rather short downtime. I followed the instructions described in my Xen p2v post. The virtual server has been running fine for several months already. In terms of performance there are no problems – it helps that the Xen host has more powerful hardware and this server doesn’t need much I/O.

Besides this P2V conversion I’ve also created a Xen guest with Fedora Core 8 to run our CruiseControl server. The previous installation was running on a Fedora Core 5 host. I’ve used the occasion and upgraded CruiseControl to the latest version. Installing CruiseControl took me a long time. Mostly because I couldn’t find proper instructions. I’ve tried few different ones from the CruiseControl website, from IBM and some others. Unfortunately I just couldn’t get it to work. So in the end I’ve just duplicated the setup we had before. It’s not perfect but it works and I don’t want to spend a week on that.

Third Xen guest I’ve created is a very small installation of CentOS 5.1 running with only 64 MB of RAM. It serves as a license server for some applications we use. I’ve minimized the amount of the services running to reduce the memory footprint and the virtual machine uses only 41 MB of RAM.

So far all three guests are running nicely together. I’ll be adding more of them in the future.

Related post(s)

• Xen p2v conversion in seven simple steps (0)
• USB forwarding on Xen (0)
• Setting up UPS on CentOS 5.2 with SELinux, part 2 (1)
• Setting up UPS on CentOS 5.2 with SELinux, part 1 (0)
• What files did I change? (2)

These four RAID levels are examples of nested RAID levels. These RAID configurations are created by combining two simple RAID configurations like for example RAID-0 and RAID-1 into a more complex set-up which offers better performance and better redundancy than each of the single RAID levels. The general approach to creating a nested RAID is to divide the disks into sets of the same size. Each set is then used to create a RAID array. These arrays are usually called nested arrays. Then another RAID is created from the nested arrays. Usually nested RAID levels are created by combining two RAID levels which complement each other’s strengths and weaknesses. Note: regardless of which RAID level you are using remember that raid is not backup. For a short list of risks from which RAID cannot protect you see my previous post.

Let’s start with RAID-01. This RAID level is constructed from an even number of disks, usually four. It is set up by first creating two RAID-0 configurations and then a RAID-1 on top of them. The RAID-1 uses the RAID-0 nested arrays as its components instead of just plain disks. RAID-01 combines features of RAID-0 and RAID-1. The effective size of your storage in an N disk RAID-01 is going to be equal (N/2) times the size of the smallest disk in RAID. The read and write performance is two times that of RAID-1 created with 2 disks. If any disk fails in RAID-01 the whole RAID-0 nested array will be unavailable but your data will be still safe on the second RAID-0 nested array. RAID-01 can survive failure of the second disk from the same RAID-0 nested array but since it won’t be used at this point anyway this doesn’t really offer any additional redundancy compared with RAID-1. After any disk in RAID-01 fails the data is still available but you need to replace it as soon as possible otherwise when any disk in the other RAID-0 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-01 with four disks: A, B, C and D.
Disks A and B are combined into one RAID-0 nested array, disks C and D into another.
The whole RAID-01 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+B or C+D.
If A+C, A+D, B+C or B+D fail you lose your data.

RAID-10 uses a similar approach as RAID-01 but offers better redundancy. RAID-10 also requires an even number of disks, minimum four. It is set up by creating two RAID-1 configurations and then combining them with RAID-0. The RAID-0 uses the RAID-1 arrays as its components. The effective storage size is the same as for RAID-01 and is equal (N/2) times the size of the smallest disk. The read and write performance is the same as of RAID-01. The advantage of RAID-10 is that it can deal with two simultaneous disk failures as long as both failures are in different RAID-1 nested arrays. For a four disk RAID-10 array the probability that the second disk failure will be in a different RAID-1 pair than the first is 66.6% which is pretty good. After any disk in RAID-10 fails the data is still available but you need to replace it as soon as possible otherwise when another disk in the same RAID-1 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-10 with four disks: A, B, C and D.
Disks A and B are combined into one RAID-1 nested array, disks C and D into another.
The whole RAID-10 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+C, A+D, B+C or B+D.
If A+B or C+D fail you lose your data.

RAID-50 is similar to RAID-10 but instead of composing RAID-0 out of two RAID-1 arrays it uses RAID-5 arrays as its building blocks. RAID-50 requires an even number of disks, at least six. The effective storage size of RAID-50 is equal [(N/2)-1] times the size of the smallest disk. The read and write performance is two times that of RAID-5 created with half the amount of disks. RAID-50 can deal with two simultaneous disk failures (similarly as RAID-10) as long as the failed disks are in different RAID-5 nested arrays. For a six disk RAID-50 array the probability that the second disk failure will be in a different RAID-5 nested array than the first is 60%. After any disk in RAID-50 fails the data is still available but you need to replace it as soon as possible otherwise when another disk in the same RAID-5 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-50 with six disks: A, B, C, D, E and F.
Disks A, B and C are combined into one RAID-5 nested array, disks D, E and F into another.
The whole RAID-50 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+D, A+E, A+F, B+D, B+E, B+F, C+D, C+E or C+F.
If A+B, A+C, B+C, D+E, D+F, or E+F fail you lose your data.

RAID-51 is a very rare configuration and sometimes is called “RAID for very very paranoid”. It’s similar to RAID-01 but instead of building a RAID-1 out of RAID-0 arrays it uses RAID-5 arrays as its components. RAID-51 requires an even number of disks, at least six. The effective storage size of RAID-51 is the same as of RAID-50 and equal [(N/2)-1] times the size of the smallest disk. The read performance is equal to RAID-5 with N/2 disks – it can be double if the reads are using both RAID-1 mirrors in parallel. The write performance is equal to RAID-5 with N/2 disks and cannot be improved. RAID-51 can handle three simultaneous disk failures regardless of which disks failed. RAID-50 can survive even more simultaneous disk failures as long as at least one of the RAID-5 nested arrays has only one failed disk. After three disks in RAID-50 fail the data is still available but you need to replace at least one of them as soon as possible otherwise when another disk fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-51 with six disks: A, B, C, D, E and F.
Disks A, B and C are combined into one RAID-5 nested array, disks D, E and F into another.
The whole RAID-51 array can survive simultaneous failure of more than three disks without any data loss if at least two disks in either of the two nested arrays are OK.
If both nested arrays had two disk failures you would lose your data.

This concludes the RAID mini series. If you want to find out more about RAID I recommend the reference guide at storagereview.com website.

Any questions about RAID? Write a comment and I’ll get back to you.

Related post(s)

• Do you RAID? (part 1) (3)
• How to backup / download more than one Google doc at a time (part 3) (0)
• How to backup / download more than one Google doc at a time (part 2) (0)
• How to backup / download more than one Google doc at a time (part 1) (0)
• Xen and CentOS 5.1 on PowerEdge 2950 (2)

What is RAID? It’s an acronym meaning either “Redundant Array of Independent Disks” or “Redundant Array of Inexpensive Disks”. The two meanings are just something inherited from the past, many times both meanings apply. The basic idea is that you store the data on more than one disk and by using a few pretty cool ideas you can protect your data from problems like a failing disk and/or increase the performance of the storage system. An important thing to remember is that RAID is not a substitute for backup. While RAID can guarantee that you can still access all your data even if some of the disks are not working it won’t help if you accidentally delete the wrong file, your data gets wiped out by a virus, a power supply failure in your computer fries all your disks, a disgruntled employee deletes files on purpose or formats a disk, a hacker gets into your system, your disks get stolen, the building burns or gets flooded, a hostile country starts dropping bombs in your location, a meteorite strikes your home … do you get the point? To protect yourself from these situations you need a backup, preferably stored off-site. If the meteorite is very big you better keep your backups very far away, like on another planet. After all we know what happened to the dinosaurs, right? They obviously had no backups…

RAID can be setup in many different ways, these configurations are described as RAID levels. Most common RAID levels are RAID-0, RAID-1, RAID-5 and RAID-6. There are also nested RAID levels like RAID-10 or RAID-50 where multiple RAID setups are placed on top of each other to provide more functionality than each of the individual levels. Each RAID level can be implemented either using specialized hardware or software. Today I will talk about RAID levels 0, 1, 5 and 6. Nested RAID levels will be the topic of the following post.

Let’s start with RAID-0, which is not really RAID in a sense that it doesn’t provide any redundancy. What RAID-0 does is that it provides higher performance for the storage. In RAID-0 data is spread over multiple disks in so called stripes. Basically each file (unless it’s very small) is stored on all the disks in the storage. The advantage is that when you want to read the file the system can read each piece of the file from a different disk and because of that it can do this N times faster, where N is the number of disks in your RAID. The same happens when writing. The effective size of your storage is going to be equal N times size of the smallest disk in RAID. RAID-0 provides high performance but unfortunately it doesn’t provide any redundancy for your data, in fact RAID-0 makes it more likely that you’re going to lose data. If any disk in RAID-0 fails you’re going to lose all your files since each file had a piece on this failed disk. And since there are multiple disks in RAID-0 it is more likely that one of them is going to fail than if you had just one disk. After any disk in RAID-0 fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

A different approach is taken with RAID-1. This scheme fully deserves to be called RAID since it does provide redundancy for the data. In RAID-1 all data is stored on all the disks in RAID. Usually RAID-1 is used with 2 disks since this is most economical. You can use 3 or more disks in RAID-1 but if you really need this level of availability for your data then maybe you should look into other solutions like clustered storage. The biggest disadvantage of RAID-1 is the effective size of your storage is going to be equal to the size of the smallest disk in RAID. In terms of performance RAID-1 can provide an improvement when reading files (depending on the implementation different fragments of the file can be read from different disks since they all contain the same data) similar to RAID-0 but the write performance is the same as for a single disk (all data has to be written to each disk). When a disk fails in RAID-1 the data is still available but you need to replace it as soon as possible otherwise when the other disk fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Another common RAID scheme is called RAID-5. RAID-5 requires at least 3 disks to be used. It provides redundancy and also increased performance. In RAID-5 each data block is stored on one disk only but an additional information (called parity) for each block is stored on a different disk. The parity is a checksum which can be used by the system to recover data in case of disk failure. The parity information is spread over all the disks. This is done in such a way that the system can tolerate the failure of any one disk and not lose any data. The effective size of storage with RAID-5 is equal to (N-1) times the size of the smallest disk. Read performance of RAID-5 can approach (N-1) times that of a single disk. Write performance is rather poor since each write requires updating not only the data but also the parity information. The usual method to improve the write performance is to include cache memory in the RAID controller. RAID-5 is best used for systems where on average the number of reads highly exceeds the number of writes. When a disk in RAID-5 fails the data is still available but you need to replace the failed disk as soon as possible since from that point any further disk loss will mean you’ll have to restore all your data from the backups. You do have backups, don’t you?

An improvement on RAID-5 in terms of data redundancy is RAID-6. The difference here is that in RAID-6 parity is calculated twice independently and is stored on two disks. RAID-6 requires at least 4 disks to be used. Similar as with RAID-5 the parity information is spread over all the disks. The advantage is that the system can now survive the failure of any two disks without losing any data. The disadvantage is that the effective size of the storage is reduced to (N-2) times the size of the smallest disk. Performance of RAID-6 is a bit lower than of RAID-5 due to additional parity data. RAID-6 makes more sense with larger arrays (N>=12) because of the increased probability of double simultaneous disk failure. When two disks in RAID-6 fail the data is still available but you need to replace at least one of the failed disks as soon as possible since any further disk failure will mean you’ll have to restore all your data from the backups. You do have backups, don’t you?

Next week I’ll talk about nested RAID levels: RAID-01, RAID-10, RAID-50 and RAID-51 in Do you RAID? (part 2).

Any questions about RAID? Write a comment and I’ll get back to you.

Related post(s)

• Do you RAID? (part 2) (0)
• How to backup / download more than one Google doc at a time (part 3) (0)
• How to backup / download more than one Google doc at a time (part 2) (0)
• How to backup / download more than one Google doc at a time (part 1) (0)
• Xen and CentOS 5.1 on PowerEdge 2950 (2)