Version 1.0 of Wine has been published today as well. Wine is a compatibility layer for running Windows programs directly in many Unixes, including Linux, FreeBSD, Mac OS X and Solaris.

Firefox team had announced the release date in advance and for some time the website from where you can download the new version of the web browser has been unavailable. Wine website was also down temporarily.

At this moment both websites seem to be up again but one wonders if this was the case of a bigger success than expected…

Related post(s)

• Automatic backup of Windows computers using Linux (0)
• Convert videos to audio (1)
• I can’t stop smiling… we got an Asus Eee PC! (2)
• USB in Xen - how to make it work (0)
• How to Xenify office servers (0)

My results:

ping: 26 ms

download: 5768 kb/s

upload: 538 kb/s

Random unrelated posts

• 10 weirdest things I’ve seen in… Jena (Germany) (part 1) (3)
• 10 weirdest things I’ve seen in… Brussels (Belgium) (part 2) (2)
• The challenge continues (2)
• USB forwarding on Xen - it just does not work (0)
• DVDs (2)

What do you need:

1. Linux, any distribution will do. Linux can be installed on the physical hardware or in a virtual machine (like for example VMware, Xen, VirtualBox or Virtual PC). It is important that you have access to a DVD drive. Otherwise you’ll have to first create an .iso image of the video DVD you want to convert. mplayer is also available for Windows so you could try the same process using Windows if you prefer.
2. mplayer. Installation of mplayer depends on the Linux distribution, some have it preinstalled, others require you to install it yourself. For most users the easiest option will be to find pre-built binaries in .rpm, .deb or other appropriate format and install them. I’m not going to go into details here, if you need directions drop me a comment.
3. DVD with the videos you want to convert. If your Linux installation can access a DVD drive you just need to insert the DVD disc, otherwise you need to create an .iso image.
4. Some program to convert .wav files into .mp3. Anything you use to convert CDs to mp3 can usually convert .wav files also.

Preparations:

1. Decide whether you want to convert the whole video or only some specific fragments.
2. Decide whether you want to convert the video into one audio file or maybe you prefer separate file for each chapter.
3. If you decided to do the conversion from an .iso image you need to create it.
4. Make sure you have enough disk space for the audio files. You need about 6 MB per one minute.

Conversion:

1. Create .wav files from your video using mplayer. The command to execute is:
   mplayer dvd://1 -chapter [first chapter]-[last chapter] -vc null -vo null -alang [language code] -ao pcm:waveheader:fast:file=[output file] -dvd-device [path to .iso file] -ss [start time in seconds] -endpos [duration in seconds]
   where:
   • dvd://1 usually is the correct option although sometimes (depends on the DVD) it will have to be another number instead of 1. The number corresponds to the DVD title on the disc; usually 1 is the main video.
   • [first chapter] - is the first video chapter to start the conversion from.
   • [last chapter] - is the last video chapter to finish the conversion with; in case your video has no chapters or you want to convert the whole video into a single audio file you should skip the whole -chapter [first chapter]-[last chapter] fragment of the command; in case you’re converting each chapter of the DVD to a separate audio file the [last chapter] must be the same as [first chapter] and you need to run this command for each chapter (with a different output file each time).
   • [language code] - code of the audio language, this is important if the video has audio track in multiple languages; for example English is en, Spanish is es, Hungarian is hu.
   • [output file] - the path to the output audio file including the .wav extension.
   • [path to .iso file] - the path to the source .iso file of the video; in case you are converting without first creating the .iso file you should skip the whole -dvd-device [path to .iso file] fragment of the command.
   • [start time in seconds] - this option can be used to specify how many seconds at the beginning of the video should be skipped; you should not use this option if you want to convert the whole video.
   • [duration in seconds] - this option can be used to specify how many seconds of the video to convert; you should not use this option if you want to convert the whole video. This option together with -ss is useful when you want to do a trial run and convert just a short fragment first to see if it works or if you need to skip some fragments at the beginning or at the end.
   Some examples:
   • Save audio of chapter 4 of the DVD disk as file chapter4.wav:
     mplayer dvd://1 -chapter 4-4 -vc null -vo null -alang en -ao pcm:waveheader:fast:file=chapter4.wav
   • The same but from an ISO image:
     mplayer dvd://1 -chapter 4-4 -vc null -vo null -alang en -ao pcm:waveheader:fast:file=chapter4.wav -dvd-device image.iso
   • Save audio of chapters 2 to 14 of the DVD disk as file chapters2-14.wav:
     mplayer dvd://1 -chapter 2-14 -vc null -vo null -alang en -ao pcm:waveheader:fast:file=chapters2-14.wav
   • Save the last 10 seconds of the first minute of chapter 3 from the DVD disk as file chapter3.wav, this time we want the Spanish soundtrack:
     mplayer dvd://1 -chapter 3-3 -vc null -vo null -alang es -ao pcm:waveheader:fast:file=chapter3.wav -ss 50 -endpos 10
2. Convert .wav files to mp3 format or any other you prefer using a program you like.

That’s it. Enjoy the new mp3s!

Related post(s)

• Too much success and popularity? (0)
• I can’t stop smiling… we got an Asus Eee PC! (2)
• USB in Xen - how to make it work (0)
• How to Xenify office servers (0)
• The attack of the inodes - how to find out your number (3)

The server has a Quad Core Xeon CPU E5440 and 4 GB of RAM. The storage needs are covered by a MD1000 disk array connected to a Perc5/E controller. The controller supports different RAID configurations: 0, 1, 10 and 50. I’ve tried most of them plus different combinations of software raid and hardware plus software raid. In the end I’ve settled on RAID-5 on 3 disks.

As the operating system I’ve installed CentOS 5.1 which is basically a free copy of RHEL 5.1. It comes with Xen 3.0.3 and of course I’ve installed that too. I want to convert the existing servers we have at work to Xen guests and have them running on this new machine. This will let me update OS on all the servers to the same version without bringing the whole office to halt. And once the servers are virtualized it will be much easier to deal with maintenance of the physical servers. Maybe I will even set up some cluster to host all these Xen guests. I didn’t have time to fully explore this idea but it sounds like it could offer further advantages.

So far I was able to convert one physical server to a virtual one with rather short downtime. I followed the instructions described in my Xen p2v post. The virtual server has been running fine for several months already. In terms of performance there are no problems - it helps that the Xen host has more powerful hardware and this server doesn’t need much I/O.

Besides this P2V conversion I’ve also created a Xen guest with Fedora Core 8 to run our CruiseControl server. The previous installation was running on a Fedora Core 5 host. I’ve used the occasion and upgraded CruiseControl to the latest version. Installing CruiseControl took me a long time. Mostly because I couldn’t find proper instructions. I’ve tried few different ones from the CruiseControl website, from IBM and some others. Unfortunately I just couldn’t get it to work. So in the end I’ve just duplicated the setup we had before. It’s not perfect but it works and I don’t want to spend a week on that.

Third Xen guest I’ve created is a very small installation of CentOS 5.1 running with only 64 MB of RAM. It serves as a license server for some applications we use. I’ve minimized the amount of the services running to reduce the memory footprint and the virtual machine uses only 41 MB of RAM.

So far all three guests are running nicely together. I’ll be adding more of them in the future.

Related post(s)

• Xen p2v conversion in seven simple steps (0)
• USB forwarding on Xen (0)
• USB in Xen - how to make it work (0)
• How to Xenify office servers (0)
• Do you RAID? (part 2) (0)

Right, what a girlie comment to say. I mean, it is a laptop, woman, not a puppy!

Yeah, so it has another positive thing besides being cute, it does not lick you!

This is a post about the tiny Asus laptops for all audiences. Check what is the first question that pops in your head when you hear someone say “I just bought myself an EEE” and you will know the group you belong to:

1. eee… eeeh… what?
2. wow! so, is it as cool as they say it is?
3. I see, so which one did you get? what are the full specs? Anything else I should know about that model?

The answers for the eee… eeeh… what? people:

This is an Asus Eee laptop.

Well, not any Eee laptop, but our own Eee PC Yup, the pen is also mine and it shows you how tiny the Eee is And it weighs less than a kilo and a minute after taking it out of the box, you can already start working (with OpenOffice.org or GoogleDocs or Paint…), surfing the net (Firefox is there, Skype, Pidgin -the Linux MSN equivalent-, and Thunderbird as well) or even playing solitaire or with a Mr. Potato. And why did we buy it you ask? Well, it will be a better traveling companion than my 3 kg laptop which makes me walk around airports worse than the Hunchback of Notre-Dame did. Oh and the first answer I gave my mother when she asked the same question is also true: because… they had it in the shop…

The answers for the wow! so, is it as cool as they say it is? people:

Yup! So far… Do you want the pros and the cons? Ok:

Pros:

1. Yes, it weighs less than 1 kg.
2. The 7′ screen although small, actually displays a lot more/better than I expected.

   

3. Setting it up is like no time. Seriously, I think it took us about 2 minutes to be at the screen where you can open Firefox already.
4. The kiddo-laptop connected to wireless without a glitch and we were able to download and install the newer version of skype (to be able to use the built-in camera) within a minute.
5. If yous mess up the system, you can restore the whole laptop back to factory settings in under two minutes. In jkOnTheRun they even have a video showing you how quick it is!
6. It boots in 30 seconds and powers off in 12! Yes, I timed both processes.

Cons:

1. The keyboard IS small. I have problems most of all with pressing the Shift button. Look at the following image:

   

   You can see how small the keyboard is, compared to a “normal-size” one. I have placed a 1 cent coin on the letter D on both keyboards so you can see the relative sizes better. Oh, the mouse is a small-mouse as well (which did not come with the Eee).

2. ZoltarStark thinks the display organization with the buttons in the main screen to open Firefox, Spreadsheets, etc is not very efficient if you’re a power user since you need to point and click instead of selecting items from a menu.

   

3. The laptop seems to be designed with Windows users in mind and it has the same folder architecture as Windows Explorer with My documents folder and so on set up by default. Even worse, though, is that it asks you to reboot the laptop after every software update, like Microsoft does. Then again, with a total of 40 seconds required per reboot, it’s not as bad as it could be.

The answers for the I see, so which one did you get? what are the full specs? Anything else I should know about that model? people:

We bought the Asus Eee PC 4G. For those of you that don’t know the difference between the Eee versions, check out this table. As a summary, all four models have a 7′ display and Linux pre-installed (with the number of software applications varying depending on the disk size), Ethernet, WLAN, built-in speaker and microphone, and weigh 0.92 kg. They all have three USB 2.0 ports and a MMC/SD (HC) card reader slot. Furthermore:

Update (17th April 2008): A review just came out on the “big” brother of the Asus Eee 7″ laptop, the Asus Eee PC 900 (as in 8.9″ ~ 9 inch screen). It seems it has the same CPU, but 1GB of RAM to start with, and 20GB of storage if you buy the version running Linux (there is Eee 900 version running Windows XP, for the same price, that has 12 GB of space instead). I am starting to save to get my ASUS 700 Eee a big brother

Related post(s)

• Too much success and popularity? (0)
• Convert videos to audio (1)
• USB in Xen - how to make it work (0)
• How to Xenify office servers (0)
• The attack of the inodes - how to find out your number (3)

What can you do now? One suggested approach is to use some kind of USB over IP solution. That does involve spending some money though. USB over IP devices are about €300, a well known example are products from Digi International. Unfortunately I didn’t find any device of this type which would work with Linux clients so this looks like a bad direction to go even if it was cheaper. Another solution mentioned is using a software version of USB over IP. Again, they mostly support Windows clients. I was able to find only one that includes a Linux client - USB Server. At this point it’s in beta so it could be unstable - better test it before you use it on a production system. It includes a kernel driver (released under GPL 2) and binary only programs for the client and the server. It’s not a perfect solution since you still don’t know what’s inside of these binaries.

There are also people working on a similar solution in USB/IP Project. According to them “It is still at a *very* early development stage.” and is not stable.

I’ll try out these options and report here my findings. In the meantime if someone has experiences with any of these two solutions please comment and let us know.

Related post(s)

• How to Xenify office servers (0)
• USB forwarding on Xen - it just does not work (0)
• USB forwarding on Xen (0)
• Too much success and popularity? (0)
• Convert videos to audio (1)

I think I should start with a redundant file system to be free from depending on the 100% availability of a file server. The redundant file system should possibly be clustered so extra redundancy can be provided for the services and load balancing where necessary.

I’m not sure how to go about it in an efficient way. I’d like to hear from anybody who’s done something like this before and is willing to share their experiences.

Related post(s)

• USB in Xen - how to make it work (0)
• Too much success and popularity? (0)
• Convert videos to audio (1)
• Xen and CentOS 5.1 on PowerEdge 2950 (0)
• I can’t stop smiling… we got an Asus Eee PC! (2)

When you upload pages to your site, you know how many files and directories there are. If you check your email, the number of emails will give you an additional inode count. As long as in your site you are only going to have files uploaded by you and email accounts, you won’t probably need to worry about inodes. The problem appears if you are planning on having a blog, a forum, a content-management system or any other software package. These packages all have a large quantity of files even just installed. And some are made in such a way as to generate many additional files when you start using them.

Example-case: a basic Moodle installation, with menus only in English and no courses, has 4,850 inodes approximately to start with. If you upload 30 courses, this number goes up to 11,000 inodes. And that’s without any students (real example taken from here). 30 courses are loads. Ok, another example: one Moodle platform with “only” 7 courses BUT 500 students has 20,000 inodes (example taken from here). The reason is that every time a student uploads an exercise, Moodle creates two inodes: one corresponding to the student’s folder inside the exercise folder, and one corresponding to the actual exercise. 5 exercises, 10 inodes. 500 students doing the 5 exercises, 5,000 inodes. And it keeps growing… Furthermore, each language pack has over 600 inodes. So if you want to support different languages, be aware of how the number of inodes is going to increase even more. So knowing this, it is wise not to have more than one Moodle installation in a single web hosting account if you want to have many students. But if you don’t know about inodes, you might get a web hosting package with loads of disk space and as many mysql databases as you want, install Moodle, b2evolution and say Joomla and collapse your server in no time.

I have not been able to find a list of the inodes all well-known packages have to start with. I think this information is important to be able to decide what hosting account to get or how to use your hosting accounts in a more efficient way (for example combine Moodle and a few small scale web pages in a server and a forum and a blog in another). Therefore I have set upon myself the task of compiling an inode count list as complete as possible. At the initial release of this post, the table is far from complete. I will be adding more entries to the table as I find out the number of inodes each installation comes with. If you find out any additional data, do let me know, and I’ll add it to the list. Also, if you’d like me to find out the inode count of any program I don’t have yet, drop me a note as well, and I’ll see what can I do.

Related post(s)

• Hostgator: pros and cons of their web hosting offer (13)
• The attack of the inodes - how to find out your number (3)
• The 5 stages of bloggers (3)
• Blogging strategies: length, frequency and topics (3)
• How to feed Google properly (2)

To find out how many inodes are there in a folder or web hosting account, there are three ways. Whether one or another are possible, or all three, will depend on your web hosting provider.

Option 1. Submit a ticket

This one is easy and only takes as long as your provider’s support takes. Just ask them for an inode report on your account. Hostgator complied within 4 hours, and that was for three different accounts.

Advantage of this method: little effort and you don’t have to learn any commands.
Disadvantage of this method: if you haven’t been warned of being above the inode limit, you might be calling their attention to your account unnecessarily. Furthermore, you depend on others to know the number of inodes.

Option 2. Via ssh

If your provider allows you to ssh to your account, then once you are connected, all you have to do is type the following:

find . -printf “%i\n” | sort -u | wc -l

Advantage: quick (it can take 20 seconds to count 10,000 inodes).
Disadvantage: not all web hosting providers allow you ssh access. In Hostgator, for example, you can ask for ssh access but you need to give them a copy of your photo id.

Option 3. Via ftp

This can’t be done using FileZilla or similar. From a Linux machine you must open a terminal and:

1. First make sure you have the curlftpfs package installed. Depending on your Linux distribution it can be as simple as running (as super-user)
   yum install curlftpfs
2. Then you have to create the directory where you want to mount the ftp connection to your hosting account.
   mkdir ftp-domain-folder
   where ftp-domain-folder is the name of the directory you chose.
3. After that you connect to your account.
   curlftpfs -o user=xxxx:yyyy ftp://domain-name.zzz/ ftp-domain-folder/
   where xxxx = user name and yyyy = password.
4. And you run this command to count the inodes.
   find ftp-domain-folder/ | sort -u | wc -l
5. Finally, unmount the folder (requires super-user priviledges):
   umount ftp-domain-folder/

Advantage: you can do it from your computer, quite easy, no need for ssh,…
Disadvantage: it takes quite a bit (over an hour to count 10,000 inodes) and you need to run the commands from a Linux computer.

To check the total number of inodes, if you don’t have ssh access, I would suggest using option 1. Once you know the total number, you can use option 3 to find out the inode count in folders within your hosting account. Each individual folder will not have as many inodes as the whole account so the command will take less time. For example:

find ftp-domain-folder/mail/ | sort -u | wc -l

where mail is the folder where all your email accounts and emails are, or:

find ftp-domain-folder/public_html/folder-name/ | sort -u | wc -l

to check the inodes inside a folder in your account. This way you can check if you have erased enough files if you were reaching the limit.

Related post(s)

• Hostgator: pros and cons of their web hosting offer (13)
• The return of the inodes - tell me your package and I’ll tell you your inodes (4)
• Too much success and popularity? (0)
• Convert videos to audio (1)
• I can’t stop smiling… we got an Asus Eee PC! (2)

The pros they advertise in their page are true. In their shared baby and swamp web hosting packages you do get what they say, including the most attractive features:

• unlimited domains (as in, in one account you can add as many domains as you want)
• 6,000 GB of bandwidth in the baby package and unlimited in the swamp one
• 600 GB of disk space in the baby package and 1,000 GB in the swamp one
• unlimited MySQL databases (important if you are going to install Wordpress, Drupal, Joomla or other packages)
• Fantastico: easy one-click installation of most of the known scripts/CMS
• …

And an advantage which I have tested myself on a dozen of occasions by now and has always been excellent: answers from their support quick (usually in less than 6 hours) and helpful.

But, these pros and perks of Hostgator have a hidden side to them, which you will see when I explain the cons I have “suffered”.

Now the cons, or fine print details that you should consider before getting one package or another or even deciding if Hostgator is good enough for you. Right now I know of three cons. They have appeared at different times and although they haven’t made me switch to another web hosting provider, they have made me frown at them for a few hours/days. As I said, they do mention them, but until you understand what they mean, they sound like unimportant, when they are not. I am going to describe the three cons and when should you take them into account and take appropriate measures.

1. “Users may not: Use 25% or more of system resources for longer then 90 seconds”

Translation: if you have many web pages or visitors or scripts, even if you have unlimited space, bandwidth, etc., you might use up more than 25% of the CPU, which would cause them to shut down your service. So far it only happened once when a cron job lost the pot: they automatically shut down my cron job facilities without any warning and without me realizing till two weeks later. I say lost the pot because I still don’t really understand what happened. After talking to their support, they enabled the option again.

2. “The use of more than 50,000 inodes on any shared account may potentially result in suspension.”

Translation: each file and directory counts as an inode. When I encountered this problem, less than a week ago, I searched for explanations of the problem in several forums and I found quite a few places where people were saying that 50,000 inodes is such a large number that surely people that fret over them are exaggerating. Then how come I got two of my three Hostgator accounts with quotas above the inode limit? and not above by a few inodes, I’m talking about a few tens of thousands in one and even a hundred thousand in another! With a week to reduce the amount of inodes within the allowed quota, I’ve had to do some research to understand it better. I mean, I had 50 GB of my 600 GB disk full. How come I had over 200,000 inodes? Two things to know regarding inodes: how to find out how many do you have and how many inodes do different programs/scripts generate. I am working on two follow-up posts, one with instructions on how to obtain the number of inodes of your account or folders and a second one with a table with the number of inodes that standard packages like Wordpress or Moodle have to start with. So the problem is, you might have quite a large amount of disk space, but you cannot have more than 50,000 files, which means 12 MB per file if you were to use the 600 GB. An image displayed on a web page is usually around 30 to 50 kB and a simple html page say around 100 kB… so the magnanimous disk space will be usually unachievable, due to other restrains, and a bit of a misleading offer. Then again, you could use the extra disk space for backups, if not for many web pages.

3. “Mail policy and limits: There is a 200 hourly email limit per domain. If you send over this amount in any hour, most of the e-mails will bounce back with a undeliverable error.”

Translation: Now this policy I did not find in the Terms and Service page. You have to go search for it in their Knowledge pages. Again, surely 200 emails per hour are enough? Think about it, if you have one email account, it is unlikely you’ll ever send 200 emails in an hour. But what if you create 10 accounts? That means that each user should not send more than 20 emails in an hour. Doable? sure… unless you are emailing your siblings, which means you send an email to say two people, they answer and you reply again to both of them and… you can only do it 10 times in an hour. Still ok? The Hostgator offer is “Unlimited mail accounts”… sure, but the moment you have a few users, you will reach the maximum number of emails allowed per hour a lot quicker. It seems Yahoo mail has a limit between 25 and 100 emails per hour, depending on the account and Googlemail 60 per hour. And Hostgator only gives you 200 although you could in theory create a hundred email accounts. By the way, each email counts as an inode! So if you have 20 email accounts with say 500 emails in their accounts each (you can make their inboxes as big as you want, so why not be generous?), that’s 10,000 inodes. But when this really matters is when you have some package installed like Moodle or any other which relies on emails to inform students/members of any news. Say you have a Moodle installation with one course and 5 forums. Say 9 students enrol and there is one tutor. If you want all the students and the tutor to receive an email when somebody comments, that’s 10 emails that will be sent. Say a student enters the course and comments on all 5 forums, one after the other: that’s 50 emails. If another student happens to participate in the forums in the same hour, that’ll be another 10 emails per comment. Add to that exercises (and emails sent to the tutor letting him/her know) and then offer a few courses and you can reach the 200 email limit very easily.

Update: just when I was about to publish this post I decided to contact Hostgator live chat and ask about this email limit. It seems that from quite recently the limit has been increased to 500 emails per hour and that it’s 500 emails per hour per domain, not per hosting service as I thought. So, for example, if in one hosting package you have 3 domains, each of them has a 500 email limit. That still means that if you have 10 email accounts under one domain, you have 500 emails per hour to be divided among those 10 emails. Furthermore, there is a 60 pop-check per hour per domain. This means that if you are checking your email using Thunderbird or MS Outlook, you can only do it 60 times per hour. More than enough. If you have 5 email accounts, each can only check it, 12 times in an hour. If you have 20 accounts, each person should maximum check for new emails every 20 minutes. And if a user has a quick finger, other users might not be able to check their emails in an hour. And you still have to remember that each email is an inode! You can check out the complete new mail policy of Hostgator here.

Some conclusions:

Hostgator’s offer is very good, but you should consider what packages are you going to need (Moodle, b2evolution, Xoops, WebCalendar…), how many emails do you think you’ll need to send per hour, how many domains you’ll want to host and their inodes sizes, etc., before deciding what web hosting you need. You might need a few web hosting accounts, in Hostgator or any other provider! Getting a few hosting accounts could still be cheaper than a semi-dedicated or dedicated server and if you take into account the limitations of the service I’ve described, it can help you choose the best hosting package and distribution of domains per hosting account for your project/situation.

To know how to obtain the inode count of your folders, check out my next post. And if you need any help on deciding what would be the best option for you, do comment, and I’ll be happy to help you out.

Related post(s)

• The return of the inodes - tell me your package and I’ll tell you your inodes (4)
• The attack of the inodes - how to find out your number (3)
• php.ini, symlink and moodle: I’m calling it the php.ini fiasco (0)
• Register_Globals Off and PHP4 (1)

These four RAID levels are examples of nested RAID levels. These RAID configurations are created by combining two simple RAID configurations like for example RAID-0 and RAID-1 into a more complex set-up which offers better performance and better redundancy than each of the single RAID levels. The general approach to creating a nested RAID is to divide the disks into sets of the same size. Each set is then used to create a RAID array. These arrays are usually called nested arrays. Then another RAID is created from the nested arrays. Usually nested RAID levels are created by combining two RAID levels which complement each other’s strengths and weaknesses. Note: regardless of which RAID level you are using remember that raid is not backup. For a short list of risks from which RAID cannot protect you see my previous post.

Let’s start with RAID-01. This RAID level is constructed from an even number of disks, usually four. It is set up by first creating two RAID-0 configurations and then a RAID-1 on top of them. The RAID-1 uses the RAID-0 nested arrays as its components instead of just plain disks. RAID-01 combines features of RAID-0 and RAID-1. The effective size of your storage in an N disk RAID-01 is going to be equal (N/2) times the size of the smallest disk in RAID. The read and write performance is two times that of RAID-1 created with 2 disks. If any disk fails in RAID-01 the whole RAID-0 nested array will be unavailable but your data will be still safe on the second RAID-0 nested array. RAID-01 can survive failure of the second disk from the same RAID-0 nested array but since it won’t be used at this point anyway this doesn’t really offer any additional redundancy compared with RAID-1. After any disk in RAID-01 fails the data is still available but you need to replace it as soon as possible otherwise when any disk in the other RAID-0 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-01 with four disks: A, B, C and D.
Disks A and B are combined into one RAID-0 nested array, disks C and D into another.
The whole RAID-01 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+B or C+D.
If A+C, A+D, B+C or B+D fail you lose your data.

RAID-10 uses a similar approach as RAID-01 but offers better redundancy. RAID-10 also requires an even number of disks, minimum four. It is set up by creating two RAID-1 configurations and then combining them with RAID-0. The RAID-0 uses the RAID-1 arrays as its components. The effective storage size is the same as for RAID-01 and is equal (N/2) times the size of the smallest disk. The read and write performance is the same as of RAID-01. The advantage of RAID-10 is that it can deal with two simultaneous disk failures as long as both failures are in different RAID-1 nested arrays. For a four disk RAID-10 array the probability that the second disk failure will be in a different RAID-1 pair than the first is 66.6% which is pretty good. After any disk in RAID-10 fails the data is still available but you need to replace it as soon as possible otherwise when another disk in the same RAID-1 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-10 with four disks: A, B, C and D.
Disks A and B are combined into one RAID-1 nested array, disks C and D into another.
The whole RAID-10 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+C, A+D, B+C or B+D.
If A+B or C+D fail you lose your data.

RAID-50 is similar to RAID-10 but instead of composing RAID-0 out of two RAID-1 arrays it uses RAID-5 arrays as its building blocks. RAID-50 requires an even number of disks, at least six. The effective storage size of RAID-50 is equal [(N/2)-1] times the size of the smallest disk. The read and write performance is two times that of RAID-5 created with half the amount of disks. RAID-50 can deal with two simultaneous disk failures (similarly as RAID-10) as long as the failed disks are in different RAID-5 nested arrays. For a six disk RAID-50 array the probability that the second disk failure will be in a different RAID-5 nested array than the first is 60%. After any disk in RAID-50 fails the data is still available but you need to replace it as soon as possible otherwise when another disk in the same RAID-5 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-50 with six disks: A, B, C, D, E and F.
Disks A, B and C are combined into one RAID-5 nested array, disks D, E and F into another.
The whole RAID-50 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+D, A+E, A+F, B+D, B+E, B+F, C+D, C+E or C+F.
If A+B, A+C, B+C, D+E, D+F, or E+F fail you lose your data.

RAID-51 is a very rare configuration and sometimes is called “RAID for very very paranoid”. It’s similar to RAID-01 but instead of building a RAID-1 out of RAID-0 arrays it uses RAID-5 arrays as its components. RAID-51 requires an even number of disks, at least six. The effective storage size of RAID-51 is the same as of RAID-50 and equal [(N/2)-1] times the size of the smallest disk. The read performance is equal to RAID-5 with N/2 disks - it can be double if the reads are using both RAID-1 mirrors in parallel. The write performance is equal to RAID-5 with N/2 disks and cannot be improved. RAID-51 can handle three simultaneous disk failures regardless of which disks failed. RAID-50 can survive even more simultaneous disk failures as long as at least one of the RAID-5 nested arrays has only one failed disk. After three disks in RAID-50 fail the data is still available but you need to replace at least one of them as soon as possible otherwise when another disk fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-51 with six disks: A, B, C, D, E and F.
Disks A, B and C are combined into one RAID-5 nested array, disks D, E and F into another.
The whole RAID-51 array can survive simultaneous failure of more than three disks without any data loss if at least two disks in either of the two nested arrays are OK.
If both nested arrays had two disk failures you would lose your data.

This concludes the RAID mini series. If you want to find out more about RAID I recommend the reference guide at storagereview.com website.

Any questions about RAID? Write a comment and I’ll get back to you.

Related post(s)

• Do you RAID? (part 1) (3)
• Xen and CentOS 5.1 on PowerEdge 2950 (0)
• Automatic backup of Windows computers using Linux (0)
• That’s how I’ve learned to like Jazz (2)

What is RAID? It’s an acronym meaning either “Redundant Array of Independent Disks” or “Redundant Array of Inexpensive Disks”. The two meanings are just something inherited from the past, many times both meanings apply. The basic idea is that you store the data on more than one disk and by using a few pretty cool ideas you can protect your data from problems like a failing disk and/or increase the performance of the storage system. An important thing to remember is that RAID is not a substitute for backup. While RAID can guarantee that you can still access all your data even if some of the disks are not working it won’t help if you accidentally delete the wrong file, your data gets wiped out by a virus, a power supply failure in your computer fries all your disks, a disgruntled employee deletes files on purpose or formats a disk, a hacker gets into your system, your disks get stolen, the building burns or gets flooded, a hostile country starts dropping bombs in your location, a meteorite strikes your home … do you get the point? To protect yourself from these situations you need a backup, preferably stored off-site. If the meteorite is very big you better keep your backups very far away, like on another planet. After all we know what happened to the dinosaurs, right? They obviously had no backups…

RAID can be setup in many different ways, these configurations are described as RAID levels. Most common RAID levels are RAID-0, RAID-1, RAID-5 and RAID-6. There are also nested RAID levels like RAID-10 or RAID-50 where multiple RAID setups are placed on top of each other to provide more functionality than each of the individual levels. Each RAID level can be implemented either using specialized hardware or software. Today I will talk about RAID levels 0, 1, 5 and 6. Nested RAID levels will be the topic of the following post.

Let’s start with RAID-0, which is not really RAID in a sense that it doesn’t provide any redundancy. What RAID-0 does is that it provides higher performance for the storage. In RAID-0 data is spread over multiple disks in so called stripes. Basically each file (unless it’s very small) is stored on all the disks in the storage. The advantage is that when you want to read the file the system can read each piece of the file from a different disk and because of that it can do this N times faster, where N is the number of disks in your RAID. The same happens when writing. The effective size of your storage is going to be equal N times size of the smallest disk in RAID. RAID-0 provides high performance but unfortunately it doesn’t provide any redundancy for your data, in fact RAID-0 makes it more likely that you’re going to lose data. If any disk in RAID-0 fails you’re going to lose all your files since each file had a piece on this failed disk. And since there are multiple disks in RAID-0 it is more likely that one of them is going to fail than if you had just one disk. After any disk in RAID-0 fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

A different approach is taken with RAID-1. This scheme fully deserves to be called RAID since it does provide redundancy for the data. In RAID-1 all data is stored on all the disks in RAID. Usually RAID-1 is used with 2 disks since this is most economical. You can use 3 or more disks in RAID-1 but if you really need this level of availability for your data then maybe you should look into other solutions like clustered storage. The biggest disadvantage of RAID-1 is the effective size of your storage is going to be equal to the size of the smallest disk in RAID. In terms of performance RAID-1 can provide an improvement when reading files (depending on the implementation different fragments of the file can be read from different disks since they all contain the same data) similar to RAID-0 but the write performance is the same as for a single disk (all data has to be written to each disk). When a disk fails in RAID-1 the data is still available but you need to replace it as soon as possible otherwise when the other disk fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Another common RAID scheme is called RAID-5. RAID-5 requires at least 3 disks to be used. It provides redundancy and also increased performance. In RAID-5 each data block is stored on one disk only but an additional information (called parity) for each block is stored on a different disk. The parity is a checksum which can be used by the system to recover data in case of disk failure. The parity information is spread over all the disks. This is done in such a way that the system can tolerate the failure of any one disk and not lose any data. The effective size of storage with RAID-5 is equal to (N-1) times the size of the smallest disk. Read performance of RAID-5 can approach (N-1) times that of a single disk. Write performance is rather poor since each write requires updating not only the data but also the parity information. The usual method to improve the write performance is to include cache memory in the RAID controller. RAID-5 is best used for systems where on average the number of reads highly exceeds the number of writes. When a disk in RAID-5 fails the data is still available but you need to replace the failed disk as soon as possible since from that point any further disk loss will mean you’ll have to restore all your data from the backups. You do have backups, don’t you?

An improvement on RAID-5 in terms of data redundancy is RAID-6. The difference here is that in RAID-6 parity is calculated twice independently and is stored on two disks. RAID-6 requires at least 4 disks to be used. Similar as with RAID-5 the parity information is spread over all the disks. The advantage is that the system can now survive the failure of any two disks without losing any data. The disadvantage is that the effective size of the storage is reduced to (N-2) times the size of the smallest disk. Performance of RAID-6 is a bit lower than of RAID-5 due to additional parity data. RAID-6 makes more sense with larger arrays (N>=12) because of the increased probability of double simultaneous disk failure. When two disks in RAID-6 fail the data is still available but you need to replace at least one of the failed disks as soon as possible since any further disk failure will mean you’ll have to restore all your data from the backups. You do have backups, don’t you?

Next week I’ll talk about nested RAID levels: RAID-01, RAID-10, RAID-50 and RAID-51 in Do you RAID? (part 2).

Any questions about RAID? Write a comment and I’ll get back to you.

Related post(s)

• Do you RAID? (part 2) (0)
• Xen and CentOS 5.1 on PowerEdge 2950 (0)
• Automatic backup of Windows computers using Linux (0)
• That’s how I’ve learned to like Jazz (2)

Usually the Windows PCs store the user profiles on the server. Backing the user profiles can then be done as part of backing up the whole server. You might have some other files though which are not part of the user profile. Backing up these files can be easily done using rsnapshot and autofs.

We’ll use autofs to setup file sharing between the Windows PCs and the Linux server. This is especially useful for computers which are not always connected to the network like laptops, although people do turn off their desktops as well from time to time. What autofs allows us to do is to activate file sharing only when it’s needed for the purpose of doing a backup.

Performing the actual backup will be done using rsnapshot. This tool, which I already mentioned in a previous post, allows space efficient backups which are stored incrementally while offering ease of access usually associated with full backups.

Before setting up autofs we need to create a network share on the Windows PC. Remember the user and the password since we’ll need them later on. In this article I’m going to assume the user name is backup and the password is secret but you should use something more difficult to guess depending on your security requirements. I’m also assuming the Windows domain is called company. On the Linux server you then need to make sure you’ve got autofs installed. Installation of autofs is not covered here but usually it’s as simple as executing yum install autofs. You need to edit /etc/auto.master file and add this line

/smb /etc/auto.windows --timeout=60

Then you create the file /etc/auto.windows containing the following, in a single line:

<share> -fstype=smbfs,
credentials=/etc/auto.windows.<credentials file>,
uid=<user>,gid=<group>,
fmask=0755,dmask=0755 ://<windows PC>/<network share>

This must be written on a single line, with no separating spaces starting from -fstype to dmask=0755

<share> is the name of directory where you want to mount the network share from Windows PC.

/etc/auto.windows.<credentials file> is the name of file where you store the user name and password needed to access the network share on Windows PC. In our case it will look like this:

username = backup
password = secret
domain = company


After these preparations it’s time to tell the autofs daemon about these changes by executing service autofs reload or a corresponding command. You should be able now to see that the network share gets automatically mounted when you execute ls /smb/<share>. After 60 seconds the share will be unmounted.

Now the last part - setting up rsnapshot. Just follow the instructions for rsnapshot as if the directory /smb/<share> was local. Thanks to autofs, rsnapshot doesn’t need to know that it’s really on a different computer running Windows. Remember to set up a cron job if this is your first time you’re using rsnapshot.

That’s it. You’ve configured an automatic backup of a Windows computer using Linux. You can repeat this procedure on further Windows clients.

P.S. Another approach is described here but it has the drawback that it requires installing additional software on the Windows PCs.

Related post(s)

• Too much success and popularity? (0)
• That’s how I’ve learned to like Jazz (2)
• Convert videos to audio (1)
• I can’t stop smiling… we got an Asus Eee PC! (2)
• USB in Xen - how to make it work (0)

First let me introduce the meaning of the title tags since while discussing the topic with LadyRostand we realized it’s not as clear as we thought. It all started in 1980 … - OK let’s not go back so far, even though the beginnings of HTML are really related to the matter at hand. The title tag is what your web browser displays in the title bar. It’s basically the title of the page. Big deal, you say, I knew that. Sure, it’s not a big deal, it’s rather simple. The big deal is that when Google, Yahoo, MSN or any other search engine is indexing your page they pay big attention to this title. Why? The title is supposed to be a summary of your post. It’s not supposed to be witty or funny. It’s just a summary.

So if you write a post about installing an external USB enclosure you better give it a good Google friendly title tag. Like for example: “Enermax EB307S-B Jazz: installing external USB enclosure” rather than “That’s how I’ve learned to like Jazz”. Now the second one is funnier but Google has no sense of humor (quite a big shortcoming if you ask me) and it interprets the title tags rather literally. After seeing the title tag it will “make up its mind” about what should be the content of the page. If the page content doesn’t match it will lower your ranking in the search engine results. Why? How would you like to see a page about penguins (or more likely porn if you’re using the same Internet as I do) when you were searching for rabbits? See, if the search engines didn’t compare the page content with the page title they would allow pages to be ranked high for results to which they are not relevant. I think I’m mumbling here a bit. Basically it’s a technique to prevent or at least reduce web spam.

Back to the “That’s how I’ve learned to like Jazz” post. So far you know that the title tag as seen by the search engines should be relevant to the content of the post. It should be even devoid of articles and include relevant keywords. Now it starts to sound more like a super concentrated abstract. In fact, that’s what it is. The problem is that this super concentrated soup of keywords is not very human friendly and believe me you are really writing for humans not for search engines.

OK, so is there a way to have nice human friendly post titles where you can show off your wit and at the same time allow for high keyword concentration in the title tag? Yes, it’s possible. In fact, if you’re using Wordpress for your blog it’s just one plug-in away. The plug-in we’re using here at olivetalks is called SEO Title Tag. Once you install it you can have a separate post title for human consumption and a title tag to feed the search engines.

Installing SEO Title Tag plug-in is rather simple. First download it from this location. Then you need to unzip the file and upload the whole seo-title-tag directory to the wp-content/plugins directory of your blog. After that you activate the plug-in using Wordpress admin. The default options are just fine so you don’t need to change them. Editing the Header in Theme Editor is also unnecessary even though it’s included in the plug-in installation instructions.

With SEO Title Tag plug-in already installed you can now see “Title Tags” under “Manage” in the Wordpress admin. Here’s where you can quickly mass update the title tags of your existing posts. You can also do it by editing individual posts - you’ll notice a new field “Title Tag (optional)” which appears below “Save” button. The field “Title Tag (optional)” will also appear when you edit a new post.

In this post you can see that the title of the post (for humans) is different than the title which appears in the title bar (for search engines). So now everybody is well fed… and happy.

Related post(s)

• How to write a great blog post? (0)
• The 5 stages of bloggers (3)
• Blogging strategies: length, frequency and topics (3)
• Today olivetalks celebrates three months (5)
• The return of the inodes - tell me your package and I’ll tell you your inodes (4)

I’ve recently performed two conversions of physical hosts to Xen guests running on CentOS 5.1 host. The physical hosts were running RHEL 3.7 and CentOS 4.6. It is important to know that after the conversion you end up with a fully virtualized guest - HVM. This has its effect on the performance, especially when it comes to I/O. It is a general limitation of fully virtualized Xen guests and not of this particular P2V method. I don’t recommend using it for virtualizing for example a file server. In such a case your only real choice is to create a new paravirtualized guest and migrate the services. This article won’t be describing that approach.

These instructions are based on the article by Kyle Rankin. I’ve adapted it for Xen and added comments on those things which did cause me trouble. The instructions have been used to virtualize RHEL and CentOS servers to run as Xen guests on CentOS 5.1 but you should be able to use most of them for other distributions.

First step in the conversion is to create LVM volume(s) in the Xen host to be used by the guest. To figure out how much space you need, have a look at the physical server you want to virtualize. Each LVM volume will be visible in the guest as a separate hard disk. Note that HVM guests support only up to four disks. You will partition the disks from inside the guest in step two.

Second step is to create a new HVM guest and boot it from a bootable rescue CD/DVD. You should use a rescue CD which matches the architecture of the physical host being virtualized. In step four you need to run chroot command on the guest and it won’t work if you try for example a 32 bit rescue CD on a 64 bit system. After you have already booted the guest from a rescue CD you need to create partitions to match the physical host. The easiest thing is to match exactly the number and sizes of partitions on the physical system. You can also put several file systems which were on separate partitions on the physical server on a single partition on the guest as long as you prepare enough space.

Third step is to copy files from the physical server to the guest. This can take some time, so depending on whether you want to minimize the downtime you can do it either in one step or in two. Both methods use rsync command. The second method is described by Kyle in his article. The first method is basically the same but you only do the final synchronization. If you use a single transfer method you don’t need to use the --delete parameter for the final (and in this case the only) synchronization. The single transfer method is faster but it might not be available for you if you need to minimize the downtime.

Fourth step is to adjust the boot settings in the guest. Without that the guest won’t boot. To do that make sure all the guest file systems are mounted and chroot into the guest root file system. From there you need to adjust the boot loader configuration. I use GRUB on my systems. For instructions on adjusting LILO you’ll have to look elsewhere.

First you need to change the file /boot/grub/device.map. If you see /dev/sda there you need to replace it with /dev/hda. This should represent your boot drive. Also verify that the guest boot partition is in /etc/mtab. After that you execute grub-install /dev/hda.

Another file which needs modifications is /etc/grub.conf. Pay attention to the lines starting with boot= and splashimage= as they probably need adjusting. The boot parameters need to be updated as appropriate. The important entries are root (hd0,0) and root= kernel parameter. Note that the first one selects the boot partition while the second selects the partition for the root file system. So for example if the boot partition is in /dev/hda3 and root partition is /dev/hda1 these entries should look root (hd0,2) (here the partitions are counted starting from zero) and root=/dev/hda1 (here they are counted from one).

Fifth step is to generate a new initrd file for the guest. First copy the existing initrd file somewhere in case you need it later. Then adjust the /etc/modules.conf or /etc/modprobe.conf file (only one will be present depending on the distribution and OS version). Remove all entries starting with alias scsi_hostadapter. After that you can run mkinitrd command. I recommend you use -v parameter for mkinitrd. This will list all the modules which will be included in the initrd. You will probably need at least libata and ata_piix - even if the physical server uses SCSI disks the HVM guest will use IDE. This depends on the OS of the physical machine you’re virtualizing because it was only important when I was converting the RHEL 3.7 server but not with CentOS 4.6. If they’re missing you can add them using --with= parameter. Another advice in case you need to verify the content of initrd file (probably because the guest doesn’t boot): take into account that older versions of mkinitrd produce just a gzipped ext2 filesystem while the newer ones create a gzipped cpio archive.

Sixth step is to adjust the network settings and mount points for the guest. The configuration files you need to edit to change network settings are in /etc/sysconfig/network-scripts. Most likely you will have at least ifcfg-eth0. RHEL and CentOS usually specify the MAC address of the network adapter there so you need to change it to match with the MAC address of your guest. Otherwise when the guest boots the interface might not get activated. If you want to change the IP address of the guest you can also do it although I’d recommend to leave it for later when you already know the guest works. For the initial boot it should be easier if the guest runs with the same IP address as was used before by the physical server.

You also need to edit file /etc/fstab to match with the partition scheme defined for the guest.

Seventh step is to boot the HVM guest which will be now able to replace the physical server. In order to avoid conflicts you should either power down the physical server, disconnect it from the network or change its IP address(es). After that you can reboot the Xen guest. If you followed the instructions and if the initrd file contains all required modules the system should boot and start all the configured services. If you’re using kudzu it will ask you a few questions about removed/added hardware. Kyle suggests to select “Keep Configuration” for any removed SCSI or network hardware, and select “Ignore” for any added SCSI or network hardware. If you are prompted about any removed video, sound, USB and similar hardware you can safely select “Remove Configuration”. When booting finishes verify that all required service are running and that you have network connectivity from within the guest.

That’s it. You should have now a fully virtualized guest running on the Xen host which has the same configuration as the physical server it replaces. It would be a good idea to perform this process first on a test physical server so you have more time to figure out solutions to any problems you may encounter.

Related post(s)

• Xen and CentOS 5.1 on PowerEdge 2950 (0)
• USB forwarding on Xen - it just does not work (0)
• USB forwarding on Xen (0)
• USB in Xen - how to make it work (0)
• How to Xenify office servers (0)