• you need to be able to download more than one file from Google docs at a time (part 1)
• you have already installed the Firefox extensions needed to do it (part 2)
• the instructions on how to download the files, by Doriano “Paisano” Carta, are, albeit brilliant, too concise for your taste.

Good reasons. Ok, then, here we go.

Summary of the steps to follow:

1. mark all files to be downloaded (Select:All at the bottom of the page)
2. go to Download Your Documents

3. Select the format you want the files to be downloaded in

4. Righ-click --> DownThemAll!
5. Press Start

Where is what?

Once inside Google docs (apologies for the duhism…):

Step 1:  mark all files to be downloaded (see num 1 in Image 1)

You can find Select: All at the bottom of the screen.

Step 2:  go to Download Your Documents (see num 2 in Image 1)

This is at the top right corner of the Google docs menu (not of Firefox, where I was initially looking… sigh…)

Step 3: choose the format for your documents (see num 3 in Image 1)

You can choose among a number of options, including Microsoft Office, OpenOffice.org, csv and pdf.

Image 1: Click on it to see larger (clearer) version:

Step 4: Download them all

After choosing the format, you will be sent to a new page where you will see your documents (see Image 2).

Image 2

On this page:

• Right-click with your mouse
• Choose DownThemAll!...

Image 3 shows the new window that will appear.

Image 3

Step 5: In the DownThemAll window

In the DownThemAll window (Image 3) make sure you have the options of Image 4 ticked (specially All files…)

Image 4

Choose where do you want it to download your files (press the folder to browse your computer) and then Start.

A window like the one in Image 5 will let you know when is the process finished.

Image 5

The end!

PS. I wish to thank the following people for their time and effort.

• Doriano “Paisano” Carta
• The authors of Greasemonkey
• Peter Shafer for his Google Docs: Download Greasemonkey Script
• The authors of DownThemAll
• And of course, the people in charge of Google docs and Firefox!

Thanks to all of you, my job is a lot easier.

Related post(s)

• How to backup / download more than one Google doc at a time (part 2) (1)
• How to backup / download more than one Google doc at a time (part 1) (0)
• Nuke your city (2)
• Do you RAID? (part 2) (0)
• Do you RAID? (part 1) (3)

• searched for something like ”mass download Google docs”
• you feel you are not familiar enough with Firefox Add-ons and/or prefer more step by step instructions and followed the link from How to backup / download more than one Google doc at a time (part 1)

If the former, I would suggest you visit part 1 of this post, specially if you are familiar with Firefox Add-ons and want straightforward instructions. If the latter, welcome to the step by step installation of the applications that will allow you to mass download files from Google docs (regardless of whether they are documents, spreadsheets or presentations).

Step 1: Make sure you have Firefox 3

To check, go to Help --> About Mozilla Firefox.
If it says 2.whatever, update!

Step 2: Install the Greasemonkey Add-on (extension)

Which means,

• in Firefox, go to Tools --> Add-ons
• In the Add-ons windows (see Image 1) search for Greasemonkey
• Press Add to Firefox…
• Press Install Now

Image 1:

Which means:

• Go to http://1st-soft.net/gdd/ and press Install Now
• Greasemonkey will open a window and you have to press Install again.

Step 4: Install the DownThemAll Firefox extension

Which means:

• in Firefox, again, go to Tools --> Add-ons
• In the Add-ons windows (see Image 2) search for Downthemall
• Press Add to Firefox…
• Press Install Now

Image 2

How? Oh, another bit of “choose your own adventure”…

• If you are familiar with Google docs, follow the clear and straightforward instructions by Doriano “Paisano” Carta 
• If you are not so familiar with Google docs and/or prefer more step by step instructions, go to How to backup / download more than one Google doc at a time (part 3).

Related post(s)

• How to backup / download more than one Google doc at a time (part 3) (0)
• How to backup / download more than one Google doc at a time (part 1) (0)
• Nuke your city (2)
• Do you RAID? (part 2) (0)
• Do you RAID? (part 1) (3)

• 9 out of 10 times I will use Google docs over MS Office or even OpenOffice.org.
• I log in every single day and check/edit a file there at least once every couple of hours.

Reasons to be using Google docs more and more often lately? Two mainly:

1. I travel quite a bit, and it is handy not to have to connect an external hard-disk to work on my files, have to update the versions I left at the office when I go back, etc. (Obviously, the drawback of always needing Internet is there).
2. I collaborate with different people on different projects, and Google docs allows me to share each document with only the appropriate people and we can all work on the same file.

Anyone that works with computers on a daily basis is aware that files can be lost, damaged, etc. For this reason, backups are very important. We do not always make them (and cry because of it when we lose the files) but we know we should make them.

As my use of Google docs increased, so did my fear of losing all these files. Google docs is a brilliant free service. Not only because it is free, but because things happen, the files stored in Google docs could disappear one day suddenly and I would be… let’s say… in a pickle.

So, from time to time, I would go to each document I was specially scared of losing, and did the usual:

File --> Download file as  for documents
File --> Export for spreadsheets

and saved a copy of the file on my laptop as a backup.

Ok… no biggie… unless you have 250 files. And you want to make a backup of all 250 files on a regular basis…

So today I  finally decided to stop postponing the inevitable and did some searching. Reading around I learnt about about a few tools you could install (if you had Windows) and/or paid.  Most of them, additionally, mass downloaded only doc files, as in no spreadsheets nor presentations. Of my 250 files, I would say > 200 are spreadsheets…

Luckily (as in, I was almost about to give up), I found this brilliant post HOW TO: Use Web-Based Office Tools Offline which talks about several Online Office applications:

• ThinkFree Office
• Zoho
• Google Docs

And then as an after thought has:

• Two Ways to Export Everything Out of Google Docs
  • Firefox Script Method
  • Free Online Backup Method (Syncplicity is a backup service)

Being a Firefox user myself, I read that Firefox Script Method to download files from Google Docs avidly. I followed their straight forward instructions and… voilà!  45 minutes later I had all 250 files on my laptop.

You might think 45 minutes are still many minutes, but you see, I have the files sorted in files, and the  Firefox Script Method’s one glitch (that I have noticed so far) is that it does not like Google docs folders. So I had to go folder by folder. Luckily I had 15+ files per folder… Add to this that between each download I got entertained looking at other stuff, and you will agree that the 45 minutes for 250 files is an upper limit. Its 20 seconds per file, not bad… Not convinced? Let me do a tiny test. I will do the experiment with 5 files, ok?

method 1:

1. marked one of the five files (without opening it)
2. went to Google docs’ top menu: More Actions --> Save As (and chose xls or doc accordingly)
3. saved file on my laptop
4. unmarked file in Google docs
5. repeat steps 1 to 4 with the other four files

Time taken: 1 minute (20 seconds per file).
Level of annoyance = high (result of being bored of repeating the same procedure over and over).

method 2:

1. mark all five files to be downloaded (Select: All at the bottom of the page)
2. go to Download Your Documents --> as Microsoft Office files
3. Right-click --> DownThemAll!
4. Press Start

Time taken: 27 seconds.
Level of annoyance = none (it would be low if I had files in several folders, as I would have to repeat the same process for each folder).

So, how does one install this mass Google docs download application? Now, time for a bit of “choose your own adventure”…

• If you are familiar with Firefox Add-ons and Scripts, follow the clear and straight forward instructions by Doriano “Paisano” Carta 
• If you are not familiar with Firefox Add-ons and/or prefer more step by step instructions, go to How to backup / download more than one Google doc at a time (part 2).

Related post(s)

• How to backup / download more than one Google doc at a time (part 3) (0)
• How to backup / download more than one Google doc at a time (part 2) (1)
• Nuke your city (2)
• Do you RAID? (part 2) (0)
• Do you RAID? (part 1) (3)

These four RAID levels are examples of nested RAID levels. These RAID configurations are created by combining two simple RAID configurations like for example RAID-0 and RAID-1 into a more complex set-up which offers better performance and better redundancy than each of the single RAID levels. The general approach to creating a nested RAID is to divide the disks into sets of the same size. Each set is then used to create a RAID array. These arrays are usually called nested arrays. Then another RAID is created from the nested arrays. Usually nested RAID levels are created by combining two RAID levels which complement each other’s strengths and weaknesses. Note: regardless of which RAID level you are using remember that raid is not backup. For a short list of risks from which RAID cannot protect you see my previous post.

Let’s start with RAID-01. This RAID level is constructed from an even number of disks, usually four. It is set up by first creating two RAID-0 configurations and then a RAID-1 on top of them. The RAID-1 uses the RAID-0 nested arrays as its components instead of just plain disks. RAID-01 combines features of RAID-0 and RAID-1. The effective size of your storage in an N disk RAID-01 is going to be equal (N/2) times the size of the smallest disk in RAID. The read and write performance is two times that of RAID-1 created with 2 disks. If any disk fails in RAID-01 the whole RAID-0 nested array will be unavailable but your data will be still safe on the second RAID-0 nested array. RAID-01 can survive failure of the second disk from the same RAID-0 nested array but since it won’t be used at this point anyway this doesn’t really offer any additional redundancy compared with RAID-1. After any disk in RAID-01 fails the data is still available but you need to replace it as soon as possible otherwise when any disk in the other RAID-0 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-01 with four disks: A, B, C and D.
Disks A and B are combined into one RAID-0 nested array, disks C and D into another.
The whole RAID-01 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+B or C+D.
If A+C, A+D, B+C or B+D fail you lose your data.

RAID-10 uses a similar approach as RAID-01 but offers better redundancy. RAID-10 also requires an even number of disks, minimum four. It is set up by creating two RAID-1 configurations and then combining them with RAID-0. The RAID-0 uses the RAID-1 arrays as its components. The effective storage size is the same as for RAID-01 and is equal (N/2) times the size of the smallest disk. The read and write performance is the same as of RAID-01. The advantage of RAID-10 is that it can deal with two simultaneous disk failures as long as both failures are in different RAID-1 nested arrays. For a four disk RAID-10 array the probability that the second disk failure will be in a different RAID-1 pair than the first is 66.6% which is pretty good. After any disk in RAID-10 fails the data is still available but you need to replace it as soon as possible otherwise when another disk in the same RAID-1 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-10 with four disks: A, B, C and D.
Disks A and B are combined into one RAID-1 nested array, disks C and D into another.
The whole RAID-10 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+C, A+D, B+C or B+D.
If A+B or C+D fail you lose your data.

RAID-50 is similar to RAID-10 but instead of composing RAID-0 out of two RAID-1 arrays it uses RAID-5 arrays as its building blocks. RAID-50 requires an even number of disks, at least six. The effective storage size of RAID-50 is equal [(N/2)-1] times the size of the smallest disk. The read and write performance is two times that of RAID-5 created with half the amount of disks. RAID-50 can deal with two simultaneous disk failures (similarly as RAID-10) as long as the failed disks are in different RAID-5 nested arrays. For a six disk RAID-50 array the probability that the second disk failure will be in a different RAID-5 nested array than the first is 60%. After any disk in RAID-50 fails the data is still available but you need to replace it as soon as possible otherwise when another disk in the same RAID-5 nested array fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-50 with six disks: A, B, C, D, E and F.
Disks A, B and C are combined into one RAID-5 nested array, disks D, E and F into another.
The whole RAID-50 array can survive simultaneous failure of two disks without any data loss if the failing disks are: A+D, A+E, A+F, B+D, B+E, B+F, C+D, C+E or C+F.
If A+B, A+C, B+C, D+E, D+F, or E+F fail you lose your data.

RAID-51 is a very rare configuration and sometimes is called “RAID for very very paranoid”. It’s similar to RAID-01 but instead of building a RAID-1 out of RAID-0 arrays it uses RAID-5 arrays as its components. RAID-51 requires an even number of disks, at least six. The effective storage size of RAID-51 is the same as of RAID-50 and equal [(N/2)-1] times the size of the smallest disk. The read performance is equal to RAID-5 with N/2 disks – it can be double if the reads are using both RAID-1 mirrors in parallel. The write performance is equal to RAID-5 with N/2 disks and cannot be improved. RAID-51 can handle three simultaneous disk failures regardless of which disks failed. RAID-50 can survive even more simultaneous disk failures as long as at least one of the RAID-5 nested arrays has only one failed disk. After three disks in RAID-50 fail the data is still available but you need to replace at least one of them as soon as possible otherwise when another disk fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Example: RAID-51 with six disks: A, B, C, D, E and F.
Disks A, B and C are combined into one RAID-5 nested array, disks D, E and F into another.
The whole RAID-51 array can survive simultaneous failure of more than three disks without any data loss if at least two disks in either of the two nested arrays are OK.
If both nested arrays had two disk failures you would lose your data.

This concludes the RAID mini series. If you want to find out more about RAID I recommend the reference guide at storagereview.com website.

Any questions about RAID? Write a comment and I’ll get back to you.

Related post(s)

• Do you RAID? (part 1) (3)
• How to backup / download more than one Google doc at a time (part 3) (0)
• How to backup / download more than one Google doc at a time (part 2) (1)
• How to backup / download more than one Google doc at a time (part 1) (0)
• Xen and CentOS 5.1 on PowerEdge 2950 (2)

What is RAID? It’s an acronym meaning either “Redundant Array of Independent Disks” or “Redundant Array of Inexpensive Disks”. The two meanings are just something inherited from the past, many times both meanings apply. The basic idea is that you store the data on more than one disk and by using a few pretty cool ideas you can protect your data from problems like a failing disk and/or increase the performance of the storage system. An important thing to remember is that RAID is not a substitute for backup. While RAID can guarantee that you can still access all your data even if some of the disks are not working it won’t help if you accidentally delete the wrong file, your data gets wiped out by a virus, a power supply failure in your computer fries all your disks, a disgruntled employee deletes files on purpose or formats a disk, a hacker gets into your system, your disks get stolen, the building burns or gets flooded, a hostile country starts dropping bombs in your location, a meteorite strikes your home … do you get the point? To protect yourself from these situations you need a backup, preferably stored off-site. If the meteorite is very big you better keep your backups very far away, like on another planet. After all we know what happened to the dinosaurs, right? They obviously had no backups…

RAID can be setup in many different ways, these configurations are described as RAID levels. Most common RAID levels are RAID-0, RAID-1, RAID-5 and RAID-6. There are also nested RAID levels like RAID-10 or RAID-50 where multiple RAID setups are placed on top of each other to provide more functionality than each of the individual levels. Each RAID level can be implemented either using specialized hardware or software. Today I will talk about RAID levels 0, 1, 5 and 6. Nested RAID levels will be the topic of the following post.

Let’s start with RAID-0, which is not really RAID in a sense that it doesn’t provide any redundancy. What RAID-0 does is that it provides higher performance for the storage. In RAID-0 data is spread over multiple disks in so called stripes. Basically each file (unless it’s very small) is stored on all the disks in the storage. The advantage is that when you want to read the file the system can read each piece of the file from a different disk and because of that it can do this N times faster, where N is the number of disks in your RAID. The same happens when writing. The effective size of your storage is going to be equal N times size of the smallest disk in RAID. RAID-0 provides high performance but unfortunately it doesn’t provide any redundancy for your data, in fact RAID-0 makes it more likely that you’re going to lose data. If any disk in RAID-0 fails you’re going to lose all your files since each file had a piece on this failed disk. And since there are multiple disks in RAID-0 it is more likely that one of them is going to fail than if you had just one disk. After any disk in RAID-0 fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

A different approach is taken with RAID-1. This scheme fully deserves to be called RAID since it does provide redundancy for the data. In RAID-1 all data is stored on all the disks in RAID. Usually RAID-1 is used with 2 disks since this is most economical. You can use 3 or more disks in RAID-1 but if you really need this level of availability for your data then maybe you should look into other solutions like clustered storage. The biggest disadvantage of RAID-1 is the effective size of your storage is going to be equal to the size of the smallest disk in RAID. In terms of performance RAID-1 can provide an improvement when reading files (depending on the implementation different fragments of the file can be read from different disks since they all contain the same data) similar to RAID-0 but the write performance is the same as for a single disk (all data has to be written to each disk). When a disk fails in RAID-1 the data is still available but you need to replace it as soon as possible otherwise when the other disk fails you’ll have to restore all your data from the backups. You do have backups, don’t you?

Another common RAID scheme is called RAID-5. RAID-5 requires at least 3 disks to be used. It provides redundancy and also increased performance. In RAID-5 each data block is stored on one disk only but an additional information (called parity) for each block is stored on a different disk. The parity is a checksum which can be used by the system to recover data in case of disk failure. The parity information is spread over all the disks. This is done in such a way that the system can tolerate the failure of any one disk and not lose any data. The effective size of storage with RAID-5 is equal to (N-1) times the size of the smallest disk. Read performance of RAID-5 can approach (N-1) times that of a single disk. Write performance is rather poor since each write requires updating not only the data but also the parity information. The usual method to improve the write performance is to include cache memory in the RAID controller. RAID-5 is best used for systems where on average the number of reads highly exceeds the number of writes. When a disk in RAID-5 fails the data is still available but you need to replace the failed disk as soon as possible since from that point any further disk loss will mean you’ll have to restore all your data from the backups. You do have backups, don’t you?

An improvement on RAID-5 in terms of data redundancy is RAID-6. The difference here is that in RAID-6 parity is calculated twice independently and is stored on two disks. RAID-6 requires at least 4 disks to be used. Similar as with RAID-5 the parity information is spread over all the disks. The advantage is that the system can now survive the failure of any two disks without losing any data. The disadvantage is that the effective size of the storage is reduced to (N-2) times the size of the smallest disk. Performance of RAID-6 is a bit lower than of RAID-5 due to additional parity data. RAID-6 makes more sense with larger arrays (N>=12) because of the increased probability of double simultaneous disk failure. When two disks in RAID-6 fail the data is still available but you need to replace at least one of the failed disks as soon as possible since any further disk failure will mean you’ll have to restore all your data from the backups. You do have backups, don’t you?

Next week I’ll talk about nested RAID levels: RAID-01, RAID-10, RAID-50 and RAID-51 in Do you RAID? (part 2).

Any questions about RAID? Write a comment and I’ll get back to you.

Related post(s)

• Do you RAID? (part 2) (0)
• How to backup / download more than one Google doc at a time (part 3) (0)
• How to backup / download more than one Google doc at a time (part 2) (1)
• How to backup / download more than one Google doc at a time (part 1) (0)
• Xen and CentOS 5.1 on PowerEdge 2950 (2)

Usually the Windows PCs store the user profiles on the server. Backing the user profiles can then be done as part of backing up the whole server. You might have some other files though which are not part of the user profile. Backing up these files can be easily done using rsnapshot and autofs.

We’ll use autofs to setup file sharing between the Windows PCs and the Linux server. This is especially useful for computers which are not always connected to the network like laptops, although people do turn off their desktops as well from time to time. What autofs allows us to do is to activate file sharing only when it’s needed for the purpose of doing a backup.

Performing the actual backup will be done using rsnapshot. This tool, which I already mentioned in a previous post, allows space efficient backups which are stored incrementally while offering ease of access usually associated with full backups.

Before setting up autofs we need to create a network share on the Windows PC. Remember the user and the password since we’ll need them later on. In this article I’m going to assume the user name is backup and the password is secret but you should use something more difficult to guess depending on your security requirements. I’m also assuming the Windows domain is called company. On the Linux server you then need to make sure you’ve got autofs installed. Installation of autofs is not covered here but usually it’s as simple as executing yum install autofs. You need to edit /etc/auto.master file and add this line

/smb /etc/auto.windows --timeout=60

Then you create the file /etc/auto.windows containing the following, in a single line:

<share> -fstype=smbfs,
credentials=/etc/auto.windows.<credentials file>,
uid=<user>,gid=<group>,
fmask=0755,dmask=0755 ://<windows PC>/<network share>

This must be written on a single line, with no separating spaces starting from -fstype to dmask=0755

<share> is the name of directory where you want to mount the network share from Windows PC.

/etc/auto.windows.<credentials file> is the name of file where you store the user name and password needed to access the network share on Windows PC. In our case it will look like this:

username = backup
password = secret
domain = company


After these preparations it’s time to tell the autofs daemon about these changes by executing service autofs reload or a corresponding command. You should be able now to see that the network share gets automatically mounted when you execute ls /smb/<share>. After 60 seconds the share will be unmounted.

Now the last part – setting up rsnapshot. Just follow the instructions for rsnapshot as if the directory /smb/<share> was local. Thanks to autofs, rsnapshot doesn’t need to know that it’s really on a different computer running Windows. Remember to set up a cron job if this is your first time you’re using rsnapshot.

That’s it. You’ve configured an automatic backup of a Windows computer using Linux. You can repeat this procedure on further Windows clients.

P.S. Another approach is described here but it has the drawback that it requires installing additional software on the Windows PCs.

Related post(s)

• Skype 4.0 for Windows is out: Pros and cons (compared with Linux Skype 2.0) (1)
• Too much success and popularity? (0)
• That’s how I’ve learned to like Jazz (2)
• Setting up UPS on CentOS 5.2 with SELinux, part 2 (1)
• Setting up UPS on CentOS 5.2 with SELinux, part 1 (0)

After doing some research I decided to purchase Enermax EB307S-B Jazz. It just arrived today (thanks alternate.de for a quick delivery). As advertised the enclosure allows a very quick installation. There’s no need to use any tools. Simply unlock the enclosure, slide out the cover and lift it. Now you can install the hard disk – this takes another 15 seconds. Replace the cover, lock it and that’s it. Now just connect power and the USB cable. All this took barely any longer than if the disk was already installed.

Enermax Jazz has a very nice design. Brushed aluminum housing with steel mesh provides very good cooling and sturdiness. It also looks cool since you can see the disk inside. Because the casing is a mesh it doesn’t need a fan which makes it suitable for quiet environments. But be careful with liquids – if you spill anything on the enclosure you will probably fry your data!

After connecting the enclosure to the file server I’ve formatted it. The file server runs Linux so I’ve used ext3 file system but you could use this hardware also with a Mac or Windows machine. Then another couple of minutes to decide what to backup (more or less everything) and I’ve started the copy process. It’s been running for the last 40 minutes and so far 47 GB have been transferred to the external disk. 120 GB more to go… I’m using rsnapshot to perform the backups. It’s a really nice tool I’ve recently discovered. It creates regular snapshots of the directories you tell it to monitor. To save space it uses hard links to store files which haven’t changed since the previous snapshot. Usually that means that your backup space requirements are only about 20-30% more than the amount of data you have, even if you want to have a history of your files going back a few months.

I like Jazz.

Btw, if anybody from UK wants to buy this enclosure I’d suggest to check first if you can get it from a shop on the continent. Price will be much lower.

Related post(s)

• Automatic backup of Windows computers using Linux (5)
• How to backup / download more than one Google doc at a time (part 3) (0)
• How to backup / download more than one Google doc at a time (part 2) (1)
• How to backup / download more than one Google doc at a time (part 1) (0)
• Internet is the extension of my brain (0)