Storage and Backup Options

[Burninator]

I would like to digitize all of my DVD's onto my computer for use in a HTPC (not sure what direction yet, still racking my brain). My computer currently has a 500GB hard drive which isn't big enough for all my files and I would like to have room for future expansion. So I would like to buy some more storage, but I am not sure which direction is best.

My first thought was to buy a 1TB hard drive and dump everything on there. This is clearly the easiest and cheapest solution. Of course if the hard drive fails I lose all my media. This is also when my expertise runs out. I am not really familiar with anything else.

Some other options that I have stumbled upon are Drobo and NAS. The Drobo seems great. It has good redundancy and seems very easy to use. Of course it's $400 without any storage and I don't know if it's worth the expense. I don't know much else about the Drobo and I am not really sure what NAS is.

Also since I am buying more storage I figure it is a good time to include something to backup my non-media files with. I've been flirting with danger as I haven't backed-up my computer in the 2 and half years I have owned it. Yikes, I know. What is a good solution for this as well?

What is the best solution for me that is cost effective? I suppose I should mention I am running a PC, I have a wireless router, my PC case has lots of room for expansion and my MB is the ASUS P5K Premium. I am running Windows Vista Home Premium and I would to upgrade to some version of Windows 7 in the future. I am not a total idiot when it comes to computers, but I am by no means an expert either. So I would like something that is fairly easy to use.


Thanks a lot for the help!

[Buzzard]

Maybe give carbonite a try. It's about as easy to use as it gets. Short term it's cost effective, long term not so much.

[photon]

I have a DNS-323 NAS, (much cheaper than the Drobo) and it works well, 1 1.5TB mirrored drives = lots of room for backups and media.

Though that's not really backup, Carbonite as mentioned or one of the other online backups is good for if your house burns down or a power surge toasts the drives.

I'm thinking of changing my NAS in the future to one of QNAP's. The DNS-323 does what it does, but it's not that fast (fast enough for streaming media though).

[Burninator]

Originally Posted by photon

I have a DNS-323 NAS, (much cheaper than the Drobo) and it works well, 1 1.5TB mirrored drives = lots of room for backups and media.

Though that's not really backup, Carbonite as mentioned or one of the other online backups is good for if your house burns down or a power surge toasts the drives.

I'm thinking of changing my NAS in the future to one of QNAP's. The DNS-323 does what it does, but it's not that fast (fast enough for streaming media though).

How exactly is a NAS different from a external drive or a drobo?

[photon]

A NAS just means it's attached to the network instead of attached directly to the computer. Useful if you have more than one computer on the network, or want to stream media while your computer is off.

[sclitheroe]

Depending on how much you value your digital media I think it's important to back it up as well, not just your non-media files. In my case, its paramount - I've gone completely digital and have no DVD or CD media left in my house. Same for photos, they are all on my server and displayed on the HD TV. Mozy is only 5 bucks a month for unlimited online backups, I assume Carbonite and others are comparable.

The drobo and other consumer NAS/DAS (direct attached storage) devices are nice, but they don't have redundant power supplies, they have short warranties, and there is little to no guarantee that the internals are going to remain consistent across revisions, allowing you to move your disks to a new enclosure if it goes boom. They also don't protect against file system corruption, and worse, in the case of the Drobo, your partitions actually live on a lower level form of file system that is specific to the Drobo. If that low-level file system corrupts, I have no idea what you'd be able to do to recover from it.

So while I do agree that having some redundancy at the disk level is important, because its a guaranteed failure point (the reality is that disks begin to fail the second you turn them on - it just takes a long time on average until they are unusable) devices like the Drobo don't cover a lot of the other spectrum of failures that can potentially occur. They also don't like to talk about them on their website, which is a shame - they should take some responsibility in explaining where the Drobo fits into the spectrum of data availability and recoverability.

What I'm trying to say is that when you are planning large scale home storage systems, don't assume that buying a fancy consumer grade NAS or DAS is all you need to do (unless that data you are putting on it isn't worth anything). Put some thought into how you would handle a failure of that part of the overall system, and how much impact it would have on you, and put countermeasures in place from the start

[photon]

Good point about the media backup, I haven't reached that completely digital state yet.

And the only problem with Moby or any other online backup is for really huge amounts (1TB say), it would take YEARS to do the first upload if you tried to stay within the transfer limits of most ISPs

Yeah the DNS-323 uses ext2 or ext3 so if the unit dies completely I can still mount the drives elsewhere and read the files, that was one benefit.

They should make a NAS with a hot-swap tray, 2 mirrored drives and one tray that you can hot-swap out, and a built in jungle disk (or whatever) module for backup to the cloud.

[sclitheroe]

Originally Posted by photon

Good point about the media backup, I haven't reached that completely digital state yet.

And the only problem with Moby or any other online backup is for really huge amounts (1TB say), it would take YEARS to do the first upload if you tried to stay within the transfer limits of most ISPs

It takes a while, which is why you get started on it before you accumulate too much stuff. I also did my initial uploads in small batches so that I could prioritize what went up - it was most important to get the photos uploaded, then the music I no longer had physical copies of that would not be available elsewhere, etc. In the meantime, I simply kept a second copy locally as well.

It's quite unobtrusive - mine runs throttled to 256 kilobits upstream during the day, and full speed from 11pm to 7am each night. I hardly ever know its there, even with latency sensitive devices like the PS3 and online games.

[sclitheroe]

Originally Posted by photon

Yeah the DNS-323 uses ext2 or ext3 so if the unit dies completely I can still mount the drives elsewhere and read the files, that was one benefit.

That's pretty neat - and likely its just simple Linux md raid underneath. The drobo is a whole different story though, I don't know what they do to manage their volumes, and they won't tell us

Although.... if you RAID-1'ed a pair of Drobo's....hmmmm...

[Burninator]

You both raise lots of interesting things to think about. In my mind I wasn't making a distinction between redundancy and backup. I was only thinking about drive failure and not about all the other problems that can mess up your data.

I have two types of data. Data that can be replaced and data that can't (err..I guess that is obvious). The data that can be replaced are things like my DVD's and my CD's. It would take a lot of time to rip everything again, but that's all it would take is time. However data like pictures, iTunes purchases and documents can't be replaced. I think I can find a happy medium between redundancy and backup while keeping costs down (my gadget budget isn't huge, but the list of gadgets is).

I think I will buy two 1 or 1.5 TB hard drives, RAID 1 them and put all my media on them (movies, music and pictures). This will give me hard drive redundancy and some piece of mind. It will also save me the money of buying something like a NAS or Drobo.

Then I think I will try a trial of the online backup sites for the data that cannot be replaced. In terms of file size the irreplaceable data is significantly smaller than my whole music and DVD library ripped onto my computer is, so it shouldn't take a long time to put everything on there. But I suppose it also allows me to backup everything else if I decide to as well. I'll have to put more thought into the online backup and see if it is worth it or not.

[GoinAllTheWay]

Ok, no one has brought it up yet so I will.

http://www.memoryexpress.com/Products/PID-MX23988(ME).aspx

I've pimped this thing numerous times in other threads. It's awesomeness is awesome.

It will do everything you need it to. I built one of my own on an existing PC I had and it has proven to be VERY useful and VERY reliable. It fully restored my gaming rig in less than an hour, drivers/OS, everything.

Only downside to this unit it you can't expand it much. Ideally I would re-use an old PC and load WHS on to it. It's hardware requirements are really low. I intend to replace the one I currently have with one that has around 10 HDD bays in it.

If you have any questions about it, please feel free to PM me.

[sclitheroe]

Originally Posted by Burninator

I think I will buy two 1 or 1.5 TB hard drives, RAID 1 them and put all my media on them (movies, music and pictures). This will give me hard drive redundancy and some piece of mind. It will also save me the money of buying something like a NAS or Drobo.

Then I think I will try a trial of the online backup sites for the data that cannot be replaced. In terms of file size the irreplaceable data is significantly smaller than my whole music and DVD library ripped onto my computer is, so it shouldn't take a long time to put everything on there. But I suppose it also allows me to backup everything else if I decide to as well. I'll have to put more thought into the online backup and see if it is worth it or not.

Very sensible approach.

I am going to make a suggestion though:

Think about RAID-1 in terms of availability, not redundancy. Two 1.5 TB drives in a RAID-1 will protect against drive failure, ensuring your data remains available, but not file system corruption (and the chances of that are greater with software RAID-1), meaning your setup is not redundant.

What I would do is use one 1.5TB drive as your data drive. The second 1.5 TB drive, you use to keep an up to date clone of your data on the first drive. On the Mac, you could use Time Machine, on the PC, you could use something like Acronis. Either way, find a solution that will do it on a regular schedule for you - maybe daily, maybe weekly if the data on the 1.5 TB drive doesn't change very often. For a large iTunes or other multimedia library, I find weekly is fine - I don't add or change much stuff on a daily basis.

This way, if your primary drive dies, you are only down for as long as it takes you to replace the drive, and copy the data from your backup drive. If the file system corrupts, you just re-format and copy the data back. Either way, the second copy of the data is now impervious to both file system corruption and mechanical drive failure. The only cost, from your perspective is if the primary drive dies, you are offline for a little while while you restore. For a home scenario, this is hardly the end of the world.

[sclitheroe]

Originally Posted by GoinAllTheWay

Ok, no one has brought it up yet so I will.

http://www.memoryexpress.com/Products/PID-MX23988(ME).aspx

I've pimped this thing numerous times in other threads. It's awesomeness is awesome.

It will do everything you need it to. I built one of my own on an existing PC I had and it has proven to be VERY useful and VERY reliable. It fully restored my gaming rig in less than an hour, drivers/OS, everything.

Only downside to this unit it you can't expand it much. Ideally I would re-use an old PC and load WHS on to it. It's hardware requirements are really low. I intend to replace the one I currently have with one that has around 10 HDD bays in it.

If you have any questions about it, please feel free to PM me.

WHS is a good solution for sure - if you look at my previous post about not using RAID-1 to achieve redundancy, WHS nails this. In WHS, "protected files" are replicated between file systems on the WHS server, meaning that there are two copies, on two separate drives, on two separate file systems. This is transparent to the client machines, but accomplishes both goals - protecting against drive failure, and protecting against file system corruption.

[mykalberta]

I was in the same position. I garbaged almost all my DVD cases and kept my original DVD in case the MPAA Nazzis come knocking.

I bought 2 1.5 TB Hard drives and made a clone copy onto the second one then took it out of my computer. Its kept in a cool low humidity location with its antistatic bag. I have outlook reminders set to test it twice a year just to make sure.

Online backup options arent cost effective for that kind of storage. They are good for documents etc but not the archiving of all your DVDs.

FYI - Handbrake + AnyDVD is the shiz. 30 day free trial of AnyDVD - if you need it longer just rebuild your computer - I did 3 times. If AnyDVD was reasonable priced I would have no problem dropping $20 for it, but not $60 for a 1 year license.

[sclitheroe]

Originally Posted by mykalberta

I was in the same position. I garbaged almost all my DVD cases and kept my original DVD in case the MPAA Nazzis come knocking.

I bought 2 1.5 TB Hard drives and made a clone copy onto the second one then took it out of my computer. Its kept in a cool low humidity location with its antistatic bag. I have outlook reminders set to test it twice a year just to make sure.

Online backup options arent cost effective for that kind of storage. They are good for documents etc but not the archiving of all your DVDs.

FYI - Handbrake + AnyDVD is the shiz. 30 day free trial of AnyDVD - if you need it longer just rebuild your computer - I did 3 times. If AnyDVD was reasonable priced I would have no problem dropping $20 for it, but not $60 for a 1 year license.

Online backup options can be cost effective - Mozy is 5 bucks a month for unlimited storage - as long as the file is live on your server machine, its backed up (Mozy is not remote storage though - if you delete the file from your server, or put it on removable media, and leave the removable media disconnected, its gone after 60 days). I consider Mozy to be a $5 monthly surcharge on my house insurance, to ensure I can get all my digital possessions back in the event of a catastrophe.

Sucks about AnyDVD - I bought it last year (I think) before they went to a yearly fee, so I have lifetime updates. Instead of reinstalling your machine all the time, if you don't want to pay that much yearly, install the trial version in a virtual machine, take a snapshot of the VM, and keep rolling the VM snapshot back as required. Much cleaner

[photon]

The other downside of the online backups is they don't allow backups of networked drives.. I might have some of my documents on the network drive because they are shared between my wife and I, or have all the media on the NAS because I don't need it on my computer. How do you get around that restriction sclitheroe?

[Burninator]

Originally Posted by sclitheroe

Very sensible approach.

I am going to make a suggestion though:

Think about RAID-1 in terms of availability, not redundancy. Two 1.5 TB drives in a RAID-1 will protect against drive failure, ensuring your data remains available, but not file system corruption (and the chances of that are greater with software RAID-1), meaning your setup is not redundant.

What I would do is use one 1.5TB drive as your data drive. The second 1.5 TB drive, you use to keep an up to date clone of your data on the first drive. On the Mac, you could use Time Machine, on the PC, you could use something like Acronis. Either way, find a solution that will do it on a regular schedule for you - maybe daily, maybe weekly if the data on the 1.5 TB drive doesn't change very often. For a large iTunes or other multimedia library, I find weekly is fine - I don't add or change much stuff on a daily basis.

This way, if your primary drive dies, you are only down for as long as it takes you to replace the drive, and copy the data from your backup drive. If the file system corrupts, you just re-format and copy the data back. Either way, the second copy of the data is now impervious to both file system corruption and mechanical drive failure. The only cost, from your perspective is if the primary drive dies, you are offline for a little while while you restore. For a home scenario, this is hardly the end of the world.

Thanks for the suggestion. That sounds like a good idea. So if I purchase this Acronis software will I be able to backup more than one hard drive? I currently have a 500GB hard drive with my OS (and currently everything else) on it. Will that software allow me to back up both my 500GB hard drive and my future 1.5TB hard drive? So if either of those drives fails I will be able to restore them? Would I just partition the backup hard drive or will Acronis be smart enough to figure it out? I know that is a very specific question, but it would be nice to be able to back up my whole computer and separately restore hard drives if I needed to.

I took a quick look on the website and it didn't get into those specifics that I could find. But I do like the idea of making a restore point for upgrading to Windows 7 and having that as a backup encase the upgrade/install fails as I plan to do that in the future. The install, not the failure. Also I like that it keeps track of what files have changed and updates those in the backup instead of the whole thing every time.

[Burninator]

Originally Posted by GoinAllTheWay

Ok, no one has brought it up yet so I will.

http://www.memoryexpress.com/Products/PID-MX23988(ME).aspx

I've pimped this thing numerous times in other threads. It's awesomeness is awesome.

It will do everything you need it to. I built one of my own on an existing PC I had and it has proven to be VERY useful and VERY reliable. It fully restored my gaming rig in less than an hour, drivers/OS, everything.

Only downside to this unit it you can't expand it much. Ideally I would re-use an old PC and load WHS on to it. It's hardware requirements are really low. I intend to replace the one I currently have with one that has around 10 HDD bays in it.

If you have any questions about it, please feel free to PM me.

I did think about the WHS. But it just seemed like overkill for my application. I think if I had multiple computers and laptops I would consider it, but I would rather have everything contained in one box under my desk for the moment. Perhaps in the future I might buy one when I want to spend the extra money. At least then I will already have the hard drives for it.

[sclitheroe]

Originally Posted by photon

The other downside of the online backups is they don't allow backups of networked drives.. I might have some of my documents on the network drive because they are shared between my wife and I, or have all the media on the NAS because I don't need it on my computer. How do you get around that restriction sclitheroe?

I don't use a NAS, I have a physical machine acting as a file server (among other chores neccessary for running my one-man empire)

That is definitely one issue with online backup systems as they exist today.

[sclitheroe]

Originally Posted by Burninator

Thanks for the suggestion. That sounds like a good idea. So if I purchase this Acronis software will I be able to backup more than one hard drive? I currently have a 500GB hard drive with my OS (and currently everything else) on it. Will that software allow me to back up both my 500GB hard drive and my future 1.5TB hard drive? So if either of those drives fails I will be able to restore them? Would I just partition the backup hard drive or will Acronis be smart enough to figure it out? I know that is a very specific question, but it would be nice to be able to back up my whole computer and separately restore hard drives if I needed to.

I took a quick look on the website and it didn't get into those specifics that I could find. But I do like the idea of making a restore point for upgrading to Windows 7 and having that as a backup encase the upgrade/install fails as I plan to do that in the future. The install, not the failure. Also I like that it keeps track of what files have changed and updates those in the backup instead of the whole thing every time.

For sure. You could even set up multiple scheduled jobs I believe, one to do the OS drive frequently, the other job to do the media drive less frequently. Or whatever.