Tell me about RAID and/or other backup solutions!

[Table 5]

So I'm looking to get some sort of long-term storage/backup solution, and frankly I'm confused with all the terminology and options out there. It's not my forte by any means.

Wondering if someone could recommend something for what I need.

Needs:
1. Need a way to backup all my work and photo files. Look for at least 8 terrabytes.
2. I would like to have some sort of Raid solution were the files are houses on two separate drives (so if one dies, it will still all be there on the other). Not sure what "Raid number" this is.
3. Ideally I'd like to be able to swap out drives, so I can store a couple off-site (in case of fire etc.). Again not, sure what Raid number allows this.
4. I currently only have a USB 2.0/firewire computer, but it seems like Thunderbolt is the next big thing with Apple computers, so I'd like to future proof. Ideally, Id get something with both Tbolt/USB connection. If not, then Thunderbolt, and get an adapter to USB/FW.

Any particular brands or models to recommend? I'm definitely not a fan of Lacie. Ive heard G-Tech is pretty good, although the stuff isn't cheap. I'll pay for the right brand though if its worth it.

I currently have a 1TB Airport Time Capsule, that acts as my daily backup, which is nice. Although Ive had it for 4-5 years now and it looks like it's ready to crap out, so I'm thinking of upgrading to the 3TB Time capsule...although I think this would be outside of my regular backup.

Any thoughts would be appreciated!

[Rathji]

RAID is not a backup.

The simplest way to remember how to back up your images safely is to use the 3-2-1 rule.

Keeping 3 copies of any important file (a primary and two backups)
Having the files on 2 different media types (such as hard drive and optical media), to protect against different types of hazards.*
1 copy should be stored offsite (or at least offline).

[Hack&Lube]

Multiple hard drives via RAID are for near-line data, performance, or redundancy in case of failure. It is not a backup strategy. If you still want to proceed with this, you sound like you want RAID 1 (mirroring) with hot swap (swap drives in and out) and auto-rebuild (rebuild the data set on blank drives going in). You'll need to mirror all your current disks.

There are certainly a bunch of NAS consumer products that can do this. Requiring Tbolt will limit your options in terms of product choice as it's more Mac centric.

[Table 5]

Thanks for the thoughts, although I'm confused why it's not considered a backup strategy. I'd be making regular copies of my work onto the...how is that not a backup?

If that's not the best strategy or thing to buy, then what would you recommend?

[Hack&Lube]

Originally Posted by Table 5

Thanks for the thoughts, although I'm confused why it's not considered a backup strategy. I'd be making regular copies of my work onto the...how is that not a backup?

If that's not the best strategy or thing to buy, then what would you recommend?

Backups are a safe copy of your data that is stored off line or in a secure place.

A RAID set just means that if you lose one hard drive in the group, your data is safe because it's mirrored or spread out in the rest of the group by parity. That's near-line redundancy.

If your NAS or all your harddrives or your computer goes boom, then all your data is *poof* because they were all plugged in and all in the same place.

That said, mirroring all your drives (RAID 1) would be the simplest way to get what you want if you simply take the mirrored drive out every week and replace it with an empty one and allow the set to rebuild (depends on the software or technology that comes with your system). RAID is not originally intended for this purpose. You might be better off with a separate large storage system that you can do a nightly backup to via software because to get 8TB of swapping means you will be swapping several disks a week.

[photon]

RAID isn't a backup in and of itself. RAID 1 is mirroring of drives (so whatever one looks like the other will look like).

If, for example, you had two hard drives mirrored in your computer (or some other type of RAID that involves multiple copies of your data), that's not a backup because if you delete the file or save over the file with undesired changes, you can't recover the old state from a backup. Or if an app corrupts the file it'll be corrupt and you can't recover a non-corrupt version. Or if the operating system decides to eat the file (or the disk) for lunch it'll eat all involved files or drives.

So in that respect RAID is not a backup.

[photon]

As to your specific scenario, with 8TB capacity desired, you might need/want to look at some scenario where you swap out an entire NAS, rather than just drives.

A couple of Drobo minis where one is connected to the computer and the other is offsite. Not sure how the software would work, if just pointing time machine to it would be enough or if you'd need some more robust backup software to keep track of things.

An online one would be good, but you'd probably want to find one where you could send hard drives with your initial backup since uploading 8TB would take quite a while and you'd have to spread it out over a long time anyway to not go over bandwidth limitations from your ISP likely.

A tape drive is also an option for this kind of capacity, but quite expensive.

[photon]

Or just use hard drives and send them offsite kind of like a tape, have two sets with trays and hot-swap enclosure, and run a more traditional backup software that creates the backup set on the drives, weekly full daily incremental.. that seems like a lot of extra work.

[sclitheroe]

Originally Posted by Hack&Lube

That said, mirroring all your drives (RAID 1) would be the simplest way to get what you want if you simply take the mirrored drive out every week and replace it with an empty one and allow the set to rebuild (depends on the software or technology that comes with your system). RAID is not originally intended for this purpose. You might be better off with a separate large storage system that you can do a nightly backup to via software because to get 8GB of swapping means you will be swapping several disks a week.

/curmudgeon mode on

I detest this approach to using RAID rebuild capabilities to create mirrors of drives. It's not what it's meant for, and its a fools errand to deliberately degrade a RAID data set. If I caught someone doing this with a client's data, I'd fire them.

I'm so glad you pointed out that this is not what RAID is intended for. It's absolutely not, full stop. If _anything_ goes wrong during the rebuild of an N+1 redundant data set, you are potentially hooped. I feel physically ill thinking about the times when things have gone bad, and on the next boot the RAID controller decides the partially initialized mirror drive is actually the primary, intact copy, and mirrors an incomplete stripe set back onto your only good copy.

/curmudgeon mode off

I cannot stress enough H&L's recommendation that it's not how RAID is meant to be used.

[Rathji]

I would suggest a NAS/HDD enclosure, with multiple drives. A 4 bay unit with a couple 4TB HDD gives you lots of room for expansion.

I don't know about Thunderbolt devices, so I won't give any specific product recommendations.

[sclitheroe]

At 8TB of anticipated storage, you should be looking at a low end SMB storage product, not consumer grade stuff.

One of these would be a good place to start:
http://www.promise.com/storage/raid_...rsn1=3&rsn3=43

You want at least one more drive bay than you need (eg. 4 x 2TB is 8 TB on 4 drives, but you want a 5th drive as a hot spare - rebuilding a 2TB drive takes a long time, and you are vulnerable to failures in the interim)

You then want to stock it with enterprise class drives. You will thank me later for the $30 extra you spent per drive (and I'll tell you why you want these if you care, but the reasons are sound):
http://www.memoryexpress.com/Products/MX43655

That model drive there used to be called the Constellation series, and they are extremely solid.

[Hack&Lube]

I wish there were more affordable consumer level tape backup options. Has anybody done tape backup at home?

[Rathji]

Tape Backup is so expensive per GB and slow compared to a HDD solution that it really doesn't make sense to do it at home.

I have a couple LTO drives that I am about to mothball at work because we just can't justify the cost for our increased (current and expected) data needs.

[sclitheroe]

Originally Posted by Hack&Lube

I wish there were more affordable consumer level tape backup options. Has anybody done tape backup at home?

Yeah, still pricy:
http://www.cdw.com/shop/products/Quantum-LTO-5-HH-SAS-HBA-Bundle-tape-drive-LTO-Ultrium-SAS-2/2075533.aspx?enkwrd=ALLPRODTC-L52BN-EZ)

It's not unaffordable if you need many generations of backups - LTO5 tapes have come way down in price, but most home users don't need more than 1 generation of backup, since 95% of their data is static (eg. images). 1 tape a month + cost of the drive amortized over 24 months is only like $150 a month for backups, which is reasonable for what it gets you.

[Hack&Lube]

Originally Posted by Rathji

Tape Backup is so expensive per GB and slow compared to a HDD solution that it really doesn't make sense to do it at home.

I have a couple LTO drives that I am about to mothball at work because we just can't justify the cost for our increased (current and expected) data needs.

What are you going to then? I just destroyed all our LT01s and LT02s and over 3000 DLTs but 3s and 4s are still viable. Doing that alone gave me thoughts of maybe being able to find surplus tapes and equipment from larger companies on the cheap.

[sclitheroe]

Originally Posted by Rathji

Tape Backup is so expensive per GB and slow compared to a HDD solution that it really doesn't make sense to do it at home.

I have a couple LTO drives that I am about to mothball at work because we just can't justify the cost for our increased (current and expected) data needs.

LTO-5 is very close to HD based backup for cost effectiveness, and outruns all but my fastest RAID arrays; I can easily saturate a SAS link with tape, but my 16 spindle NAS can't. No way a single spindle HD solution hits the same throughput as LTO-5.

But at home, no it doesn't make sense

[sclitheroe]

Originally Posted by Rathji

I don't know about Thunderbolt devices, so I won't give any specific product recommendations.

I'd skip thunderbolt as an interface standard for backups - if you had to take your backup device somewhere else to do a restore in an emergency, do you really want to be hunting around for someone else with Thunderbolt? Maybe next gen when its more entrenched.

Better to develop your strategy around a common interface, or via IP networking, which a decent NAS box will do handily.

[photon]

Originally Posted by sclitheroe

At 8TB of anticipated storage, you should be looking at a low end SMB storage product, not consumer grade stuff.

How do you do offsite though then? Do you swap out sets of drives (to avoid the rebuild)?

Or get more than one and keep one offsite?

[sclitheroe]

Originally Posted by photon

How do you do offsite though then? Do you swap out sets of drives (to avoid the rebuild)?

Or get more than one and keep one offsite?

Those Promise arrays will backup to S3 as a two-tier solution. 8TB would be about $80 a month I think in S3 Glacier.

Failing that, you could try and get away with Crashplan or Backblaze, although I don't know how they'd feel about 8TB of utilization - there's unlimited, and then there's the fine print

[photon]

Crashplan you could also do a peer to peer at a computer setup offsite just for that purpose.

I'm thinking more of the initial upload, 8TB is probably many many years worth of work, with my plan I get 500GB transfer, that's 1.3 years to upload

I guess you could get Shaw's Unlimited 250 which has 15Mbps upload and test their unlimited fine print, you could do 8TB in what, a couple of months? Lol...