Monday, 19 July 2010

Drobo Pro - My New full time job with the time Vampire



Last year filing for my 20 years of photographs was very chaotic

With my library being spread over many different Lacie hard drives of many shapes, sizes and capacities

Lost and failing power supplies were the bain of my life so I took the plunge and bought a Drobo

Which was just fine, apart from the fact it only took 4x HDD of a maximum capacity of 2TB each, so I soon ran out of space

So I bought the Drobo Pro

Over the last 6 months this has degenerated into something of a full time job

I am now on my third

Yes, you did read right, my third

One failed power supply and one directory failure which meant I had Terabytes and Terabytes of space but though I could read the files I could no longer write to the drive

The units were swapped out speedily by Drobo

The real hell is transferring all your data over which takes DAYS, AND DAYS AND DAYS if you have had to reset (erase the drives)

But now

After a busy weekend at the seaside with my daughter I returned to my office

Woke my iMac up which was in sleep mode, after a few seconds the OS sent me the warning that I had removed the Drobo Pro before ejecting it

Which I had not

The Drobo Dashboard software reported that I had used only 55gb of the 12TB space



Infact I had only 3 Tb free

Upon first inspection I see I have 'lost' months of photos from my filing system

See below






Just where are the missing months?????????

I would be having a breakdown if they had 'gone'

But thankfully I have those years backed up in 2 other locations, one on a Drobo (assuming it still works!) and on my old Lacie's

But imagine....just imagine, it does not bear thinking about


And when the fix comes which surely it will,as their service is pretty good, I will have to spend an age working out what is and what isn't missing before beginning the restore process which could take days

The Drobo Pro has been nothing but a time vampire for me, no one should have to face this problem repeatedly

And before anyone chips in from Drobo it is primarily used on 2 computers, my Mac Pro and my iMac

Is there anyone else out there who has similar experiences?

Better still can anyone suggest an alternative to this unsatisfactory situation?

41 comments:

De Seb zijn moeder said...

I had exactly the same problem. Reading was ok, writing not.
Luckily I had everything backed up as was on Lacie harddrives.

Formatted the Drobo again and now it works fine (for now)

really not happy with the drobo...

Drew Gardner said...

Hi De Seb

I'm starting to get the feeling I'm not alone!

It is such a big investment, the idea being it will be trouble free when nothing is further from the truth

It is just how long it takes to fix things after, locating and making sure its all there

The amount of time I spend on it is amazing

How long did it take you to rewrite and check on your files?

Cheers

Drew

Larry P said...

Here's a similar story:

http://terrywhite.com/techblog/archives/1633

Good luck.

Drew Gardner said...

Hi Larry

Seeing you comment finishes with the words 'Best of luck' my blood is turning to ice

Fingers crossed

Cheers

Drew

Edmond said...

Hi Drew,

Sorry to say that Drobos seem to be full of fault; it's just too complex a system and not reliable; the forums are filled with horror stories.

I'd suggest you got RAID 1 drives; simple, effective, safe and robust. I'd also stay clear of LaCie as they have more than their fair share of issues.

Have a look at:

http://photothisandthat.co.uk/2009/12/30/storage-for-photographers/

Good luck :-)

Edmond

typingtalker said...

Consider an HP MediaSmart server.
http://h71036.www7.hp.com/hho/cache/599445-0-0-225-121.html?jumpid=reg_R1002_USEN

Mine sits quietly in a corner doing its thing (automatic backups and storing/serving files). Has been for years. I use Capture One Pro for cataloging and editing images and those stored on the server look and act just like they're on my local hard drive.

Adding a drive consisted of ... just plugging it in. I used the server to restore my local hard drive when I upgraded to a larger one and once again, it just worked. Quickly.

Supports off-site and on-line remote backups.

James said...

Only a few years ago managing a multi-terabyte high availability file store wasn't just a full time job for a single person, it required the attention of an entire IT department. Now, we expect it to "just work", but life just isn't that easy.

Drew, the answer to your problems probably isn't a piece of hardware, it's a suitably trained person that can manage your IT infrastructure for you. It's become a critical part of your business.

Drew Gardner said...

Hi Guys


Thanks for the comments

It seems I have opened a can of worms here

I'm aware of the HP solution and it is a candidate, though I would prefer Drobo would sort me out once and for all.

In answer to James's comment?

Well I'm a one man band, and as diligent as I am, I do not expect data to just 'disappear' with not warning

No clicking hard drive or other precursor warning, it is just 'gone' something I was unfamiliar with until my time with the drobo pro, no matter how chaotic my many drives were.

I was not seeking the magic bullet, just that it did what it said on the tin as advertised

'simple, safe, reliable and scaleable'

So far it has fallen short on the 'safe and reliable' front

I fully expected a response from Drobo by now but none has been forthcoming, which is rather a surprise

Mike said...

Drew,

Sorry to hear about your troubles. We used the Dell MD3000 for large powerful systems for the government. Not cheap but effective. Set it up with a mirrored RAID scheme and get the Dell Gold service. Your hard drives are hot swappable. All manufactures have an equivalent version.

Back that up with a tape backup system where you do Monthly rotating full backups and daily incremental backups.

If you can afford it it is time to move past consumer grade equipment and get commercial/enterprise grade.

Hope this helps or at least gives you some ideas.

Mike

donorkoskey said...

I only keep 3TBs of data "live". Everything else I just keep two data DVD's (one local, one stored offsite).

It's not that big of a change from the "old days" when the negatives were stored in archival sleeves in fire-proof file safes as far as convenience locating them. In fact I still keep my local copy in a fire proof safe with some of my negatives.

There will always be a trade off; the convenience of quick retrieval matched with the endless days worth of data transfer (coupled with the time and resources needed to do the transfer).

--- OR ---

The need to look up a date, flip open a safe, search through a box of discs, mark the spot where it goes, insert the disc just to access the file.

I take B but that's because I don't trust hard drives after working with them for 20 some year.

Then I just remind myself that it is still just as fast as it would have been 12 years ago when I started doing this, faster really because I don't need to get out the loop, turn on the lightbox, and well you get the picture.

Peter Karlsson said...

Drew,

I've spent lots of thinking about backup, being a professional photg just like you, and I've come to the conclusion that cheap external drive docks, connected via ESATA to the Mac Pro, hosting a good ESATA card, is what is scalable, cheap, and pretty future proof. I just dock any 3,5 SATA disk right into the docks. Offline copies are simple, as well as upgrade. Also a lot quicker than Drobo in tests.

Cheers /P

Drew Gardner said...

Hi Guys

I must say I cannot bear the thought of DVD's

In early days of burning Cd's, I thought I had everything abcked up safe and sound only to discover some years later my new computers would not read them, that horror still lives with me

Tape is one route but it seems very 20th century, but what do I know?

Cheap externals could be it yet, I have not had the best experiences with esata, I fitted a card which worked well for a while then after an Apple OS update was the route of kernel panics which I never got to the bottom of , bad driver apparently....

Cheers

Drew

Tay said...

I use a Buffallo Terastation 2TB Server... I run it in RAID 10 so I get only 1TB of storage.. but complete redundancy of data. Additionaly, the Terastation has the abillity to run its own backups to an external usb drive.. which I run a backup to a WD MyBook !TB drive once a week. its a pretty simple setup, but has given me no problems in the past year and a half. I am almost full now... So I am looking at one of the newer models with 8GB capacity..

Drew Gardner said...

Hi Tay

Thank you for your suggestion

Buffalo had disappeared from my radar and it sounds like you have a safe, well as safe as it can be, set up

Still no reply or feedback of any kind from Drobo following all of this. Not from support or from their PR companies

Remarkable

I would appreciate the offer of some kind of solution from them

Alex Thorpe said...

Hi Drew, Have you ever looked at a company called lime tech? They make something called an unRAID server that has the same hot swappable HHD design as the drobo but is much more reliable. For the same price as a drobo pro you can get a pre-built system with 15 drive bays.

The great thing about these systems is that they protect your data not only if one drive goes out but if multiple drives fail then you still wont loose all your images.

But the best part is that all the parts are replaceable just like a desktop computer so if your power supply goes out you can just pop open the case and put in a new one.

Here is the link: http://www.lime-technology.com/

Have a nice day

Todd Allen Design said...

Chase Jarvis did a great presentation of his backup and workflow system last month. Did you happen to catch it?

http://blog.chasejarvis.com/blog/2010/06/workflow-and-backup-for-photo-video/

Victor said...

I agree with Todd Allen's comment, Chase did an awesome job showing his workflow/backup. As a student/budding photographer, I can't afford a NAS storage system so I'm currently using externals which is not good at all because harddrives fail and stuff, but I was looking at the G-safe website and they have hot swappable NAS storage options which is what I'm looking for. I don't need access all of my data all the time, so I would just swap out the older hds with new ones and store them somewhere safe. I personally would get the G-speed Q because it fits me perfectly. But for now...I will cross my fingers with externals. ><

Darrell said...

I work from three internal drives, one for OS and software, one for working files and one for scratch disk. Externally, I have two Firewire connected hard drives, one for each computer, each configured as RAID1. Then there is a network connected drive, which can double as a media server, also configured as RAID1. I do all the work on the computers, never from the external/network drives. I use Cobian Backup to make automatic, simultaneous backups from the internal working drive to all the external/network drives at the end of each day. Cobian is very light on resources, so I could make it do backups more often, but once a day is enough for me. Brand names of the external/network drives are not important, except that I've had back experiences previously with Lacie and IOGear. The drives inside Iomega Ultramax and Ultramax Plus die within months, but not since I replaced the failed drives with server-grade one.

John said...

Honestly if you use consumer electronics, expect that they will behave like consumer electronics. If this is your business, then take the time and money to invest in a proper business storage server. Yes, it will cost you more, I would estimate 2K usd for a setup from Dell. However this will include overnight shipping of failed parts and direct access to your service rep who will hopefully do their best to protect you. You will likely also want to hire someone to setup the system for you. I know this sounds excessive, and tough to budget, however this is just the basics of an entry level in house data retention system. I would also recommend that less of your data is kept "in production". Retain the last year on your active drives, put the rest onto triplicate tapes and store them in multiple locations. Again, this will cost you. But in the end, when it's time to say "Hey son I lost your childhood photos" it will be worth it.

Kim said...

Drew, whats your time worth? If you dropped 1500 US on the drobo and an additional 5-10k US worth of time, youve just spent the same as you would have on enterprise storage.

It might be worthwhile to look into LTO4 for a long term solution. Back up and retreval take time, but its a safe, long term answer and is what most CG and animation studios use for long term data filing.

From there, considering your business depends on the retention of your data, it might be time to look into a professional NAS/SAN solution as John said previously. maybe a mac NAS http://www.apple.com/server/storage/ , or the dell solution.

I bought my mac pro when I was sick of consumer PCs crashing constantly. Essentially, your going to end up spending money on either your time or hardware.

Sean Bolton said...

Hi Drew

My backup is as follows ;

2 seperate Lacie 1TB drives, USB 2.0.

After each job I initially download to my working drive, an external eSATA Lacie 2TB drive.

After initial sift and selection of images I copy selected files to each Lacie 1TB drive. Later I copy any finished files to them as well.

I often also burn a DVD as well as a further backup.

I also have a networked Lacie RAID 1 drive which holds documents, paperwork and any delivered files to clients.

In several years of using Lacie I have only had one drive fail.

Hope you get sorted out, backup and archiving is a major headache but needs to be done.

Sean

Anders C. Madsen said...

Hi Drew

While I can appreciate that you would prefer to keep your Drobo setup I have to agree with the posters recommending an enterprise storage solution along the lines of a Dell MD3000 or similar. I do a lot of work with the MD3000i and while it has its limitations I think it will serve your purpose well.

A word of caution: Should you decide to ditch the Drobo it is important to make sure that the solution you select has drivers for all the OS'es you use on your computers, and that your network infrastructure can handle it, should you opt for an iSCSI solution.

Also, know that enterprise storage in general is "raw" storage that can be used as an ordinary hard drive and nothing more - no backup features, no FTP-service, webserver, user management and such. Plain stupid storage like a normal USB drive - just one helluva lot bigger and more resilient.

I just checked and the website pricing for a MD3000i with 14 500 GB SATA drives is a little less than 9K GBP (ditch the enterprise licensing, you will not need it), and I bet you that a phone call to Dell will bring that price down considerably.

If not, I think IBM has a similar model. Or HP. :)

While it is still a lot of money, we are talking 6 TB of storage in a RAID5 configuration with one hot spare disk for added safety. On top of that, you can add two MD1000 enclosures with a similar amount of space, all run through the controller on the MD3000i. 18 TB of data with no additional management sounds like just what the doctor ordered for a one-man show.

Nick WB said...

Drew,

Long time no see...

Drobos are long known to crocks of ..... Whatever they write on the box, they are consumer toys.
With the level of data you have you need a very serious Raid 6 or 10 solution (not Raid 5).

Speak to Paul Ellis, the digital plumber (AoP) - he will recommend a solution that works.

For anyone else reading this, the words LaCie and Data security are not compatible. Buyer beware!

Nick W-B

Ali said...
This comment has been removed by the author.
Ali said...

Drew i would sugest going with an enterprise solution. like a Lacie big rack, G tech G-speed, HP storage works etc..

Run the raid array to a server Xserve or windows server of your choice with fiber channel. Server has dual Gigabit lan and iscsi.

Im sorry to say drew this type of setup is the only thing that will be reliable and scalable the way you need it to. These types of setups have been used for years in one form or another, so they work better than any consumer product ever will.

Your looking at £3k - £5K to have safe reliable 24/7 availability. You can shave a allot of if you cut down the amount of data that is always available.

John D said...

Local storage only works easily until you need more space provided than a few drives can store. About 4-6 TB right now. Solutions that support more than that amount come in two types: cheap, "simple" and unreliable, or expensive, complicated and reliable. As the old saying goes: "Cheap, simple, reliable. Pick two."

One option that I'm surprised hasn't been mentioned yet is cloud storage. Even if you stick with local hardware for storage you should seriously consider using a cloud storage provider for additional redundancy. The only downside is transfer time because upstream bandwidth is limited for most home internet connections.

Here are few examples to get you started:

Amazon S3 - http://aws.amazon.com/s3/
BackBlaze - http://www.backblaze.com/
JungleDisk - https://www.jungledisk.com/

saperboy said...

Hi Drew, i don't have any experience with the Drobo, however i could recommend to you the NAS that i have been using since 2007, ReadyNAS NV+. It has 4 drive bays that can accommodate, i believe more than 2TB each. I currently have three 1TBs and plan to add in one more 1TB hdd. I am using it for timemachine backups for my 3 macs. It also serves as a location for my other media files (photo, audio, video) that's being served into my minimac that works as a media server (ala HTPC). I ran my ReadyNAS in X-Raid mode that keeps the data redundant, meaning if 1 of the hdd dies, data is still kept intact. There's a newer (more expensive) version which is the ReadyNAS Pro. I think this one has 6 hdd bays, which allows you to have 2 hdd dying on you and your data is still intact. The ReadyNAS forum is very active and supportive, coming from members and the admins themselves. This NAS can be configured such that you can access your data even if your away from your home, speed depends on your subscribed down/upload from your provider. Since 2007, i never had any issues with this. ReadyNAS allows you to choose the file protocol you wish, AFP for apple environment, CIFS or SMB for windows environment and apple supports this too, FTP, HTTP, HTTPs. It's hot-swappably (insert/remove hdd without powering down).

Faolan said...

I concur with the commenter above about ReadyNAS, they are expensive but they are designed for uptime and business users. They are amongst the fastest NAS systems on the market for transfer speeds (NVX and Pro).

I installed a Pro into a business environment where they are doing content creation and I've yet to hear a peep about the unit, it just keeps going with 6Tb of storage.

Google shows that they consistently earn the highest marks and praise as well. It's just annoying they're expensive. I personally run a NVX for my photographic needs. This is a back up of the 3 drive system I use to preserve my data (main, back up and version control).

Alternatively you could use Windows Home Server (WHS) which works well, though I've not used it with OS X.

I've seen a lot of issues for Drobo over the years so you're not alone.

Data Management is one of the first things I teach new photographers, it's not easy and it's certainly not cheap.

p4pictures said...

Drew I'm going to recommend a Netgear readyNAS. I've had one for years it just keeps on working.

In those years I had a power supply fail - switched out quickly with no data loss. I've upgraded the drives - a pain in terms of data copying, but fine otherwise.

For you, the ReadyNAS Pro is probably the way forward. It's networked storage that you connect to a small UPS and it just works - forever!

Drew Gardner said...

Thanks for the comments guys

The options are endless and hopefully we can all learn something from this discussion

Some sort of server seems to be the way to go perhaps

Drobo HAVE been helpful though

Read on.....

Cheers

Drew

Garret said...

You should have a look to the netgear readyNAS pro.

Up to 12 TB of storage (6x2) hot swapping, strong system, really good community with a lot of plugins etc...

Unexpected said...

I use an iOmega IX4-200D with 4TB of storage in a RAID5 config. Dual gigabit LAN, USB, managed interface,... it's aimed towards small companies but works great at home too. You have a 2TB and 8TB edition as well.

Frank said...

Hi drew,
I use two towers with several drives I believe the new one is 8 max.
They use port multipliers to esata.
Use one as main and one as mirror.
Never lost a file and when one hdd fails just replace it because they are seen as separate drives.

Greetings,
Frank Doorhof
Www.frankdoorhof.com

Eugene said...

I have the SansDigital 5 Bay Hardware RAID box and the Drobo and with the SansDigital 3/4 full, I am considering a second unit. Yes I can not plug in a larger capacity drive on my existing RAID; however, a new one with eSATA and USB 3.0 interface for $349 and the Drobo S 5 drive unit $799 with a $100 coupon that expires today I can not justify the increased expense for the Drobo S. For $600 I can get another Sans Digital RAID 5 fully poputlated for less than the empty Drobo S (Hitachi 1TB drives for $50 each). I also can go with Hitachi 2TB drives for $110 each. I think I will purchase another Sansdigital over a 2nd Drobo.

Eugene said...
This comment has been removed by the author.
Drew Gardner said...

Dear Eugene,

Cheaper than a Dobo I think, let me know which way you go.

I have just switched to the QNAP TS859 which is VERY good.

Much faster than the Drobo and super stable too

I will be blogging about it in the near future

Regards

Drew

Charlie Clift said...

Hi Drew, I am looking at getting a server based large storage solution and was wondering what you ended up going for? My internet searching has given me so many options it would be good to hear what you went with.

Simply I need a lot of space, duplicate copies and a server that runs well on Mac OSX. Did you manage to tick these boxes?

Kind regards,
Charlie
www.charliecliftphotography.com

Drew Gardner said...

Hi Charlie

Thanks for getting in touch

Before I say another word...FANTASTIC pic of the man with the cat(s) on your website.

A cracking shot.

OK, storage.

I still hold the same opinions I did when I posted about the Drobo.

I have switched to a QNAP TS859

It has proven to be super reliable, you just turn it on and it works.

Day in, day out, without any drama whatsoever.

I forget it is even there most of the time.

Pros

It is faster than the Drobo

Super reliable.

It has industry standard 'non proprietary' software which means in the case of a failure someone could help you out much more easily than with a proprietary software solution.

It is feature rich you can do all sorts of weird and wonderful tasks with it your own cloud NAS etc

Cons

It is more difficult to set up out of the box than the Drobo

It does not look quite as good as the Drobo.

Conclusion

I cannot recommend it highly enough

At least 8 people have bought one on my recommendation and they all say much the same as I do

I hope this helps


Regards

Drew

pgsalmon said...

I've had a Drobo V.2 for a few years. Attached to a Droboshare. I have 2 1.5 T and 2 2 T drives on which I have 2.2 T's of data. Twice I have had to reset(reformat) as the device would not be recognized on my network. Thank goodness I had backed up on a few spare drives.
Support from Drobo was sketchy at best.
Speeds are as reported,..slow.
Overall experience, a bit negative.
Especially when compared to my Thecus N4100pro, 3 years and not a peep.

Charlie Clift said...

Hi Drew,

many thanks for you reply. I'm pleased you enjoyed my cat image, it was great fun to photograph!

Sorry for my late response, I've been all over the place trying to arrange moving to London.

I saw your post about the QNAP and it certainly seems like a great solution. This is the kind of thing that I am looking to purchase I think. I just need to earn enough money to be able to...but that's another matter. Server storage just seems like a great solution to my constant battle with harddrives and backing up.

Kind regards,
Charlie

Nick said...

.You put really very helpful informaI am pretty much pleased with your good work.tion. Keep it up. jobssite.ca