We are moving 55 servers, mostly Dell PowerEdge 2850's 30 miles over a mountain road.
-
Not enough data. How are they being packed? How bumpy is the road?
If you're using reasonable caution, you probably won't see any problems, with reasonably new servers, but you can always lose a hard drive. I'd make sure my backups were up to date, and the servers were well padded and secured.
@lombardm
I'd be worried too. Almost all of those things can shorten the life of your HDDs. Definitely make sure of your backups. Still, in all likelihood, this will only (again) shorten the life of those drives. Most of them are going to weather it fine.
lombardm : The servers will be moved on air ride trucks, but the road is bumpy.These servers are not new, and have been in a sketchy environment. There are regular power events and no UPS. The room's AC is not balanced very well either. I am a consultant project manager and am hoping that a 10% loss rate is reasonable.Bart Silverstrim : If budget and time allow, you could try getting some new drives and add them to the array to rebuild them, then you'd have newer drives in the array to guard against unexpected failure. It's generally a bad idea to have multiple drives in an array of the same age anyway, as they may all fail near the same time...but that's just another idea.Farseeker : Hmm, I doubt even the worst corrugated roads in outback Australia are any match for the delivery guy I saw literally chucking Dell branded boxes from the back of his truck onto the loading dock, or the day I saw a courier do an underarm throw over a fence to a front door. If it can survive that, it can survive anything (with the correct packaging, of course)!From Satanicpuppy -
How old are the servers? We had some old (5 year) video storage servers that had about a 10% hard drive failure rate when we had to just shut them down for a few hours for power maintenance. We figured the 24x7 fatigue on the hard drives finally caught up when they were spun down and then fired back up. Something we definitely didn't expect from performing a soft shutdown/startup.
Michael Kohne : Anecdotally, this doesn't seem uncommon - there's lots of stories of systems that have been running fine for long periods of time losing drives on a routine power cycle.einstiien : yeah, definitely something I will add into the budget for a power distribution upgrade in the future.From einstiien -
In all likelihood if you're not really beating the servers up during the move you probably won't see any failures, but to be safe I would say expect a few hard drive failures, so have spare drives available to swap in for RAID rebuilds (your drives are in RAID arrays, right?) and make sure your backups are up to date (you are running and testing backups, right?).
Also like Satanicpuppy said, make sure the servers are packed well for transport - If you're just chucking them in the back of a pick-up truck and doing 80 over potholes and gravel roads your failure rate will obviously be higher, to say nothing of the servers that might bounce out of the bed along the way :)
From voretaq7 -
You don't mention how long the drives have been in use, how old they are, etc. The longer they've been running 24/7 the greater the chance they won't spin back up.
The transport could be rough on them if you're talking about a long haul over potholed roads.
If you're very worried...
Have backups to tape. Backups that can restore from bare metal.
Buy some spare drives. You will most likely have a few die. If not now, definitely down the road :-)
Pack everything very well, but label and remove each and every drive and pack them separately and transport them separately from the servers/racks. Hand-transport them in your car with foam lined cartons, then plug them all back into the proper server and drive slot, so that hopefully they'll take less punishment in your car than in the back of a transport truck. Some would probably argue this is overkill, however.
Travel with the backups and servers separately. An accident or incident shouldn't kill both your data systems and your backups.
mh : +1, like you read my mind!Oskar Duveborn : +1 Especially like the part about separating the data from the server transportMichael Kohne : Eek. While separate packing of the HDs does make some sense, balance that against the likelyhood of something going wrong during re-assembly. With all those servers I'd be more worried about things getting put back together wrong than with losing a drive due to it being shipped in the chassis.Bart Silverstrim : @Michael: it's a risk, yes. That's why I said it could be overkill. But at the same time, the drives could fail due to being loosened from the interfaces during transport, or even temp and pressure changes during transport on a mountain road. Personally, I'd say it's a wash.Bart Silverstrim : @Michael: I'd also think that transporting within the servers themselves will generate more vibration and inertial wear, as there's not much "give" in most transport trucks, racks, and server mounting hardware. That means every shake and rattle gets transferred to the drives. If you remove the drives and pack them in a cushy, warm car, with foam or padding around them, chances are they'll take less of a beating in the haul compared to those poor CPU's.mh : "With all those servers I'd be more worried about things getting put back together wrong" - that's why you label them properly. I'd consider it a wash too.Satanicpuppy : Agreeing with Michael. No way would I disassemble the machines. Not only do you have potential re-assembly problems, but the drives are not going to like being pulled out of the machines, jammed in the boxes, moved, then pulled out of the boxes and shoved back into the machines any more than just riding along in the machines.Bart Silverstrim : I'd also think it's a wash, but it's an idea, that's balanced by the fact that we don't know what this terrain is like where they're travelling, the trucks being used to transport them, how old the servers are and how long the drives have been in use, and if they're marginal enough that removal and replacement (or spindown/powerup) causes a failure, he's got marginal server reliability as it is and a problem waiting to be exposed. Backups and spare drives are definitely more important in my book than debating drive removal and whether the conditions warrant babying them on a trip.mh : @Satanicpuppy: Have you seen a 2850? They're designed for easy drive removal. No substantive concerns there.Satanicpuppy : @Mh: Sure, in fact I have a couple downstairs. Still it's motion outside of their normal range, conditions outside their normal conditions, and I'd feel safer (were they mine) with the drives still locked in place in the server.From Bart Silverstrim -
The most likely components to have trouble are the ones with moving parts - HDs, PSUs and CPU fans. It's also possible for cables and even PCI cards to work loose in transit.
What I'd do is pop out the components in question, box them separately (using lots of bubble wrap) and transport them in the boot of someone's car. Making sure to clearly label what belongs where of course (goes without saying). When reassembling all cables and cards then get an extra nudge to ensure that they're seated correctly.
Having said all of that, I've placed Dell servers at the tender mercies of courier companies before, and on longer trips down twisty narrow and hilly countryside roads, and they've always come up smiling.
From mh -
I can't speak towards the 2850s specifically, but the 1950s are quite resilient and have survived far more than some bumps on a road.
From Zephyr Pellerin -
Is the mountain made of magnetite?
Farseeker : I'm not going to upvote the answer because it's not actually helpful, but it did make me laugh out loud for realNathan Long : Thanks for letting me know. :)From Nathan Long -
If you care about the data (and it sounds like you do) do NOT move them. Buy new hosts somewhere else and migrate the data. Those machines are beyond their usable life expectancy. Do some calculations on the power savings that you'll get by purchasing new R710s. You'll likely only need to buy 7 of them to replace all 55 of those 2850s. 2850s were power HOGS, Dell's capacity planner says they draw over 500 watts each. My real world experience with the R710s (dual L5520, 72GB RAM and eight 73G 15k 2.5" drives) are drawing 220 watts when in use (120 is common when idle.) 55x500 = 27,500 watts, 7x220 = 1540 watts. If power is $.15/kwh, you'll pay $2970/mo for the 55 servers or $166.32/mo for the 7. Plus cooling factor (which can easily double that cost.) If you can get a decent Dell lease those new servers, you'll come out ahead over the life of the R710s.
joeqwerty : A for effort, but really? Advising the OP to buy all new hardware and migrate all of the data, applications, etc. is hardly a suggestion I would take seriously.toppledwagon : I've had to make similar justifications to C-level management, they love this sort of thing. If you have 55 servers, you have some sort of configuration management in place so re-creating everything will be next to trivial. What part of this would you not take seriously?From toppledwagon -
We recently moved 50+ Dell servers (1650, 1850, 2850, 1950, 2950, etc.) and various other components (switches, firewalls, etc.) in two racks to a new data center 30 miles away (no mountains involved). We secured all of the equipment in both racks, wrapped the racks with moving blankets, strapped them in the back of a moving truck equipped with air shocks and had 6 movers move each one for us. When the racks were placed on the data center floor we reseated all hard drives, recabled all of the equipment and, knock on wood, 2 months later we're still running without a blip.
From joeqwerty
