Thursday, July 9, 2009

More about the Fisher Plaza fire

from the Yahoo MB, thanks to brotherford042 for the answer:

>>Given Internap's SLA, the extent of the outage and the size of the Fisher plaza data center, how much do you expect this outage will cost Internap?

Thanks in advance if you feel like elaborating...

>>No clue how this will ultimately affect inap's bottom line. Historically speaking, the SLA-based refunds have been pretty negligible for us. Then again, given the extent and duration of this particular outage, it will be interesting to see how it plays out.
I did take a look at our SLA. Best I can tell, we are looking at a 4 week refund on our bandwidth (based on an outage of > 6 hours). I'm not positive on this, as the P-NAP itself was only down for like 15 minutes, but since the Fisher facility was totally dark, the colo customers experienced an outage on the order of 29 hours. On the power side, the top tier of the SLA calls for a one week refund on our power/cabinets for an outage exceeding 1 hour. Given that this outage was about 29 times the SLA threshold amount, I'd *hope* they'd consider refunds beyond the stated 1 week.
I suspect alot of the final costs are going to be pretty fuzzy. For example, Randal Thompson said it wouldn't be an issue for us to move our existing cabinets over to the Tukwilla facility on their dime...clearly a cost outside of the remedies called out in the SLA. I supsect it's going to take awhile for this to all play out...suffice it to say there are a number of unsatisfied customers and they are going to have to make some concessions if they expect to retain the existing business. I seem to recall on the order of 120 colo customers impacted based on my earlier conversation with RT.
Furthermore, on the rumor front, I hear that a large deal to take down the bulk of the remaining space in Tukwilla was just aborted. Perhaps a measure to keep the space available to relocate existing disgruntled fisher customers. Again, just a rumor at this point.
Personally, I'm *done* with Fisher Plaza...I can't get out of their soon enough. The way they handled to situation was atrocious on all fronts. I could ramble for hours on the things they *should* have done to keep customers informed and to mitigate the downtime. For example, while the facility is running on backup generators (not even ordered up until about 8+ hours into the outage best I can tell) they aren't redundant...if one of the trailers fail, back we go. They are planning on ordering more and adding the redundancy, but I get the sense that this is out of sheer necessity (they need to service the generators every 10 days), rather than looking out for the interests of the effected customers. Again, last word is that it may be several months until fisher is back on city power.
On the plus side, they've only had one add'l building evacuation since the initial outage :) Unbelievable: http://www.techflash.com/venture/WTF_Die...

No comments: