Saturday, July 11, 2009

Fisher Plaza - Internap's info

from Locus Lingua:

>>(Reposted with permission from Internap)
Near Term:

Proceeding immediately, the landlord will 1) obtain two additional generators and paralleling gear needed to provide redundancy across the 3 central risers affected by the outage. This will provide N+1 until we can return to full commercial power and allow required maintenance on the generators (required every 10 days) without any downtime. The city has approved closing the east end of John Street for the location of the additional generators. The landlord is also working with 2) Seattle City Light to get commercial power from the vault to the Life Safety System and Retail switchboard to provide a more reliable and effective solution to providing a safe work environment.

Long Term Utility Restoration:

The restoration plan involves building a completely new switchgear room outside the current vault and Switchgear Room. This would separate the Main 1 and Main 2 systems providing more reliability, new equipment, and the ability to begin construction while they are clearing out the damaged gear from the damaged Switchgear Room. All of the old Main 1 & 2 switchgear from the damaged switchgear room will be replaced with new equipment needed to provide a new Main1 service and the paralleling gear to bring up all four of the building's 1.5 MW generators on line together.

The landlord stressed that this is a very fluid plan and that suppliers are aggressively sourcing equipment, design, build, and shipping times. Once they are able to develop a specific timeline as well as Method of Operation (MOP) they will share it with Internap, and we will of course share it with our customers. They are expecting several months until full restoration.

Wells Fargo - Equinix Institutional ownership


WELLS FARGO & CO/MN filed this Form SC 13G/A on 07/09/09

Amount beneficially owned: 2,151,570

Percent of class: 5.67%

down from the previous position:

WELLS FARGO & CO/MN filed this Form SC 13G on 04/30/09

Amount beneficially owned: 4,516,936

Percent of class: 12.09%

Friday, July 10, 2009

"several months" before a permanent solution is in place.


>>It was one week ago today that Fisher Plaza's data center facilities were knocked out by an electrical fire and power outage, taking down dozens of Web sites in the process. According to the latest reports from Internap Network Services, the facility is running smoothly on interim power, with two additional mobile generators now on site to provide extra redundancy.

As of yesterday, officials were expecting new electrical switchgear to arrive as early as today, to replace equipment damaged in the incident. Earlier this week, Internap told tenants that the plan is to build a new switchgear room. Internap said Fisher Plaza officials believed it would be "several months" before a permanent solution is in place.

Internap Taps Morrison for CMS Implementation

Internap Network Services Corporation (NASDAQ: INAP), a global provider of end-to-end Internet business solutions has selected The Morrison Agency to implement a CMS (Content Management System) for its global website. The CMS will enable Internap to be much more proactive in making updates and enhancements to its website on a daily basis.

Thursday, July 9, 2009

More about the Fisher Plaza fire

from the Yahoo MB, thanks to brotherford042 for the answer:

>>Given Internap's SLA, the extent of the outage and the size of the Fisher plaza data center, how much do you expect this outage will cost Internap?

Thanks in advance if you feel like elaborating...

>>No clue how this will ultimately affect inap's bottom line. Historically speaking, the SLA-based refunds have been pretty negligible for us. Then again, given the extent and duration of this particular outage, it will be interesting to see how it plays out.
I did take a look at our SLA. Best I can tell, we are looking at a 4 week refund on our bandwidth (based on an outage of > 6 hours). I'm not positive on this, as the P-NAP itself was only down for like 15 minutes, but since the Fisher facility was totally dark, the colo customers experienced an outage on the order of 29 hours. On the power side, the top tier of the SLA calls for a one week refund on our power/cabinets for an outage exceeding 1 hour. Given that this outage was about 29 times the SLA threshold amount, I'd *hope* they'd consider refunds beyond the stated 1 week.
I suspect alot of the final costs are going to be pretty fuzzy. For example, Randal Thompson said it wouldn't be an issue for us to move our existing cabinets over to the Tukwilla facility on their dime...clearly a cost outside of the remedies called out in the SLA. I supsect it's going to take awhile for this to all play out...suffice it to say there are a number of unsatisfied customers and they are going to have to make some concessions if they expect to retain the existing business. I seem to recall on the order of 120 colo customers impacted based on my earlier conversation with RT.
Furthermore, on the rumor front, I hear that a large deal to take down the bulk of the remaining space in Tukwilla was just aborted. Perhaps a measure to keep the space available to relocate existing disgruntled fisher customers. Again, just a rumor at this point.
Personally, I'm *done* with Fisher Plaza...I can't get out of their soon enough. The way they handled to situation was atrocious on all fronts. I could ramble for hours on the things they *should* have done to keep customers informed and to mitigate the downtime. For example, while the facility is running on backup generators (not even ordered up until about 8+ hours into the outage best I can tell) they aren't redundant...if one of the trailers fail, back we go. They are planning on ordering more and adding the redundancy, but I get the sense that this is out of sheer necessity (they need to service the generators every 10 days), rather than looking out for the interests of the effected customers. Again, last word is that it may be several months until fisher is back on city power.
On the plus side, they've only had one add'l building evacuation since the initial outage :) Unbelievable:


Company Name Emailvision
Also Known As
Company Website
Primary ASN 39905
Network Type Content
Approx Prefixes 5
Traffic Levels 100-1000Mbps
Traffic Ratios Mostly Outbound
Public Peering Exchange Points
Exchange Point Name ASN IP Address Mbit/sec
Equinix Paris 39905 connection in progress 1000

Internap peering - Europe

10 Gig connection to LINX, I guess DE-CIX and AMS-IX might be next (curious about France):

Public Peering Exchange Points
Exchange Point Name ASN IP Address Mbit/sec
LINX 22212 10000

Monster data center heads for Scotland


>>Monster data center heads for Scotland

So what is the industry view of this monster-sized data center?

"The scale, as described, is certainly massive, but Equinix won't comment on competitor's plans," said Eric Schwartz, the President of Equinix in Europe.

"What I can say is that there is a component of our industry that tends to make announcements first in order to try and acquire customers, and then execute. Equinix tends to build first and then execute. If these guys intend to open for 2010, then construction would have to start soon," he said.

So what is the timescale to build a new data center? "From idea to opening, we are experiencing anywhere from 18 to 24 months, but this varies depend on planning permission and power," said Schwartz.

Schwartz pointed out that there are emerging sub categories within the data center industry, catering for differing customers demands. "One segment tends to be nearer people [i.e. closer in], which is where Equinix operates, and the other segment is going further afield," he said. "The further out builds tends to be larger, the closer in tend to be smaller."

Wednesday, July 8, 2009

2nd CK center in London open in Dec 2009

from the IV MB, by yyy60:

>>2nd CK center in London open in Dec 2009

London Clinic to launch cancer centre campaign
Source: Marketing Week | Published: 08 July 2009 10:15

The London Clinic is readying a marketing campaign to promote its new cancer centre.

The not-for-profit medical establishment has invested heavily in the centre, which includes new technology, such as a CyberKnife and other non-invasive radiotherapy systems.

The centre is set to open in December this year and The London Clinic has appointed Hurrell Moseley Dawson Grimmer to handle a £500,000 marketing communications brief.

The agency was appointed after a pitch overseen by marketing director Karen Bullivant. Marketing will embrace press, print and online activity and HDMG will also handle media planning and buying.

Neil Dawson says that health care products and properties face a growing marketing challenge as patient demand grows. He adds: “It’s a sector that is going to grow and become much more innovative inn the way it approaches marketing.

** The first CK in London was at the Harley Street Clinic which was open early this year. This is the second CK in London also on Harley St.

Denver CyberKnife Aids in Colorado’s Economic Recovery by Creating 8 New Jobs

from the IV MB, by yyy60:

>>Denver CyberKnife Aids in Colorado’s Economic Recovery by Creating 8 New Jobs

Denver, CO, July 08, 2009 --( Denver CyberKnife, the state-of-the-art treatment device for cancer and tumors throughout the body has aided the economic recovery in Denver through the creation of 8 new jobs. Denver’s first CyberKnife was installed in late spring 2009 and was clinically operational in early June. Although Denver CyberKnife is located in a brand new building and boasts the latest technology, the clinical staff is one of the most experienced in the world. Dr. Gregg Dickerson, Steve Humphries, and Mark Post serve as the radiation oncologist, medical physicist, and radiation therapist, respectively; these professionals have over 15 years of combined CyberKnife experience, making Denver CyberKnife one of the most tenured centers in the world.

Mark Frank, Chief Administrator of Denver CyberKnife, is very happy to be able to be a part of providing the CyberKnife as a community resource. “The CyberKnife System is the ideal radiosurgical tool given its flexibility and great range of clinical applications,” he stated. “In addition to proven effectiveness in treating tumors throughout the body, one of our most important benefits is patient comfort. By eliminating the use of a head or body frame, patient discomfort is significantly reduced. We are very pleased to bring this technology to our patient, as well as do our part to bolster the local economy.”

Google Chrome vs Microsoft Windows

Google Chrome vs Microsoft Windows

Google Chrome vs Microsoft Windows

by Federico Fieni. HT to Seeking Alpha.

Telx and Tata Communications partner to expand geographic reach

from Tier 1 Research daily newsletter:

>>Telx and Tata Communications partner to expand geographic reach

Telx and Tata Communications have partnered to leverage each other's datacenter footprints to expand geographic coverage for customers. Under the partnership, which enables cross-selling, both Telx and Tata Communications will offer one-stop shopping for colocation services across both companies' datacenter footprints. The partnership permits US-based Telx to offer colocation services .....

Internap finally reached out to me with a call today

from the Yahoo MB, post by brotherford042:

>>Internap finally reached out to me with a call today

Internap finally reached out to me with a call today...spoke w/ Randall Thompson for about 15 minutes. I suspect he's having a worse week than me having to make the rounds with the effected fisher customers. BTW, I also got a boiler plate email from Cooney as well.

Still no eta on an RFO or a definitive timeline on when city power will be restored. Fisher stated "They are expecting several months until full restoration"...yikes! The facility is still currently running on (non-redundant) trailer generators. They are going to order more generators to add redundancy. IMO, that decision should have been a no brainer out of the gate. Now is certainly not the time for them to be conserving pennies...

Fisher is the real culprit in all of this. Those guys are *terrible*. All the same, its particularly frustrating communicating the situation with our customers when upstream updates since the outage have been sparse to non-existent.

Tuesday, July 7, 2009

SimpleHelix Internap CDN Partnership

SimpleHelix has partnered with Internap Network Services Corporation to bring you the best CDN access available. We are currently developing ways to improve our customers Magento stores with CDN access such as images offloading and live streaming video.

We will be launching a full suite of CDN based products come August 2009. If you have any questions or would like to beta test our CDN access, please submit a support ticket here. Hosted Exchange


>> Hosted Exchange

According to one hosted Exchange provider – Burlington, Mass.-based, “As a small or mid-sized business, it is important to look for a reliable and reputable hosted Exchange provider to ensure data security, scalability and guaranteed uptime which will help take your business to the next level of productivity and growth.”
But what are the advantages of using Microsoft Exchange as a hosted service? Like other SaaS (News - Alert)-based solutions, outsourcing e-mail communication to a hosted Exchange provider as opposed to deploying Microsoft Exchange in-house can save companies tens of thousands of dollars in costs, including the purchase of new hardware and software, maintenance support contracts, storage costs, hiring and training new personnel, ongoing administration and maintenance costs and system monitoring, including anti-virus and anti-spam filtering.
With a hosted Exchange provider, a company could be up and running with the latest version of Microsoft Exchange Server in minutes. (Consider that it could take weeks or months to implement an in-house solution.)

The company uses Internap’s (News - Alert) Tier-4 data center and its “Performance IP” service which reduces latency and routes customer data over redundant, high-speed backbones to provide faster delivery of e-mails around the world. It also uses the Microsoft HMC platform – the only platform design, tested and approved by Microsoft for Exchange Server hosting.

Telx joins Equinix in winning ISE consortium RFP for low-latency interconnection

from Tier 1 Research daily newsletter:

>>A few days ago, T1R wrote about Equinix's win of the International Securities Exchange (ISE) business in the New York area.
What T1R didn't know at the time is that there were two winners in the ISE RFP process – one was Equinix, while the other was Telx.

Sunday, July 5, 2009

Colotastrophe: The Day After

from Locus Lingua blog, a few addition infos from a customer's point of view:

>>Colotastrophe: The Day After

Our servers are a bunch of primadonnas. They demand to be pampered in the greatest colocation facility in the world (if you agree with the video of Fisher Plaza touting that fact), resting on pillows of AC and fed power in Waterford crystal goblets. We literally pay more for the 5 cabinets that house the servers* than we do our entire Groundspeak office - and then some.

To set the stage, we have been hosted at Internap in the Fisher Plaza since 2002 and in that time have only had 2 significant events that related directly to facility issues. The last issue lasted around 8 hours while this one is, by far, the most signficant downtime in the history of the web site. In total we had 29 hours of downtime. Unfortunately the 29 hours were during the geocaching peak season on the busiest weekend of the year and, to compound things, a day off from work for many. The Fates were definitely conspiring to pick the worst day to bring the site down.

What Didn't Work

Although I won't finger point at the cause of this issue, I will point out that Fisher Plaza people lacked any official communication with the first responders at the scene. Many clients of the building were in the dark, both figuratively and literally, while we were waiting outside for news of what really happened. Instead we had to join in on Twitter to figure out what happened. Was it a fire? (yes) Did the sprinklers turn on? (yes) OMG! Our machines are fried! (no. just the generator) If someone walked out of the building with some authority and told us what they knew - we could have passed that information on to our customers. Internap did a relatively good job at giving status updates though they were sparse and sometimes repeated. I'd give Internap a C and Fisher Plaza an F for communication.