this post is regarding the downtime of my webby since Sunday because of an explosion.
this is quite a dramatic story..
From: Douglas J. Erwin, Chairman & Chief Executive Officer of The Planet
As previously committed, I would like to provide an update on where we stand following Sunday’s explosion in our H1 data center. First, I would like to extend my sincere thanks for your patience during the past 28 hours.
As you have read, we have begun receiving some of the equipment required to start repairs. While no customer servers have been damaged or lost, we have new information that damage to our H1 data center is worse than initially expected. Three walls of the electrical equipment room on the first floor blew several feet from their original position, and the underground cabling that powers the first floor of H1 was destroyed.
There is some good news, however. We have found a way to get power to Phase 2 (upstairs, second floor) of the data center and to restore network connectivity. We will be powering up the air conditioning system and other necessary equipment within the next few hours. Once these systems are tested, we will begin bringing the 6,000 servers online. It will take four to five hours to get them all running.
It appears there was an electrical explosion in the underground conduit that brings utility power into the data center. The explosion and electrical fire damaged, beyond repair, the electrical gear where the utility service enters the building as well as the transfer switch and main distribution panel that feeds the first floor of the data center.
The damage was unbelievable.
We are lucky that no one was in that room or near it when the explosion occurred. Just to personalize the situation, one of our technicians who typically does his perimeter patrol at the time of the explosion was late, helping a customer. We’ve all heard stories: “If we had been in that particular place just a few minutes earlier, our lives would have been different.” How fortunate we are that this young man was helping a customer. It gives me great pause because the server destroyed all four walls of that electrical room where he would have been.
As you know, we worked closely with the fire department, and they did not allow us to use our onsite backup generators for safety reasons, which we understood.
We’re not out of the woods, but we feel good about where we are.
I realize this has been a horrendous experience for you. It is an unfortunate scenario, no matter how you look at it. Many of you have been very generous in your comments about our team, and I assure you that it has been appreciated by everyone.
then qiao yi and ren wei messaged me at 5 just now to ask if my blog was hacked. i thought she meant the server downtime because of the above incident. i didn’t realise that it was really hacked until i tried logging in myself. well, apparently it was hacked by some stupid saudi arabian as*hole, making use of a loophole during the initial server reboot. omg, hackers are such a pain in the ass…
back to life: had a 2½ hour physics lecture today, which was über boring. after that we had lunch at tiong, followed by a childish time at the tiong playground. =D
shoutbacks!
- mich // you’ve great taste. lol.
Edited: Wednesday, 4 June 20088:36 pm
i'm so jealous. haha.
