ElekTriC schreef:tfw2005 down?? kan er al 3 dagen niet op...
Ja... meerdere Transformer sites, waaronder TLF. Er is een 'Transformer' ontploft in een datacenter in Houston waardoor er 9000 servers zonder stroom zitten. Dat zijn misschien wel een miljoen sites. Is al sinds zaterdagavond. Gister was TLF voor korte tijd bereikbaar, maar nu is hij weer offline. Erg vervelend allemaal, aangezien er weinig bekend is wanneer alle problemen op gelost zijn. En dat vlak voor BOTS........
Jazz schreef:Das GVD klore ja! ben ook al 3 dagen bezig om er op te komen! Hopen dat het maar snel opgelost is!
31-5-2008 is er een gebouw ontploft waar de servers staan....gelukkig zijn der geen doden gevallen...
gister avond hadden ze 90 % weer online van de 9000 servers...
toen men de laaste 10 % ging doen hebben de airco's van de server kasten het begeven...het werd te warm...de hele boel is weer vastgelopen
vandaag rond de klok van 13 uur nederlandse tijd waren ze op de helft van de opstart procedure van stap 1
het heeft totaal 3 stappen nodig...om alles weer online te krijgen...
ik zelf verwacht ongeveer dat donderdag alles weer helemaal werkt zoals het was...
dit is het bericht namens de server...
Good Evening Everyone,
My name is Doug Erwin, and I'm the chairman and CEO of The Planet. In light of this weekend's past events, I felt compelled to do a recording for all of you to try to put a voice behind all the letters. I wanted to offer my personal assurance that we at The Planet are committed to completing the restoration of our data center and getting it back to normal operations.
We've been conducting our own investigation into exactly what caused this explosion in H1, our data center. And I thought I would tell you what we know since many of you have been asking. I do not know if we will ever be 100% sure, but here's what we've found:
It appears there was an electrical explosion in the underground conduit that brings utility power into the data center. The explosion and electrical fire damaged, beyond repair, the electrical gear where the utility service enters the building as well as the transfer switch and main distribution panel that feeds the first floor of the data center.
The damage was unbelievable.
We are lucky that no one was in that room or near it when the explosion occurred. Just to personalize the situation, one of our technicians who typically does his perimeter patrol at the time of the explosion was late, helping a customer. We've all heard stories: "If we had been in that particular place just a few minutes earlier, our lives would have been different." How fortunate we are that this young man was helping a customer. It gives me great pause because the server destroyed all four walls of that electrical room where he would have been.
As you know, we worked closely with the fire department, and they did not allow us to use our onsite backup generators for safety reasons, which we understood. As a result, our team jumped immediately into action and called in help.
We brought in temporary generators that are being fueled twice a day right not to run phase 1, and we managed to power up our own onsite generator which is now operating phase 2.
There are over 80 people on-site to get our data centers operational again for you. These people are my personal heroes. I only wish you could see the pace of what's happening and the attitudes of determination of the team there ... They are singularly focused on their work, which is going on around-the-clock. I couldn't be more proud of what they've been able to accomplish.
[Specific thanks to several groups going "way above the call of duty"]
Tremendous credit goes to internal Planet teams also who were on site. Our facilities staff, our data center operations, networking, logistics, and all the Dallas people who made their way to Houston to help. The list is long, and I'm slighting many, who worked so hard, by not mentioning their names, but I don't have the time to do that right now ... I know who they are, and I appreciate all that they have done.
If you would have told me 53 hours ago that we would have all the power back to that data center and the 9,000 servers powered up, I would have had you locked up. This is nothing short of a small miracle.
We have more work to do, and I want you to know what our next three priorities are for the upcoming 48 hours:
We need to stabilize the environment and work with the customers who might still be having problems.
We need to turn our attention to the rebuilding of the electrical room and acquire all the necessary so that we can run the operations to our high standards.
We need to develop a plan that minimizes the impact to the 3,000 servers we have in phase 1 when we do the final cut-over to the rebuilt facilities.
After that, I owe you an answer as to how we will handle the SLA credits we need to distribute — When it will get done, how it will get done — I owe you those answers after I finish the aforementioned priorities.
I'm sure there will be other things we need to do, but we'll keep you informed, as we have been doing, but we're going to change a little and not be posting updates every hour ... now it will be four times a day.
We're not out of the woods, but we feel good about where we are.
I realize this has been a horrendous experience for you. There's nothing I can say or do to give you back the time you lost for your business. It is an unfortunate scenario, no matter how you look at it. Many of you have been very generous in your comments about our team, and I assure you that it has been appreciated by everyone.
We will continue to earn your trust as we complete these renovations.
Thanks again for your business.
Douglas J. Erwin
Chairman & Chief Executive Officer