AWS Outage Cheat Sheet

Amazon Status Translation Sheet

===============================


Single Availability Zone : All Zones in a region

Degraded Performance : Total service disruption

Elevated Error Rates : No one can connect


Green Tick : Up

Green Tick with (i) : Total Service Outage

Yellow Notice Symbol : Total Service Outage

Red Tick : The guy who selects this for reporting statuses of any kind will get fired.



————-

Log for 10/22/2012 outages / failures:


10:38 AM PDT We are currently investigating degraded performance for a small number of EBS volumes in a single Availability Zone in the US-EAST-1 Region.


Translation: Some high profile sites such as Netflix, AirBnB, Reddit, Heroku, Coursera are currently down down to service disruption.


—————


11:11 AM PDT We can confirm degraded performance for a small number of EBS volumes in a single Availability Zone in the US-EAST-1 Region. Instances using affected EBS volumes will also experience degraded performance.


Translation: All of our EBS volumes across US-East-1 were down.


—————-


11:26 AM PDT We are currently experiencing degraded performance for EBS volumes in a single Availability Zone in the US-EAST-1 Region. New launches for EBS backed instances are failing and instances using affected EBS volumes will experience degraded performance.


Translation: US-East is Fubar’ed.


——————


12:07 PM PDT We are experiencing elevated error rates with the EC2 Management Console.


Translation: No one can connect anywhere in the world to the Management Console


——————


1:02 PM PDT We continue to work to resolve the issue affecting EBS volumes in a single availability zone in the US-EAST-1 region. The AWS Management Console for EC2 indicates which availability zone is impaired.


Translation: EBS is still broke, and you guys can’t log in to the AWS Console to check on which AZ is down anyway


——————-


2:06 PM PDT We are continuing to see elevated error rates with the EC2 Management Console.


Translation: Worldwide AWS console hosted only in US-EAST is down


——————-


1:02 PM PDT Customers can launch replacement instances in the unaffected availability zones but may experience elevated launch latencies or receive ResourceLimitExceeded errors on their API calls, which are being issued to manage load on the system during recovery. Customers receiving this error can retry failed requests.


Translation: You have zero chance of being able to launch new instance via the AWS Console which is still down, or the API. Keep retrying though.


———————


2:20 PM PDT We’ve now restored performance for about half of the volumes that experienced issues.


Translation: We’ve fixed the cluster of servers running the AWS Console.


———————


Customers with ELBs running in only the affected Availability Zone may be experiencing elevated error rates and customers may not be able to create new ELBs in the affected Availability Zone.


Translation: Elastic Load Balancers are dropping all requests.