App Down

Incident Report for Asana

Postmortem

Incident: AWS infrastructure had an outage affecting core services, which our application depends on, in the Virginia (us-east-1) region. After partial recovery within AWS, we made configuration changes and saw recovery for our traffic. We continued to observe while AWS worked through the incident on their side. The remaining impact on Asana's side is covered in https://status.asana.com/incidents/lvtt14d4d80g .

Impact: We saw global downtime from 2025-10-20 06:52 UTC through 09:27 UTC, 2 hours and 35 minutes. This affected all regions because we store some global state and shared application configuration in the Virginia region.

Moving forward: We are working on ways to reduce the impact of single region outages, including for this region, to limit the impact and improve options for recovery.

Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.

Posted Oct 23, 2025 - 16:06 UTC

Resolved

This incident has been resolved; error rates and latency have gone down and we've caught up on existing traffic. We will publish a postmortem here within 72 hours.
Posted Oct 20, 2025 - 10:23 UTC

Update

The root cause has been fixed, and we are monitoring recovery; we still see somehow degraded performance in the US region, and are adjusting configuration to address this.
Posted Oct 20, 2025 - 10:08 UTC

Monitoring

A fix has been implemented, and we are monitoring recovery.
Posted Oct 20, 2025 - 09:32 UTC

Update

We are working to prepare our systems to recover once AWS resolves DynamoDB issues in us-east-1.
Posted Oct 20, 2025 - 09:18 UTC

Update

We are continuing to work on this issue, and will update once we have an ETA for resolution.
Also see https://health.aws.amazon.com/health/status for status on resolution for infrastructure issues.
Posted Oct 20, 2025 - 08:32 UTC

Update

We are continuing to work on this issue. AWS DynamoDB is disrupted, and this impacts our production services.
Posted Oct 20, 2025 - 08:19 UTC

Update

We are continuing to investigate this issue.
Posted Oct 20, 2025 - 07:59 UTC

Update

We are continuing to investigate this issue from our side, and waiting for an update from AWS.
Posted Oct 20, 2025 - 07:43 UTC

Update

We are still seeing full downtime, and are continuing investigation.
Posted Oct 20, 2025 - 07:30 UTC

Update

AWS is investigating errors in Virginia, per https://health.aws.amazon.com/health/status . We are continuing to investigate and will provide an update in 15 minutes.
Posted Oct 20, 2025 - 07:15 UTC

Investigating

We are currently investigating an issue which affects all services.
Posted Oct 20, 2025 - 07:10 UTC
This incident affected: US (App, API, Mobile, Automations and Background Actions, Webhooks and Event Streams, Notifications), EU (App, API, Mobile, Automations and Background Actions, Webhooks and Event Streams, Notifications), Japan (App, API, Mobile, Automations and Background Actions, Webhooks and Event Streams, Notifications), and Australia (App, API, Mobile, Automations and Background Actions, Webhooks and Event Streams, Notifications).