Slow loads and degraded mobile app usability
Incident Report for Asana
Postmortem

Incident: On June 28, 2024 between 0334  and 0356 UTC, network maintenance was performed to remap internal endpoints, with unforecasted negative effects on communication between internal services. This caused a large amount of requests to Asana during the maintenance period to either drop or time out.

Impact: App usability was degraded for US-based users, resulting in load times of up to a minute. Users of the mobile app in our US, European, Australian, and Japanese data centers were also affected to a lesser extent by elevated error rates.

Moving forward: We've identified process gaps during internal retrospective. Moving forward, we'll perform internal endpoint remappings more incrementally to avoid app disruption, and will apply a more thorough approval process for similar maintenance procedures.

Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.

Posted Jul 03, 2024 - 18:18 UTC

Resolved
Incident: On June 28, 2024 between 0334  and 0356 UTC, network maintenance was performed to remap internal endpoints, with unforecasted negative effects on communication between internal services. This caused a large amount of requests to Asana during the maintenance period to either drop or time out.

Impact: App usability was degraded for US-based users, resulting in load times of up to a minute. Users of the mobile app in our US, European, Australian, and Japanese data centers were also affected to a lesser extent by elevated error rates.

Moving forward: We've identified process gaps during internal retrospective. Moving forward, we'll perform internal endpoint remappings more incrementally to avoid app disruption, and will apply a more thorough approval process for similar maintenance procedures.

Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.
Posted Jun 28, 2024 - 03:30 UTC