SUMMARY

On July 17, 2023, between 18:35 and 21:17 UTC, Atlassian customers using Confluence Cloud experienced longer than expected loading times or failed loads to specific experiences. The event was triggered by an issue with a new code deployment. The impacted customers were limited to the prod-east and prod-west2 regions. The incident was detected within 10 minutes of deployment by automatic monitoring systems, and mitigated by rolling back the deployment which put Atlassian systems into a known good state. The total time to resolution was about 2 hours and 42 minutes.

IMPACT

In Confluence Cloud, customers in the impacted regions may have experienced reduced functionality, slower response times, and limited access to initial loads of specific experiences including viewing of Confluence pages, space overview pages, and the home page.

ROOT CAUSE

The issue was caused by a new code deployment. After identifying the problem, we quickly reverted the change to mitigate impact to customers.

REMEDIAL ACTIONS PLAN & NEXT STEPS

We know that outages impact your productivity. We are prioritizing the following improvement actions to avoid repeating this type of incident:

• Extending internal testing periods under load before being deployed to production environments  
• Enhancing automated post-deployment anomaly detection of platform health
• Optimizing automatic rollback of new deployments that fail automated health checks
• Enabling further internal automated production fallbacks to ensure reliability at all times

We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability.

Thanks,

Atlassian Customer Support