Google Cloud Storage outage is impacting Scalr services.

Incident Report for scalr.io

Postmortem

On Thursday, June 12, 2025, starting at approximately 2:00 PM EST, our customers experienced service errors and degraded performance. Full service was restored for all users by 5:15 PM EST.

This disruption was caused by a major global outage across Google Cloud services, which our platform relies on. According to Google's post-incident analysis, a recent update to their internal "Service Control" system contained a bug. This bug was triggered by a routine policy change, causing a critical component of their API infrastructure to fail worldwide. This failure resulted in the errors our customers experienced. The detailed RCA can be found here: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

We apologize for the significant impact this had on your operations and appreciate your understanding.

The Scalr Team

Posted Jun 17, 2025 - 16:31 UTC

Resolved

This incident has been resolved.
Posted Jun 12, 2025 - 21:17 UTC

Monitoring

According to our monitoring, less than 1% of requests are currently affected by the Google incident. Majority of the users should see a full recovery at this point.
Posted Jun 12, 2025 - 20:43 UTC

Update

Google Cloud has identified and is mitigating the issue. They have confirmed the issue has recovered in some locations. Scalr is still seeing intermittent issues.
Posted Jun 12, 2025 - 19:22 UTC

Identified

Google Cloud has identified and is mitigating the issue. They have confirmed the issue has recovered in some locations.
Posted Jun 12, 2025 - 19:22 UTC

Update

We are continuing to investigate this issue.
Posted Jun 12, 2025 - 18:49 UTC

Update

Google Cloud storage is experiencing a widespread outage and not accessible.
Posted Jun 12, 2025 - 18:49 UTC

Update

Google cloud storage is currently experiencing issues.
Posted Jun 12, 2025 - 18:38 UTC

Investigating

We are currently investigating this issue.
Posted Jun 12, 2025 - 18:22 UTC
This incident affected: Scalr Platform and Scalr Worker.