Joerg Hiller
Mar 13, 2025 10:35
GitHub experienced two significant service disruptions in February 2025, affecting notifications and migration tools. The company outlines measures taken to prevent future incidents.
In February 2025, GitHub encountered two notable incidents that led to degraded performance across its services, according to GitHub. These disruptions affected both notifications and migration tools, prompting the company to implement corrective measures.
Notification Delays on February 25
The first incident occurred on February 25, 2025, from 14:25 UTC to 16:44 UTC. During this period, email and web notifications faced delivery delays. At the height of the incident, approximately 10% of notifications were delayed by over 10 minutes, while the remaining 90% experienced delays of 5 to 10 minutes. This was attributed to worker pools operating near capacity during peak times, which slowed queue processing.
To address this issue, GitHub scaled out its services to accommodate demand better and established a higher baseline capacity to prevent similar delays in the future. The company is also enhancing its capacity planning strategies.
Migration Tool Outage on February 3
The second disruption took place on February 3, 2025, at 18:01 UTC, resulting in a 30-minute outage affecting GitHub’s Migration Tools. This outage was caused by a deployment issue related to missing Docker images, leading to a complete service interruption for users attempting migrations. The problem was mitigated by reverting to a previous stable version, quickly restoring service.
In response, GitHub has improved its test coverage and workflows to ensure critical dependencies are validated, aiming to prevent similar occurrences.
These incidents highlight the challenges faced by major tech platforms in maintaining seamless service delivery. Users are encouraged to monitor GitHub’s status page for real-time updates and further details on service incidents.
Image source: Shutterstock
Credit: Source link
