Microsoft blames July software bug for 24-hour global Microsoft 365 outage
Microsoft's preliminary report says a latent bug in a networking gateway update, set off by extra traffic and reduced capacity, took down Exchange Online and other services worldwide.
By Mr Elite Published 7 min read
- Severity
- High
- Status
- Reported by outlets
- Sources
- 1
- Last checked
The short version
- Microsoft said the outage ran from 11:20 AM ET on 12 November to 11:10 AM ET on 13 November.
- The company traced it to a bug introduced in July, a traffic surge after a Teams failover, and reduced server capacity.
- At its lowest, more than 80% of gateway servers were unresponsive, Microsoft said.
- Exchange Online could not switch to alternative infrastructure, so mailboxes stayed unreachable longest.
- A final post-incident report is expected later this week.
What happened
Microsoft has said a software bug introduced in July caused the worldwide Microsoft 365 outage of 12 November, BleepingComputer reported on 16 November. The disruption lasted nearly 24 hours and hit Outlook, Exchange Online, Teams and Copilot.
In a preliminary post-incident report in the Microsoft 365 Admin Center, the company said the incident began at 11:20 AM ET on Thursday 12 November. It ended at 11:10 AM ET the next day, when the company confirmed services were fully restored. Users could not reach their email, and message delivery was significantly delayed.
The company said three problems combined: a previously undetected bug, extra traffic after a Microsoft Teams failover, and reduced server capacity. Microsoft said the bug came from an update to its Microsoft 365 networking gateway servers, the machines that handle incoming traffic. The update had run for more than three months without trouble.
Microsoft said that about 15 hours before the outage, Teams automatically switched to the networking infrastructure used by Exchange Online, which raised traffic on those servers. Nearly 14% of the gateway's capacity was also offline because of an unrelated issue. Microsoft said there was enough capacity to cover that loss, but the combination triggered the bug.
Servers began to stop responding, Microsoft said, and its systems redirected users to other servers, which were then overwhelmed too. In its words, the defect was encountered across the fleet in under two hours. Engineers built a fix and deployed it, and services stabilised through Friday.
Charles Lamanna, Microsoft's executive vice president of Copilot, Agents, and Platform, apologised in a statement on X. Microsoft said it has since added capacity, improved monitoring and put in safeguards to stop traffic redirection triggering similar cascades.
Why it happened
Microsoft's preliminary report names a latent bug in the gateway update shipped in July. It said the bug could overwhelm servers under certain traffic conditions, so they stopped responding. The company said its load testing missed it because labs used weaker hardware with fewer cores, while the problem is worse on machines with more cores and newer hardware.
The trigger, per Microsoft, was the Teams failover traffic landing on Exchange Online's infrastructure while about 14% of capacity was already out. Failed servers pushed their users onto healthy ones, which failed in turn. Microsoft also said safeguards meant to stop problems spreading between regions did not contain it, so the failure went global.
Recovery was slow, Microsoft said, because restarted servers were quickly swamped again by backed-up traffic. Microsoft has not said which team's change set the sequence going or why the unrelated capacity loss occurred. It is also unclear whether a separate earlier Teams outage that day is connected.
How this kind of failure works
A latent bug is a defect that sits quietly in production until specific conditions wake it up. Code can pass every test and run for months because the conditions that expose it have never occurred together. Contention bugs, where many processes fight over the same resource, are a common example, and they tend to appear only under heavy load or on hardware with many cores.
Picture a shop with ten tills and one narrow door. On a normal day, customers trickle through. If a coach party arrives while two tills are shut, the queue jams the door, and staff send customers to other tills, which jam as well. Nobody designed the door to fail, but nobody tested it with a crowd.
That pattern is a cascading failure. Rerouting traffic from failed servers is meant to protect users, yet it can spread the overload to every healthy machine. Restarting makes it worse, because the returning servers face a backlog before they can stabilise.
Testing on hardware unlike production is a well-known gap. Lab machines are cheaper, so teams use them, but a bug that depends on core count or hardware generation can pass cleanly there and fail on the real fleet.
Who is affected
Microsoft said the outage disrupted Microsoft 365 services worldwide, naming Outlook, Exchange Online, Teams and Copilot. Exchange Online customers were hit longest, because that service could not move to alternative infrastructure while Teams, OneDrive and SharePoint could and recovered earlier.
The company has not published customer numbers or a list of affected regions beyond saying the impact was global. Administrators can check the incident entry and preliminary report in the Microsoft 365 Admin Center. Users would have seen failed sign-ins to mailboxes, or messages arriving late.
This was an availability failure. Neither Microsoft's report nor BleepingComputer's coverage describes data loss or an attack. In practice the cost was lost access to email and delayed messages during working hours.
How to defend against it
Tick each step as you check it. Your progress is saved on this device.
What to learn from this
The concept: Cascading failure
A cascading failure happens when one component's failure raises the load on others, and they fail too. Each step is a reasonable reaction, such as redirecting users, yet the sum is a spreading collapse. Microsoft described exactly this as servers handed traffic to peers that then failed.
The same pattern appears in power grids, where a tripped line shifts load onto neighbours, and in databases where a slow node causes clients to retry and flood the rest. Defences include spare headroom, load shedding, circuit breakers and isolation between regions.
Microsoft said its isolation between regions did not hold. A safeguard that has never been exercised under real stress is a hypothesis, not a guarantee.
Redirecting traffic from failed systems can spread the failure unless capacity, limits and isolation are tested together.
Quick check
Sign in to earn 10 XP for a right answer
Why did rerouting users away from unresponsive gateway servers make the outage worse?
Microsoft said redirected users overloaded the remaining servers, which then failed as well, spreading the problem across the fleet.
Timeline
- July 2026 Microsoft says the buggy networking gateway update was introduced.
- About 15 hours before the outage, 12 November Teams automatically switched to the infrastructure used by Exchange Online, raising traffic.
- 11:20 AM ET, 12 November The outage began, according to Microsoft.
- Within two hours of the start Microsoft says the defect hit servers across the fleet, with over 80% unresponsive at the lowest point.
- Through 13 November Engineers deployed a fix and services stabilised.
- 11:10 AM ET, 13 November Microsoft confirmed active customer impact had ended.
- 16 November BleepingComputer reported Microsoft's preliminary findings; a final report is due later this week.
What is not known yet
- The final post-incident report has not been published.
- Microsoft has not said how many customers or regions were affected.
- It is not clear whether the separate earlier Teams outage is related.
- Microsoft has not explained the unrelated issue that took about 14% of capacity offline.
Questions people ask
What caused the Microsoft 365 outage on 12 November?
Microsoft's preliminary report names three factors: a software bug introduced in July in a gateway update, extra traffic after a Teams failover, and reduced server capacity. Together they left more than 80% of gateway servers unresponsive at the lowest point.
How long did the Microsoft 365 outage last?
Microsoft said it began at 11:20 AM ET on 12 November and ended at 11:10 AM ET on 13 November, close to 24 hours. Some services recovered earlier, while Exchange Online took longest.
Was the Microsoft 365 outage a cyberattack?
Neither Microsoft's preliminary report nor BleepingComputer's account describes an attack. Microsoft attributes it to a latent software bug, a traffic surge and lost capacity. The final report may add detail.
Why did Microsoft's testing miss the bug?
Microsoft said its load testing used weaker hardware with fewer cores than production. The contention problem worsens on machines with more cores and newer hardware, so lab tests did not reveal it.
Sources
- BleepingComputer: Microsoft says months-old bug caused global Microsoft 365 outage