Vorco Status - Unplanned Event - International Connectivity

Unplanned Event - International Connectivity

Incident Report for Vorco

Postmortem

We have today received an RFO from our upstream transit provider. The cause of this incident was an upstream international transit outage between AS9790 and AS4826 caused by AS9790 bringing a new BGP peer via AS6939 online.

Upon AS9790 bringing that new peer with AS6939 online, additional BGP routes got re-advertised by AS9790 to AS4826 resulting in AS9790 prefix limits with AS4826 being reached and accordingly BGP sessions automatically shut down. The impact of this was the loss of about 18% of international transit.

As soon as the prefix limits were noticed the change was rolled back to restore connectivity and a post-incident review has been conducted to identify and address opportunities for improvement.

Posted Nov 23, 2023 - 16:14 NZDT

Resolved

This was resolved at 3:04pm. We'll update this incident with a root cause when available.

Posted Nov 17, 2023 - 16:44 NZDT

Update

We've confirmed this is due to an upstream transit fault which for the time being has resolved itself. We're awaiting confirmation on whether a permanent fix is in place and will update this incident when we receive more details.

Posted Nov 17, 2023 - 15:30 NZDT

Identified

Start Time: 1434
Service(s) Affected: International internet connectivity
Customer Impact: Customers may be unable to reach websites and services hosted internationally, particularly those that route via the USA. This appears to be a fault upstream and we are liaising with our upstream transit for more information. Connectivity to Australia appears to be working still.

We apologise for any inconvenience this may cause.

Posted Nov 17, 2023 - 14:34 NZDT

This incident affected: Core Network and Internet & WAN Access.