Ticket Number : SUNETTICKET-8220
Ticket Type : Unscheduled
Ticket Status : Open
Ticket Summary : Link down
Ticket Scope :
Ticket Opened : 20221028 7:29 UTC
Ticket Closed :
Problem Start : 20221028 7:21 UTC
Problem End : 20221028 15:51 UTC
Affected organisations:
* SUNET, NORDUnet, Uninett, Netnod
----
Description:
* Link down between Kongsvinger - Jössefors.
----
Impact:
* The following services is affected:
SU-S003718 - AW Narvik - Oslo
SU-S003715 - AW Trondheim - Oslo
SU-S003693 - Wave for Netnod "dwdm1-stc.sth.netnod.se" to
"HMG9"
SU-S003664 - Alien wave for Nordunet TUG - GAUS23 192900GHz
Reduced redundancy for SUNET, NORDUnet, SIKT and Netnod.
----
Update:
* 20221028 12:12 UTC
Provider has isolated to the Norway side. Technicians has
been dispatched but no ETR.
----
* 20221028 12:13 UTC
Provider has isolated the fault on the Norway side.
Technicians has been dispatched but no ETR.
----
* 20221028 13:55 UTC
Provider has estimated that ETR is 3 hours but could take
longer depending on the size of the damage.
----
* 20221028 16:19 UTC
Service is restored.
----
* 20221031 8:36 UTC
After the incident we are seeing a deterioration on the
fiber levels of around -2 dBm. Provider will send out a
technician to continue the troubleshooting today after 17
UTC. Short outages are expected.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8207
Ticket Type : Unscheduled
Ticket Status : Open
Ticket Summary : FPC 3 Errors lla1-r2
Ticket Scope :
Ticket Opened : 20221022 17:50 UTC
Ticket Closed :
Problem Start : 20221022 15:46 UTC
Problem End : 20221022 16:29 UTC
Affected organisations:
* SUNET, RISE, Netnod, FUNET, NORDUnet
----
Description:
* Major and a minor error on FPC 3 in router lla1-r2.
All interfaces on that FPC went down.
----
Impact:
* The following services were impacted:
SU-S002138 - Backbone lla1-r2 - lla1-r1
SU-S003527 - Netnod-IX Luleå
SU-S003766 - RISE CPE lla-c2 lla-c2. Björkskataleden 112
SU-S003776 - IP-VPN customer connection RISE Björskataleden
112
SU-S003787 - IP service, RISE Björkskataleden 112
SU-S003570 - FUNET L2 circuit between klx1-r1 and lla1-r1
SU-S003517 - NDN link Kalix Luleå
SU-S003843 - Nordunet Transit Luleå
Reduced redundancy for SUNET, FUNET, NORDUnet, Netnod and
RISE.
----
Update:
* 20221022 19:45 UTC
All services were restored after a reboot of the card.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8124
Ticket Type : Unscheduled
Ticket Status : Open
Ticket Summary : FPC error on umu-r2
Ticket Scope : Hardware
Ticket Opened : 20220925 17:44 UTC
Ticket Closed :
Problem Start : 20220923 5:44 UTC
Problem End : 20221004 8:13 UTC
Affected organisations:
* UMU, SUNET
----
Description:
* Following SUNETTICKET-8110, we now have have major errors on
FPC2 on our router in Umeå.
----
Impact:
* Affected service:
SU-S003027 to HPC2N, sunet-hpc2n-phy3
SU-S003027 to HPC2N, sunet-hpc2n-phy4
----
Update:
* 20220925 17:54 UTC
A new restart of the card was successful in bringing the FPC
back online. Monitoring.
----
* 20220926 7:29 UTC
New alarms on FPC2
tobias@umu-r2-re0> show chassis alarms
1 alarms currently active
Alarm time Class Description
2022-09-25 20:33:57 UTC Minor FPC 2 Minor Errors
----
* 20220926 8:25 UTC
Escalated to our hardware provider.
----
* 20220929 19:43 UTC
FPC card went down again. Escalating to provider.
----
* 20221010 9:16 UTC
Moving NAAD one week. Will monitor issue for one week.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8220
Ticket Type : Unscheduled
Ticket Status : Open
Ticket Summary : Link down
Ticket Scope :
Ticket Opened : 20221028 7:29 UTC
Ticket Closed :
Problem Start : 20221028 7:21 UTC
Problem End :
Affected organisations:
* SUNET, NORDUnet, Uninett, Netnod
----
Description:
* Link down between Kongsvinger - Jössefors.
----
Impact:
* The following services is affected:
SU-S003718 - AW Narvik - Oslo
SU-S003715 - AW Trondheim - Oslo
SU-S003693 - Wave for Netnod "dwdm1-stc.sth.netnod.se" to
"HMG9"
SU-S003664 - Alien wave for Nordunet TUG - GAUS23 192900GHz
Reduced redundancy for SUNET, NORDUnet, SIKT and Netnod.
----
Update:
* 20221028 12:12 UTC
Provider has isolated to the Norway side. Technicians has
been dispatched but no ETR.
----
* 20221028 12:13 UTC
Provider has isolated the fault on the Norway side.
Technicians has been dispatched but no ETR.
----
* 20221028 13:55 UTC
Provider has estimated that ETR is 3 hours but could take
longer depending on the size of the damage.
----
* 20221028 16:19 UTC
Service is restored.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8223
Ticket Type : Scheduled
Ticket Status : Open
Ticket Summary : Planned Link Maintenance
Ticket Scope : Link
Ticket Opened : 20221028 14:25 UTC
Ticket Closed :
Maint. win. Start : 20221109 16:00 UTC
Maint. win. End : 20221109 20:00 UTC
Estimated outage : 120.0 min
Affected organisations:
* SUNET, LTU
----
Description:
* GEANT OC will upgrade the current Junos software to version
20.4R3, and perform an essential configuration on the
router. GEANT traffic disruption in the region of 2 hours is
expected during the Junos upgrade process, but the whole
maintenance window should be considered at risk.
----
Impact:
* Services at risk:
SU-S003629 - MS Express LTU - London Primary New
SU-S003845 - MS Express GIH, via Nordunet and Geant
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8220
Ticket Type : Unscheduled
Ticket Status : Open
Ticket Summary : Link down
Ticket Scope :
Ticket Opened : 20221028 7:29 UTC
Ticket Closed :
Problem Start : 20221028 7:21 UTC
Problem End :
Affected organisations:
* SUNET, NORDUnet, Uninett, Netnod
----
Description:
* Link down between Kongsvinger - Jössefors.
----
Impact:
* The following services is affected:
SU-S003718 - AW Narvik - Oslo
SU-S003715 - AW Trondheim - Oslo
SU-S003693 - Wave for Netnod "dwdm1-stc.sth.netnod.se" to
"HMG9"
SU-S003664 - Alien wave for Nordunet TUG - GAUS23 192900GHz
Reduced redundancy for SUNET, NORDUnet, SIKT and Netnod.
----
Update:
* 20221028 12:12 UTC
Provider has isolated to the Norway side. Technicians has
been dispatched but no ETR.
----
* 20221028 12:13 UTC
Provider has isolated the fault on the Norway side.
Technicians has been dispatched but no ETR.
----
* 20221028 13:55 UTC
Provider has estimated that ETR is 3 hours but could take
longer depending on the size of the damage.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8124
Ticket Type : Unscheduled
Ticket Status : Open
Ticket Summary : FPC error on umu-r2
Ticket Scope : Hardware
Ticket Opened : 20220925 17:44 UTC
Ticket Closed :
Problem Start : 20220923 5:44 UTC
Problem End : 20221004 8:13 UTC
Affected organisations:
* UMU, SUNET
----
Description:
* Following SUNETTICKET-8110, we now have have major errors on
FPC2 on our router in Umeå.
----
Impact:
* Affected service:
SU-S003027 to HPC2N, sunet-hpc2n-phy3
SU-S003027 to HPC2N, sunet-hpc2n-phy4
----
Update:
* 20220925 17:54 UTC
A new restart of the card was successful in bringing the FPC
back online. Monitoring.
----
* 20220926 7:29 UTC
New alarms on FPC2
tobias@umu-r2-re0> show chassis alarms
1 alarms currently active
Alarm time Class Description
2022-09-25 20:33:57 UTC Minor FPC 2 Minor Errors
----
* 20220926 8:25 UTC
Escalated to our hardware provider.
----
* 20220929 19:43 UTC
FPC card went down again. Escalating to provider.
----
* 20221010 9:16 UTC
Moving NAAD one week. Will monitor issue for one week.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8164
Ticket Type : Unscheduled
Ticket Status : Closed
Ticket Summary : FPC9 restart sthb-r1
Ticket Scope : Hardware
Ticket Opened : 20221007 11:37 UTC
Ticket Closed : 20221028 12:47 UTC
Problem Start : 20221007 10:20 UTC
Problem End : 20221025 11:28 UTC
Affected organisations:
* SUNET
----
Description:
* FPC9 restarted for no obvious reason to us:
mhert@sthb-r1-re0> show log messages | grep CHASSISD_
Oct 7 10:20:25.290 sthb-r1-re0 chassisd[9634]:
CHASSISD_IPC_CONNECTION_DROPPED: Dropped IPC connection for
FPC 9
Oct 7 10:20:25.290 sthb-r1-re0 chassisd[9634]:
CHASSISD_IPC_WRITE_ERR_NULL_ARGS: FRU has no connection
arguments fru_send_msg FPC 9
Oct 7 10:20:25.290 sthb-r1-re0 chassisd[9634]:
CHASSISD_IFDEV_DETACH_FPC: ifdev_detach_fpc(9)
----
Impact:
* The following services was unavailable during the reboot:
SU-S003813 - sthb-r1 - ume8-r1 via FUNET wdm
----
Update:
* 20221010 12:37 UTC
Escalated to provider.
----
* 20221026 5:47 UTC
FPC has been replaced. Monitoring.
----
Final ticket report:
* FPC9 in sthb-r1 rebooted for the third time within a year
and has been replaced.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8220
Ticket Type : Unscheduled
Ticket Status : Open
Ticket Summary : Link down
Ticket Scope :
Ticket Opened : 20221028 7:29 UTC
Ticket Closed :
Problem Start : 20221028 7:21 UTC
Problem End :
Affected organisations:
* SUNET, NORDUnet, Uninett, Netnod
----
Description:
* Link down between Kongsvinger - Jössefors.
----
Impact:
* The following services is affected:
SU-S003718 - AW Narvik - Oslo
SU-S003715 - AW Trondheim - Oslo
SU-S003693 - Wave for Netnod "dwdm1-stc.sth.netnod.se" to
"HMG9"
SU-S003664 - Alien wave for Nordunet TUG - GAUS23 192900GHz
Reduced redundancy for SUNET, NORDUnet, SIKT and Netnod.
----
Update:
* 20221028 12:12 UTC
Provider has isolated to the Norway side. Technicians has
been dispatched but no ETR.
----
* 20221028 12:13 UTC
Provider has isolated the fault on the Norway side.
Technicians has been dispatched but no ETR.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
Ticket Number : SUNETTICKET-8220
Ticket Type : Unscheduled
Ticket Status : Open
Ticket Summary : Link down
Ticket Scope :
Ticket Opened : 20221028 7:29 UTC
Ticket Closed :
Problem Start : 20221028 7:21 UTC
Problem End :
Affected organisations:
* SUNET, NORDUnet, Uninett, Netnod
----
Description:
* Link down between Kongsvinger - Jössefors.
----
Impact:
* The following services is affected:
SU-S003718 - AW Narvik - Oslo
SU-S003715 - AW Trondheim - Oslo
SU-S003693 - Wave for Netnod "dwdm1-stc.sth.netnod.se" to
"HMG9"
SU-S003664 - Alien wave for Nordunet TUG - GAUS23 192900GHz
Reduced redundancy for SUNET, NORDUnet, SIKT and Netnod.
----
Update:
* 20221028 12:12 UTC
Provider has isolated to the Norway side. Technicians has
been dispatched but no ETR.
----
--
For information about SUNET tickets and the sunet-fail list visit:
http://www.nunoc.org/nunocweb/open_trouble_tickets.html
