2. Break reports

Break reports

People & contact us

Here are listed all breaks in HIIT's IT services.

Break in munki-packages.hiit.fi at 2013-04-13 17:27 - 2013-04-16 12:15

Submitted by niemimak on April 17, 2013 - 12:05
Schedule: 
2013-04-13 17:27 to 2013-04-16 12:15
Duration: 
3 days
Affected services: 
Managed Software Update on OS X
Description: 

Due to configuration error in client revocation list creation the Apache server at munki-packages.hiit.fi started refusing connections until the CRL was replaced. During this time the clients were not able to install new packages from the repository. This didn't affect OS updates provided by Apple.

Break in earworms.hiit.fi at 2013-03-21 10:00 - 11:30

Submitted by nuorento on March 21, 2013 - 11:30
Schedule: 
2013-03-21 10:00 to 11:30
Duration: 
1h 30min
Affected services: 
earworms.hiit.fi project server
Description: 
Earworms.hiit.fi project server OS upgrade from Debian Linux  5.0 (Lenny)  to 6.0 (Squeeze).

Update at 11:30: The Break is over.

Break in network connections at 2013-02-27 17:00-17:06

Submitted by pmtonter on February 27, 2013 - 17:00
Schedule: 
2013-02-27 17:00 to 17:06
Duration: 
6 min
Affected services: 
HIIT network connections and UH/CS Ukko cluster's and Nodeslab's network connections
Description: 

Router reboot is needed in order to enable redundancy on the router. Reboot time should be shorter than announced 20 minutes but still few minutes.

Update at 17:31 by Pekka Tonteri: Network connections were restored at 17:06.

Short breaks in Linux servers due to kernel security updates at 2013-02-26 11:00 - 16:00

Submitted by nuorento on February 26, 2013 - 11:00
Schedule: 
2013-02-26 11:00 to 16:00
Duration: 
15-30min per server
Affected services: 
All Linux servers
Description: 

Due to kernel security update (DSA-2632-1), all Linux servers will be rebooted. This will cause approximately 15 minutes downtime on each service. Pending software and firmware updates will be installed as well, so some services will be down approximately 30 minutes.

Update at 11:10 by Markus Nuorento: Break in faramir.it.hiit.fi service:web

Update at 13:25 by Markus Nuorento: Break in elrond.it.hiit.fi  service:wiki.hiit.fi

Update at 13:55 by Markus Nuorento: Break in finglas.it.hiit.fi  service:vcs.hiit.fi

Update at 14:30 by Markus Nuorento: Break in shell.hiit.fi service:shell

Update at 15:01 by Markus Nuorento: shell.hiit.fi up and running.

 

 

References: 

Debian security advisory DSA-2632-1

Break in entry server and some project servers at 2013-02-20 17:00-17:37

Submitted by pmtonter on February 20, 2013 - 17:00
Schedule: 
2013-02-20 17:00 to 17:37
Duration: 
37 min
Affected services: 
shell.hiit.fi, foot.hiit.fi, infocapacity.hiit.fi and laaketieto.hiit.fi
Description: 

Due to storage misconfiguration some servers lost their SAN LUNs for a while. Due to that they needed a reboot. All in all, the following servers were booted: shell.hiit.fi, foot.hiit.fi, infocapacity.hiit.fi and laaketieto.hiit.fi.

All pending updates were installed as well. Also more hardware resources were allocated to shell.hiit.fi.

Update at 17:37 by Pekka Tonteri: All services are up and running.

Pages