Break reports

Schedule:

2010-09-11 02:30 - 2010-09-13 01:15

Duration:

1d 22:45 h

Affected services:

File service's (frodo, fs) group directories and some www-sites.

Reason:

Data transfer from one disk array system (Rivendell) to another (Mithlond) was interrupted because a nasty bug in logical volume management (LVM) manifested itself after 5TB (approximately 50% of the data) had been transferred. The bug corrupted at least the logical volume containing group directories.

The following web-sites were also shortly (few minutes) affected due to their dependency on file service:

• www.futureinternet.fi
• betelgeuse.hiit.fi
• cgi.hiit.fi
• cosco.hiit.fi
• packages.hiit.fi
• pgm2010.hiit.fi
• www.mdl-research.org

Update at 2010-09-11 12:46: Data in group directories is being restored from tape. File system checks are being ran on other volumes (e.g. home directories).

Update at 2010-09-11 13:39: Other volumes, including the volume containing home directories checked out fine.

Update at 2010-09-13 01:15: Group directories have been restored and are in use again. Samba connectivity has been restored. The break is over.

Schedule:

2010-09-09 08:00 - 08:40

Duration:

0:40 h

Affected services:

DNS, DHCP, file services, VCS, VPN, wiki, WWW.

Reason:

The following servers will be rebooted due to kernel upgrade.
- name.it.hiit.fi
- label.it.hiit.fi
- finglas.it.hiit.fi
- frodo.it.hiit.fi
- openvpn01.fe.hiit.fi
- bilbo.it.hiit.fi
- eowyn.it.hiit.fi
- stat.fe.hiit.fi

Pending updates will be installed as well.

Breaks to individual service should not be longer than five minutes. DNS services aren't affected at all due to their design.

Update at 08:40: Break is over.

Schedule:

2010-09-08 11:40 - 2010-09-09 11:40

Duration:

24 h

Affected services:

Printing, scanning and copying in Spektri using HIIT-PS122 Ricoh Aficio MP C4500

Reason:

Multifunction copier printer Ricoh Aficio MP C4500 is out of order. Waiting for repair.

Schedule:

2010-09-01 15:04 - 15:08

Duration:

0:04 h

Affected services:

Connections to and from HIIT's network.

Reason:

Router reboot.

Schedule:

2010-08-27 08:36 - 2010-08-27 08:56

Duration:

0:20 h

Affected services:

DNS

Reason:

Due to configuration mistake name.it.hiit.fi didn't respond to DNS queries. Other DNS servers functioned normally, so this caused only some delay in responses.