Skip to content

2022

  • 1 min read

Urgent Maintence Downtime

tl;dr; The entire OCF will be offline for at least 24 hours.

We have discovered that several of the drives in our main storage array are experiencing a firmware bug that may lead to data loss. We are focusing on minimizing the impact of this issue and as a result we have decided to take the entire OCF offline for at least 24 hours until we can get replacement drives. We will be posting updates about the downtime to this blog, so keep an eye out.


2022-09-11 3:08PM: OCF goes offline

  • 1 min read

Multiple mirrors out of date

 We noticed that multiple OCF mirror archives were out of date due to exhausted boot disk space on the mirrors server.

Affected archives include (this is not an exhaustive list):

Debian

Debian-security

Puppetlabs

Ubuntu

Ubuntu-ports

Kali


The last update of the affected archives was on Aug 13. Sync is running but might take a while.

We sincerely apologize for this. We will take steps to ensure that system monitoring alerts are handled in a timely manner.



  • 1 min read

Fedora out of sync

 I noticed that our Fedora mirror was out of sync due to a slow upstream mirror. Working on it.


Update: Switched to another upstream mirror. Sync speeds are higher now.

  • 1 min read

[RESOLVED] Raspbian out of sync

 We noticed that our Raspbian mirror is out of sync, likely due to the IP address change. 

10:16 PM - We have temporarily switched our upstream for the Raspbian mirror, and it's currently syncing. Emailed Raspbian. - rjz

5/13 - Resolved.

  • 1 min read

Emergency mirrors server maintenance today

 The OCF mirrors server will be down for about an hour staring 1300 PDT today due to a required move within the rack and IPv6 provisioning. We apologize for the inconvenience.


  • 1 min read

Emergency maintenance: hypervisors reboot today

We will be rebooting the hypervisors this afternoon at about 2 pm to apply the updated kernel that fixes several high-severity vulnerabilities. The downtime will be around 15-30 minutes, and all websites hosted by the OCF as well as the login servers, Matrix server, Slack bridge, mail server, and the IRC server will be unavailable during the reboot.

  • 1 min read

[Complete, see notes] Mirrors downtime 3/11/2022 - migration to campus DC

 migration to campus DC underway, expect downtime for ~2 hours


Update 3/12/2022: Migration completed by 14:00 yesterday. Currently our mirrors are IPv4 only, IPv6 ETA 2 weeks. Some Kali packages may not download correctly due to campus firewall settings, so if you experience any broken Kali downloads, please switch to the https version of our mirror. We are in contact with the university to get the issue sorted out as soon as possible.

  • 1 min read

Mirrors maintenance tonight at 8

We will be installing additional RAM on the mirrors server tonight at 8pm. Expect 15 mins of downtime.



Update: ETA 2130. Apologies, but the process turned out to be more involved than expected.

Update 2: The mirrors are up as of 21:30.

  • 1 min read

Scheduled downtime on Saturday, 2/12

We will be upgrading the operating system on our web hosting (death), login (tsunami), and app hosting (vampires) servers this Saturday, 2/12/2022, beginning at 6:00 pm Pacific Time. All OCF-hosted websites and apps will be unavailable during the upgrade. We expect the downtime to last for about 4 hours.

We apologize for the inconvenience.

  • 1 min read

None