Skip to content

News from the staff team

  • 1 min read

Unexpected Downtime

The OCF directory server went down today (Tuesday 9/15) at roughly 10:20 PM, taking pretty much everything else with it. We're still working on figuring out what went wrong and how to fix it.

We'll post updates here as we have them. You can also find us at on the #OCF channel on the CSUA irc server at irc.csua.berkeley.edu.

UPDATE 9/16/09 3:00 PM: It looks like everything is back up again. If you find anything missing or broken, please let us know.

And to everyone who was wondering, yes, this is why your passwords/email stopped working. This is also indirectly responsible for why you may have gotten funny looking prompts when logging in remotely.

  • 1 min read

General Meeting on 9/10

Hello dear OCF members!
The new school year has begun, so it is time again for the OCF’s semi-annual General Meeting! It will be held at 7PM in 243 Dwinelle Hall. Come vote for this semester's new General and Site Managers, give us your suggestions, volunteer to help (we obviously need it), or just pelt us with your complaints! Pizza and various drinks shall be served to enhance the experience. See you all there!

  • 2 min read

Status of OCF Services

We've been working hard to get all our services restored. As usual, we've run into some bumps along the way, but I'm happy to report that we've been able to bring up the following services:

  • Webmail: You can read your mail via our webmail interface. However, mail sending and receiving is still in the process of being restored.
  • Web hosting: The web server is mostly functional. The OS installation managed to corrupt itself during the move from our old server room in MLK to our new server room in Eshleman. We were able to hobble together a new installation from backup, but we're not sure if all our server components were restored. If something seems out of the ordinary, please comment and let us know.
  • MySQL databases: These should be completely functional.
  • Shell Access (SSH/SFTP/SCP): We know that many of you need access to your files. Our goal is to bring up our login servers by Thursday night. apocalypse is now online and accepting logins.
  • POP3/IMAP: You can now check mail using software on your computer (ex., Thunderbird, Postbox, Outlook, etc.)
  • Outgoing Mail: You can now send mail from the OCF.
  • Incoming Mail: Our mail infrastructure is split into many pieces that are hosted on multiple servers. Unfortunately, all the pieces need to be brought up before we can re-enable mail (otherwise, there would be a flood of spam, bounces, and incorrectly delivered mail). We hope to have incoming mail restored by the end of the week. Incoming mail is up as of Tuesday night.
Once again, we apologize for the extended downtime. We're exploring better ways of handling extended outages in the future, but we want to emphasize that this downtime was really an anomaly. A perfect storm of factors came together: a long power outage, a server room migration, a network disconnection, and an IP space change.

We'd also like to thank the wonderful people at Physical Plant, IST, Residential Computing, and other campus organizations for coming to our aid. Also, thanks to our users for their supportive and critical comments (hey, we deserve it!).

UPDATE (28-Aug-09, 9:30 AM): We had a bit of a hiccup in our services yesterday night. All services that were working yesterday morning should also be working now.

UPDATE (29-Aug-09, 7:00 PM): We're still restoring our mail services. We haven't forgotten about our other services (printing, web file access, etc.) We're just prioritizing our most important ones.

UPDATE (1-Sep-09, 9:00 AM): We're starting to ramp up incoming mail. Once we're sure that it's working perfectly, we'll begin restoring the rest of our services.

UPDATE (2-Sep-09, 8:00 AM): Incoming mail is completely operational. We're starting work on restoring the rest of our services.

  • 1 min read

Good News!! (About Time...)

Today, the electrical work in our new server room was finally finished! Also, our network problems are over! Connectivity has been restored - all thanks to IST, who swooped in today to hook OCF up to their own network. Not only do we receive a steady fiber connection, but we also get IPv6 and our very own subnet.

So why aren't OCF services up and running, you ask? Well, all the IP addresses have changed because we are on a different subnet. This means that in addition to flashing all the switches, we must also renumber all the IPs on our servers, which we have many of. This is quite a task, requiring staff members to sit in the server room with a screen and keyboard and reconfigure every single server we have. Since a couple staff members are returning to Berkeley this weekend, I am planning to do this on Monday so that they can help me out. And since everything that can go wrong already has, I am going to be optimistic and hope that we won't run into any problems bringing servers online. If all goes well, OCF can be back up by Tuesday morning!

  • 1 min read

You Know What They Say About Academia...

The cause of our networking outage has not been determined. We're working with campus technicians to debug the problem, but everything is going really s-l-o-w-l-y. We're just as upset as you are about the prolonged downtime, but, as a student organization, we just don't have as much sway as a campus department.

  • 1 min read

Continuing Downtime

The downtime that began on 7/28 is expected to continue for a few more days as campus electricians procure parts to complete the electrical work. If there are no other unanticipated delays, we should be back up in the coming days.

We apologize for the downtime and inconvenience.

UPDATE: The electrical work will be completed by Friday, August 14, at 4 PM. Unfortunately, we have very limited staff presence in the Berkeley area due to the summer vacation, so it may take some time to get everything up and running.

SECOND UPDATE (14-Aug-2009, 3 PM): The electrical work is mostly complete, and power has been restored to our server room. Unfortunately, something happened to our network uplink during the downtime (maybe someone forgot to turn a switch back on), so we have no Internet access. We're looking into it.

THIRD UPDATE (18-Aug-2009, 4 PM): A campus network technician is working on restoring connectivity to the OCF. The cause of the problem is still undetermined, but we're pressuring them to get things fixed ASAP.

  • 1 min read

OCF Downtime Starting 7/28

The OCF will be down next Tuesday due to electrical work being done in Eshleman Hall. Unfortunately, we have not been given a time frame, so servers may be down for up to a week. We apologize for the inconvenience, but will try hard to make sure services come back up as soon and as smoothly as possible.

  • 1 min read

The Pre joins OCF Mobile

Last year, we launched our mobile site to load for the iPhone/iPod touch, and right after included other devices like the Blackberry Storm, Android-based G1, and Opera Mini (such as on the Blackberry Curve beyond the BB browser). All other devices could still enter the site manually through m.ocf.berkeley.edu.

The Palm Pre has now been added to that list. Data detection works just as it does with the iPhone OS, so your phone dialer, Google Maps, and email client will work with our pages there too. I'll be on the lookout for Windows Mobile devices to test upon.

  • 1 min read

Unexpected Downtime

OCF servers and services are currently experiencing unexpected downtime, due to what may be a Denial of Service (DoS) attack on the subnet the OCF is on. Most staffers are currently out of town due to the summer break; while this is being looked into, it's a bit difficult without connectivity to the servers.

Further updates to come on this blog. While our Internet Relay Chat (IRC) server is down, you can connect to CSUA's IRC server at irc.csua.berkeley.edu and join us in #ocf for updates (mibbit IRC client).

2:18 PM: It seems like we're back online, but the causes of the outage is still unknown. No data loss is anticipated; please let us if otherwise.

  • 1 min read

apocalypse accepting user logins

apocalypse.OCF.Berkeley.EDU (192.58.221.245) is now accepting user logins. You should be able to connect to the server via SSH/SFTP to access your home directory and web space. If you're comfortable using a command-line mail client (e.g. mutt or pine), you'll also be able to read and send mail.

There are a few lingering issues with the server. In particular:

  • Public key authentication does not work. We're working on fixing this problem. Fixed.
  • Some pre-installed software and user commands may be missing or broken. We had to re-compile all of our software for the rebuild, and it's an ongoing process. We prioritized the most commonly used programs, but leave a comment if you think we've forgotten an important program. UPDATE: We know the software waitlist is building up, so we're getting more staff involved in this process.
  • Password changing does not work. Use kpasswd to change your account password. Windows password changing is still broken.
  • Cron jobs are not yet available. Cronjobs should now work as documented on our wiki.
The SSH host keys are:
  • DSA:
    04:68:af:14:ab:a1:4e:79:c3:42:6f:51:62:10:a2:5a
    xivit-hunuc-kogur-hizez-pazyr-bynat-pytop-zitos-dufuf-danec-sixax
  • RSA:
    72:16:9c:7e:7b:cc:a0:47:56:6d:e3:59:aa:04:bc:7c
    xipav-pizem-lysyd-kicyt-sococ-macuz-hyluc-vizoc-cotem-latig-muxyx
Thanks for your continued patience!