Last evening at roughly 17:20 server time, the site experienced an unexpected downtime. The website came back online roughly 5 minutes later, but a number of critical services were down until 21:20.

Usually when the site is inaccessible, it's caused by an ISP glitch but this was not the case this time. A quick timeline of the downtime from my perspective:

- 17:20: Site goes down. I get notified of this on Discord and verify that the site is not accessible. I'm at work at this point, but can verify that the Internet connection is functional for the site.
- 17:27: I notice that the website is back up, but all other services (FFB server, gamefinder, etc) are still down. I'm still at work and can not actually verify what happened.
- 21:10: I get back home from work and am able to connect to the machines, noting that all machines seem to have rebooted.
- 21:20: All services are back up and running again.

After this, I spent some time checking to see if I could find any clues as to what happened. The hypervisor (the server running all the virtual machines FUMBBL runs on) seemed to also have unexpectedly rebooted, and appears to have simply hard crashed only to reboot again (the equivalent of a Windows blue-screen). Bootup-logs have scary looking terms in it ("Hardware error"), but no significant additional detail.

No data was lost during this, and there seem to be no adverse consequences after this crash, but I'm still not super happy about the random crash and the hardware error indication. While things like this *can* happen randomly and genuinely be caused by solar flares or random bit flips caused by literal radiation from space, it makes me a little bit nervous.

At this point, I am basically just calling it a random crash and leaving it at that. It if happens over and over again, though, I will need to try to figure out what the issue is and solve it which potentially means new hardware.

I understand that we're close to trophy deadlines and this happened during prime playing-time for a number of you.

I saw someone ask about postponing the trophy deadlines for a day because of this, but this is simply not something I can do. The new season requires me to do things in a very time-sensitive manner at midnight local time. I am not able to move this a day further at such a short notice.

While there's not much I can do to improve response times, I will still apologize for the inconvenience caused by this downtime and the effects of this on some of your trophy runs.