Elektrine lite

← Feed

@david_chisnall@infosec.exchange

2026-11-13 11:42 UTC

This year has not been a good year for uptime on my self-hosted mail sever. At one point I accidentally went over the free limit in the cloud subscription I was using and didn't notice for a bit. I also shut it down to export the disk and move it to a different server. We have now passed the point where my personal email has better uptime in the past year than M365. I think this year M365 (which Cambridge university now uses for email) has had more down time than their prior self-hosted solution had in the ten years before they switched (and most of the downtime I remember on that system was a single incident where the room that the ja.net connection for the entire university came in flooded, which would also have affected most other services since it broke Internet access for every university building that didn't have redundancy [the computer science department had an alternative DSL line but it was accessible only via WiFi so work desktops were still disconnected). This is not a problem with the cloud. Having redundant servers, multiple independent failure domains, replicated storage, and so on are all things that are useful in providing a high-availability system. Outsourcing can allow you to share costs with many other people with similar requirements and benefit from larger economies of scale. This is 100% a failure of management. Satya 'no follow through' Nadella and Kevin 'must have the shiny thing' Scott are incapable of providing the kind of long-term stability and support for even medium-term planning that's necessary to provide reliable services for customers. And this trickles downwards.

Replies (1)

  • @dwm@mastodon.social 2026-11-13 21:56

    @david_chisnall@infosec.exchange I remember that deluge incident. That wasn't just the Internet for cam.ac.uk. It was the ja.net PoP for the entire East of England. We turned the DC off at around 0100hrs that morning as it was flooding. I recall that the NAT devices for the CUDN got turned into fishtanks. Networks team did absolute magic and had an alternate Internet link up by around 1000hrs the same day. https://web.archive.org/web/20150719173812/https://www.uis.cam.ac.uk/features/17-july-university-internet-connection-unavailable I need to find the photo of the incident whiteboard I set up that morning...

    Open ##5533814