[NETWORK] Time Warner/Level3 outage

It appears Time Warner is having an outage that is widespread, impacting multiple locations across the US. If you are a TW subscriber, or if your network traffic primarily crosses over TW/Level3, this can make it appear that your site and or services are down (or, going down and then coming back up). All servers and ...

[GENERAL] Client area application update

This evening we will installing a minor update for the client area, which hopefully will also remedy the blank result page on the primary domain checker (thanks to those of you who brought it to our attention). The client area will be unavailable for login for about five minutes. Queries submitted via email to the ...

[SERVERS] Drive replacements: various

This afternoon we will be replacing drives in the following servers: whistler, baber, parsley, and chronos. The total impact time per server will be about five minutes, to power down each server, make the drive swap, and bring each back online. During this time, services for the affected servers will not be available. If you ...

[GENERAL] Client area updates

On Saturday, May 6, we will be updating the client area. During the update, the client area will be unavailable for 5-10 minutes. During this time, direct logins will not be possible; however, any tickets can be submitted via email as usual. If you have any questions related to this maintenance, please contact support via ...

[SERVERS] Kernel updates, May 5-7

Beginning the evening of May 5 and continuing throughout the weekend, we will be rolling out kernel updates to the servers. Total impact time for each server will be 2-5 minutes for the required reboots to finalize the kernel updates. If you have any questions related to this maintenance, please contact support via the usual ...

[SERVERS] Hermes issues/emergency migrations

We are currently seeing rather significant and intermittent issues with the hermes server. Those issues include spiking loads and errors in services that fail as a result, then have to restart. In several instances, we have had to do uncontrolled reboots to bring the server back in line. This server has been in production for ...

[MAINTENANCE] Kernel, mail, client area updates

It’s time for another round of kernel updates for the servers within the network. We will be starting those this evening and continue throughout the weekend. As the number of servers in the network continues to grow, if this maintenance is not completed by late Sunday evening, we will be completing any servers that still ...

[SERVERS] Dali issues

Dali was not just surreal in real life: Dali’s namesake server is generating hardware-related notices in the system logs that are creating sustained load averages with occasional spikes that result in various services shutting down and restarting. The issues we are seeing are not in line with what we would expect to see with the ...

[SERVERS] Kernel updates, Dec 12-14, 2014

We will be rolling out kernel updates beginning Friday evening, December 12, 2014, and continuing throughout the weekend, ending on the evening of Sunday, December 14, 2014. Total impact time per server will be 1-2 minutes for reboots to finalize the new kernel builds. If you have any questions related to this routine maintenance, please ...

[SERVERS] Cratos issue

Currently investigating an issue with the cratos server, rendering it inaccessible. Updates to follow as available. Update: based on a review and testing, it appears there was a surge or short in the server. This has resulted in the primary data drive being damaged, and more specifically the boot loader, along with at least one ...