Scheduled downtime Apr. 26th, 2009 at 7am EST

We delayed the previous datacenter move and adjusted it to a better time of day: 7am EST on a Sunday.

The outage should be between 30 minutes and an hour if everything goes smoothly and indefinitely longer if it doesn’t go smoothly!

Some additional info that hasn’t been on the blog: we were generously given two more servers by a local company that was upgrading (thanks!) and have ordered gobs of RAM for them. Once that is in, we will be upgrading LyricWiki’s setup once again. We currently have 6 servers, this will bring us up to 8! That’s a significant increase and hopefully will make LyricWiki even faster.

Back to normal after DDoS attack

The problems earlier today were apparently caused by an attempted distributed denial of service attack.

The datacenter has got all of this bad traffic blocked now, and the site appears to be back to normal (for several hours in a row now).  The whole problem lasted about 1 to 2 hours but it’s tough to measure since the site was never completely down during that time.

2/26: Site will be up and down

The site is currently behaving very sporadically… there are some problems in the connection between our datacenter (where all of our servers hang out) and the internet.

Not sure whether this is related to the “emergency maintenance” Sprint was doing last night or not.

The team at the datacenter is working on it right now, and hopefully we’ll be back soon.

Recent server issues

You may have noticed an error message along the lines of “Host ‘pedlfaster.pedlr.com’ is blocked because of many connection errors” recently.

While we kept fixing that in the short-term, it kept popping back up. Now there is a more permanent fix in there, and I’m looking into what caused it (I’m assuming it was a spike in API traffic).

Regardless: you shouldn’t be seeing that anymore. If you do, harass us immediately!

Taking another crack at it… site will be down.

Once more into the breach!

I’m going to hurl myself at the database-replication problem again.  The current plan I have unfortunately involves some downtime for the site.  But it’s already after 1am so I’m hoping the impact on users is relatively low.

I’ll keep you updated.