Connection Problems

[classical_hero]

I am having problems connecting to Poly. I had this problem last night also. Sometimes I do get lucky and connect, but what is causing the problems with the site?

[Krill]

Probably either related to the previous issues with the hosting company, or Locutus messing around with the DB while the Americans sleep.

[Gramphos]

We are performing maintenance on the server. It is not finished yet, but the disturbance should hopefully only last for some shorter periods of 10 to 20 minutes if all goes well.

[Asher]

More "maintenance"?

[Gramphos]

Yes, we are performing some upgrades, which we should have done before leaving settler3, but had to postpone because of the old server giving up too early. It should be OK now at least. We might still need to tweak some things but nothing that has to be done today, and nothing that probably will be done today either.

[Gramphos]

and now IRC is back up as well

[Wezil]

Still glitching.

[Asher]

Originally posted by Gramphos
Yes, we are performing some upgrades, which we should have done before leaving settler3, but had to postpone because of the old server giving up too early. It should be OK now at least. We might still need to tweak some things but nothing that has to be done today, and nothing that probably will be done today either.

Hardware?

[Gramphos]

Originally posted by Wezil
Still glitching.

Just high load.

[Gramphos]

Originally posted by Asher
Hardware?

No, low level software. We are changing the underlying storage system for the database to simplify backups.

[Wezil]

Originally posted by Gramphos

Just high load.

Alright then.

I'm getting "missing pages".

[Wezil]

And no Return to Forum button at the bottom of a thread.

[Asher]

Originally posted by Gramphos

Just high load.

I don't believe you, at least it's not high load caused by forum users.

Are you running some kind of query?

[Gramphos]

That's because the connection times out because of the high load. It shouldn't be for mush longer. But then there will be some downtime for the database as well later.

[Gramphos]

Originally posted by Asher


I don't believe you, at least it's not high load caused by forum users.

I didn't say it were forum users.

Are you running some kind of query?

Not a query, but extensive disk IO.

[Wezil]

Yeah okay.

The forums simply aren't usable.

Catch you later.

[Asher]

Originally posted by Gramphos
That's because the connection times out because of the high load. It shouldn't be for mush longer. But then there will be some downtime for the database as well later.

WTF could be causing high load?

There's only 300 users.

[snoopy369]

Do you always assume nobody else knows anything except you, Asher?

In this case it is not users - as Gramphos said - but from a maintenance operation allowing more frequent and simple backups. Since we'd hate to lose the forums if there were a fire in the server rack they're located on...

[Locutus]

We're still doing some maintenance for our new backup system, which is causing the high load. Nothing that will require reboots like earlier, but you may continue to experience slowness and even timeouts for now (though hopefully not for much longer). Our apologies for the inconvenience.

[Asher]

Look...I'm not trying to be mean or be a dick or anything.

But as a quasi-professional community, these seemingly random unannounced outages during peak hours are completely unacceptable. The site becomes completely unusuable for random durations of time as certain nameless site technicians run extremely expensive database queries which cripple the site, or they take it down completely to change things internally.

This should be complete common sense to anyone who runs a public website. When you are performing maintenance or upgrades or doing anything that could impact site performance, you notify the site in advance and provide timelines.

Do you always assume nobody else knows anything except you, Asher?

Well, apparently it's true. This is far from the first time this has happened. It's just courtesy to let people know when you're going to **** up the site they're on intentionally.

[Gramphos]

Originally posted by Wezil
Yeah okay.

The forums simply aren't usable.

Catch you later.

I'm trying to do the workload in chunks to keep the forums somewhat responsive.

[Gramphos]

Originally posted by Asher
But as a quasi-professional community, these seemingly random unannounced outages during peak hours are completely unacceptable. The site becomes completely unusuable for random durations of time as certain nameless site technicians run extremely expensive database queries which cripple the site, or they take it down completely to change things internally.

The intention was for a quick reboot off peak time (Morning CET.) But certin things didn't go as expected, and the reboot became a series of reboot that continued into the on peak time.

This should be complete common sense to anyone who runs a public website. When you are performing maintenance or upgrades or doing anything that could impact site performance, you notify the site in advance and provide timelines.

I agree that we should have informed in advance. The maintenance was scheduled about 6 hours before it were to start, and the admins were notified. However due to the short time frame, and uncertainty of the duration of the downtime I made no public statement at that time.

The reason for the short time frame is that we want to have a new backup system up and running by the end of this week.

[Locutus]

We are not a professional community -- that would imply we get paid for this, when in reality the reverse is the case. We're doing this in our free time with our own money. We avoid maintenance during peak hours when we can, but often can't avoid it due to the fact that we all have full-time jobs and lifes to run. In fact, our lifes are so busy that quite a lot of the time we don't know we'll be doing maintenance until we're actually doing it, or shortly before it.

We try to provide ample warning when we can, but we had some unforeseen problems earlier that caused significant delay and more downtime than anticipated, and Gramphos already gave a heads-up earlier that there will be continued high-load for a little while longer. We'll try to keep it to a minimum but can't give an exact estimate: we're working with a new system that we're not exactly 100% familiar with yet, so we're still figuring things out ourselves.

We once again apologise for the inconvenience, but as snoopy points out this is important work that needs to be done sooner rather than later.

[Wezil]

Staff -

I understand and apologize if I appeared somewhat grumpy. It's frustrating to use a site that isn't working right.

[Asher]

Originally posted by Locutus
We are not a professional community -- that would imply we get paid for this, when in reality the reverse is the case.

I said quasi-professional because you do offer a paid service.

We try to provide ample warning when we can

Please don't patronize me. It would not have killed you guys to say "We're doing maintenance today at X hour." No one prevented that, you act like this was out of the blue completely and urgent. So urgent that you couldn't even post an announcement WHILE service was disrupted on the site...

, but we had some unforeseen problems earlier that caused significant delay and more downtime than anticipated, and Gramphos already gave a heads-up earlier that there will be continued high-load for a little while longer.

It is not a "heads-up" when it comes hours after the disruption starts.

[Locutus]

Originally posted by Wezil
Staff -

I understand and apologize if I appeared somewhat grumpy. It's frustrating to use a site that isn't working right.

No problem, that's perfectly understandable. We feel the same way

[Gramphos]

I'm doing some additional work in a few minutres, wich will require the database to be disabled.

[Gramphos]

Hopefully any complications should be gone now.

[AAHZ]

is this all prep for VB3?

like backing up databases, i understand, is a big step to updating forum software and stuff?

i am very anxious to see the final product.

[Gramphos]

Partly. But we are seeing over our backup routines for improving the data security, and today's maintenance were aimed at making it possible to backup the database with almost no noticeable effect on the site. (Possible increased load as data is transferred over the network, and a short delay as we take a snapshot of the database).

We will continue with this overhaul of our backup system and make sure that our data is safe, and once that is finished we will focus entirely on vB3.