Server discussion... - Page 5 - Calgarypuck Forums - The Unofficial Calgary Flames Fan Community

photon · 09-21-2009, 10:47 PM

Guests always outnumber readers 2:1, if you ban guests that's counter to what we want to do and they'll just register.

It's just simply there were over 600 people on during the game and almost 800 on at the end, not something that was normal last year. Partially because the game isn't on TV but people are streaming it from the Flames website and sitting at their PCs.

There are a few optimizations we made last year that I undid over the summer due to some circumstances, but I can re-do them and that should help.

EDIT: Over 800 now, but working better, everyone's not trying to post after every result in the shootout now

Yeah_Baby · 09-22-2009, 11:12 AM

Plus if you ban guests, the influx of posters not familiar with CP culture will drive us all batty.

Barnes · 09-22-2009, 01:02 PM

Have you guys checked out Amazon's services? I don't really know much about this stuff but we moved our site to it and have saved a considerable amount and greatly improved the speed.

http://aws.amazon.com/ec2/

You pay by the hour and can add or remove instances of your server depending on traffic. You can scale your site from one server instance to thousands in minutes.

We have a site that people use our images on an a very popular forum and a Facebook app with 33,000 users and our costs are WAAYYY down.

photon · 09-22-2009, 02:29 PM

Yeah one of the changes that we had made and that I had undone was having all our images and static files on a CDN like Amazon's Cloudfront. Dramatically reduces the # of requests to our server, so rather than having to serve up every image and javascript file the server can just worry about serving up php files.

I looked at using EC2 to host CP entirely, the difficulty came in the load and traffic generated.. The monthly costs were still quite a bit more than we are paying now, even on other services that are cheaper than Amazon. Even on the low CPU instance assuming 22 hours a day 7 days a week it's still more expensive than our colocation now, and there's no way we'd get by on that, our current server is a 2 CPU 3.2GHz box with SCSI drives and we're CPU limited right now during the slowdowns.

Mid term we might have to upgrade the CPUs in the server.. of course since we have an actual server, the CPUs aren't cheap ($1700 for a Quad Core CPU I think). I've been watching eBay though and the cost of dual core CPUs that would fit this machine have been coming down into the sub $500 range.

Short term though getting all the static files off the server back onto a CDN will help a lot.

teamchachi · 09-25-2009, 04:12 PM

Quote:

Originally Posted by photon

Yeah one of the changes that we had made and that I had undone was having all our images and static files on a CDN like Amazon's Cloudfront. Dramatically reduces the # of requests to our server, so rather than having to serve up every image and javascript file the server can just worry about serving up php files.

I looked at using EC2 to host CP entirely, the difficulty came in the load and traffic generated.. The monthly costs were still quite a bit more than we are paying now, even on other services that are cheaper than Amazon. Even on the low CPU instance assuming 22 hours a day 7 days a week it's still more expensive than our colocation now, and there's no way we'd get by on that, our current server is a 2 CPU 3.2GHz box with SCSI drives and we're CPU limited right now during the slowdowns.

Mid term we might have to upgrade the CPUs in the server.. of course since we have an actual server, the CPUs aren't cheap ($1700 for a Quad Core CPU I think). I've been watching eBay though and the cost of dual core CPUs that would fit this machine have been coming down into the sub $500 range.

Short term though getting all the static files off the server back onto a CDN will help a lot.

The costs of hosting on EC2 are coming down dramatically. Particularly when you look at the new reservation options that are available. However, we are doing things on a shoe-string here so it still isn't really in our range just yet...

Kipper is King · 09-25-2009, 07:04 PM

^^^ could there be some sort of a way for users to donate for new servers somehow? Like a Paypal thing? It sounds like if the CP users who cared enough donated a few bucks, that would make a significant upgrade much more feasible.

photon · 09-25-2009, 08:20 PM

That's how we got the server that we are on in the first place.

It's just not something one wants to abuse, and something one does not want to do lightly.

Plus it's complicated.

Getting a new server through donations would get us the ability to handle a lot more users, but it would also double the monthly costs to have those servers running.

Upgrading the CPUs to dual core or quad core processors would alleviate the current bottleneck, but personally if I'm going to think of recommending a donation drive to buy CPUs then I sure as heck want to KNOW that that will fix things properly, not buy us 3 months and then be back in the same spot. Especially since the server processors are quite expensive.

I'm not convinced it'll be a big issue once the season starts either, we never had nearly that # of people in game and post game threads all last year, once TV broadcasts and bars showing the game and all that jazz starts up I think we'll see a big drop in the # of online users during and immediately after games.

If it keeps being 800 people online during a game, then we'll have to look at things.

Azure · 09-26-2009, 10:14 AM

Part of the problem certainly has been a result of the NHL showing all the games online, coupled with the Fleury experience, and a few other things.

More people WILL be watching the games on TV when the season starts. That is inevitable, considering the games aren't even being shown on TV right now. But, one has to think about the growth of CP as a whole since last year. Is it enough to need a server upgrade, either with the CPU or a new server? Well, thats what you're here for.

Azure · 09-27-2009, 12:03 PM

Well, the game was on TV last night, and CP was doing a lot better.

Anyone else notice the same thing?

Buff · 09-28-2009, 10:51 AM

Quote:

Originally Posted by LockedOut

Can we get rid of guests to speed up the site if it helps. At least during the game. It's pretty slow and this during just an exhibition game. As I type this, in OT guests outnumber us by about 2 to 1.

I only signin if I want to post something. There are other people who do the same. Getting rid of guests probably wouldn't make a noticeable difference.

teamchachi · 10-01-2009, 12:51 AM

We're back to the old problem of having to capacity plan for a 99th percentile event.

Yes, it would be great if the server was "snappy" when there are 1000+ concurrent users hammering on it. However, the costs associated with additional hardware, colocation, and bandwidth are prohibitive. What we have works very well and has tons of excess capacity 99% of the time.

Do we really want to have to raise a bunch of money and then double (at least) our monthly costs just so the server can perform well during "one off" events?

SebC · 10-02-2009, 03:42 PM

Any update on the 174 Shaw problem? I'm still not getting whole pages most of the time.

photon · 10-02-2009, 04:26 PM

The last note I had from them was it was still a work in progress, getting the new direct connection from Seattle that is (which should resolve all the issues).

I asked if I could contact Cogent myself but they never replied to my email..

Azure · 10-02-2009, 08:08 PM

Quote:

Originally Posted by teamchachi

We're back to the old problem of having to capacity plan for a 99th percentile event.

Yes, it would be great if the server was "snappy" when there are 1000+ concurrent users hammering on it. However, the costs associated with additional hardware, colocation, and bandwidth are prohibitive. What we have works very well and has tons of excess capacity 99% of the time.

Do we really want to have to raise a bunch of money and then double (at least) our monthly costs just so the server can perform well during "one off" events?

I think, and most people here would probably agree, that there is some importance in CP being accessible when something big happens.

But, I think most people would also agree that adding a bigger and better server just to make CP accessible for 1% of the time, when its just fine 99% of the other time, especially when it would increase costs, makes absolutely no sense.

So, what are the options? Obviously its a concern that the server is having issues when over a 1,000 people connect, and I suspect that as the season goes on, and the Flames are as successful as we all hope, more and more people will sign up or start coming to CP.

Photon talked about adding a bigger processor? More memory? Would it cost more money to simply modify the server we have now? We have enough bandwidth right?

photon · 10-03-2009, 10:25 AM

Yeah as the community grows eventually we will have to do something, it's just a question of when.. the growth has been steady but we're still a ways from having 1000 people on during typical games.

The cost to modify the server we have now would be almost the same as adding a second server, so I'd be more inclined to lean towards the second option, but as has been pointed out that would double the monthly costs as well.

Ultimately it's something Bingo has to decide as he sees all the income and expenses.

teamchachi · 10-03-2009, 06:05 PM

Quote:

Originally Posted by Azure

I think, and most people here would probably agree, that there is some importance in CP being accessible when something big happens.

But, I think most people would also agree that adding a bigger and better server just to make CP accessible for 1% of the time, when its just fine 99% of the other time, especially when it would increase costs, makes absolutely no sense.

So, what are the options? Obviously its a concern that the server is having issues when over a 1,000 people connect, and I suspect that as the season goes on, and the Flames are as successful as we all hope, more and more people will sign up or start coming to CP.

Photon talked about adding a bigger processor? More memory? Would it cost more money to simply modify the server we have now? We have enough bandwidth right?

Unfortunately vBulletin requires raw horsepower. Each page is generated dynamically and that involves PHP execution and a lot of MySQL queries.

While it is theoretically possible to upgrade the existing server, we can buy a much faster server for just a bit more than the upgrade would cost.

Ideally, we would keep the existing server as a MySQL server and bring on a new server to deal with Apache/PHP.

Unfortunately, that requires an additional 1U of colocation space which doubles our monthly costs.

Increasing our capacity is technically easy, but it is really up to Bingo to decide as he's the guy who is in charge on the cash.