Performance

[Buck]

The forums are loading at a snail's pace for me. Has the site been unusually slow lately for anyone else? Or is it my DNS?

[Mike]

What does "lately" mean? Just today? Or the last couple of days?

[sir archely]

slow here too. and i can actually say that i'm on a good connection on my end, so it's not me. since friday for me. well, friday is when i could actually discern that it wasn't me.

[Buck]

It's been going on for almost a week and steadily getting worse. But at the same time it is inconsistent. Sometimes it works fine, and other times it hangs for a while. The hang time is the part that last for longer and longer. As an example, just now I opened up the quill. The main forums page loaded right away with absolute no delay, but then when I clicked on "Our Site" it took about 5 seconds to load. Earlier today when I made the first comment I tried to open Absnot and it hung for a good 20 seconds.


edit - and when I posted this post though quick replay it just took close to 10 seconds to go through.

[Mike]

You never know what to expect when it comes to performance. But when that's the case, it means it sucks

[Buck]

Mike, I've noticed that a similar performance hit on another forums I am a part time moderator on and we finally zeroed in on the source of the problem. Yahoo and Google have upped their competitiveness this year on indexing sites. Yahoo specifically is trying very hard to out index Google and by doing so is severely effecting the load times on that site. I've noticed that Yahoo Slurp Spiders are massing at the quill too and thought this might be a similar issue.

The site I am talking about handles quite a bit more users at once, and they had about 400+ Yahoo Slurp Spiders on at any time prior to taking the reigns. Not sure if you guys are doing this yet, but I'm copy/pasting the solution their admin:


Go to the server's robot.txt file and type the following:
User-agent: Slurp
Crawl-delay: 10

The numerical value can be anything you want, it represents how often the Yahoo Slurp Spider can hit your site. The above means once every 10 seconds. The admin I mention also recommends on smaller sites that you can easily increase the number to above 50.

She also said that it may take a couple days for all the Yahoo Spiders to fully recognize this change so don't expect a dramatic change after 5 minutes, but an improvement in load times is definitely noticeable after a couple days.


Again, I have no idea if that is the issue at the quill, but I'm mentioning this just in case.

*bows towards admins and lets them do that thing they do*

[Mike]

The way Yahoo behaves is totally ridiculous. They're not gathering information, they're taking whatever they can find at any cost. I guess it's about time to ban Yahoo alltogether