HalfEagle Bloggers – I need your help.

I got an email last night from one of the bloggers whose content is included on HalfEagle. It seems there maybe a problem with how HalfEagle checks your site for updated content. The problem? HalfEagle may be accessing your site far more often than necessary.

First, I’m very sorry if this has affected your server. I manage a number of servers and I know how frustrated I get every time I discover some poorly programmed script hammering away at one of my servers.

Also, I fully realize that this is most likely caused by a script which I poorly programmed. That’s an embarrasing thing to admit, but I’m not going to pretend like I’m the greatest programmer in the world. Hopefully, I’ll be good enough to find my mistake and fix it. 

Meanwhile, and this is the “I need your help” part, it will help me considerably if you could check your stats or log files and look for excessive visits from a particular IP address that starts with 72.3. When I say excessive, I mean more than a thousand visits a day. If you are affected, it should stand out.

Normal usage is for the HalfEagle “bot” to swing by and check your feed every 15-45 minutes or so.

If you find anything, please email it to me directly at gregg (at) halfeagle.com.

Thank you.

P.S. Once this is all sorted out, I’ll post back with more information. Unfortunately, I leave for summer camp tomorrow. So if it isn’t fixed before then, I will disable all updates on HalfEagle until I get back and can determine the issue. I hope it doesn’t come to that, but I simply cannot allow this site to cause any undue stress on anyone else’s server.

4 responses

  1. Good news! With the help of the affected site owner, I’ve isolated the section of code involved. I’m working on a long term fix, but have disabled that code temporarily for his site.

    I also confirmed that the error was not occurring on one of the other blogs that HalfEagle carries, so it might be isolated to just the one site. Please email me if your site seems to be affected and I’ll fix it ASAP.

  2. More details. 🙂 Another site owner has shared with me that they are also affected. I’ll include some of what I told him:

    > The good news is those excessive hits don’t request the entire page, they just check to see if the URL is a redirect. (With a bandwidth friendly HEAD-type request.)
    > Some blog RSS software (like Feedburner) generates new random redirect URLs on a frequent basis and it was causing some blogs posts to show up multiple times on the HalfEagle home page. I added a check to resolve any redirects up front to prevent anyone from accidentally gaming the home page of HalfEagle.
    > Unfortunately, I didn’t fully realize how many extra hits that redirect resolution would generate. I’m going to throttle everything back significantly, and look at some other ways of dealing with that problem.

    One immediate affect of this throttling back is that it may take longer for new posts to show up on HalfEagle. I tried getting this down to between 10-20 minutes, but I’ve bumped it back up to an hour. This also delays the auto twitter promotion by @halfeaglebot.

    I’ve also increased cache durations in a few places and generally improved every bit of code I’ve run across. I sincerely believe in “It’s not a problem, it’s an opportunity.” 🙂

    There will be more significant code changes when I get back from camp however, because I sincerely believe that a faster update will help promote everyone’s blog entries. Plus, as a blogger myself, I like to see my announcements show up while I still have the entry fresh in my mind. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *