Blog Service

Update: the problem seems to be a little different than initially advertised and the server people have re-described the reason for the shutdown. The problem is due to the number of bytes served; it’s not due to a DOS attack. We didn’t get a 1-2 million hit spike after all. The problem is that we’re running at over 1 GB being served every day. For example, two weeks ago the weekly total from the stat sheet was 7 487 504 903 bytes. It seems like a lot for a blog.

The blog seems to be under some sort of attack. We suddenly started receiving 1-2 million hits per day and the service provider shut down the site. It’s been restored under a watch. I’ve obtained access logs from the service provider, maybe that will provide a clue. The access logs are 9-10 MB per day. Does anyone know how to analyze these things?


25 Comments

  1. David H
    Posted Oct 26, 2006 at 4:09 PM | Permalink

    Getting ready for Stern maybe?

  2. David Smith
    Posted Oct 26, 2006 at 7:43 PM | Permalink

    Looks like eco-terrorism

  3. Nicholas
    Posted Oct 26, 2006 at 8:02 PM | Permalink

    I usually use a PERL script to analyze apache logs (I’m assuming it’s apache), but there are plenty of apache-log analysing tools out there – mostly for Linux. You could try searching http://www.freshmeat.net (Linux application database) for “apache log”, like this. Some of the results are relevant.

  4. Paul
    Posted Oct 26, 2006 at 8:33 PM | Permalink

    Digg?

  5. Posted Oct 26, 2006 at 8:39 PM | Permalink

    Steve,

    If you were getting 1-2 million hits per day, you should be gratified. If on the other hand, you were under attack, you should wear that as a badge of honor.

  6. cbone
    Posted Oct 26, 2006 at 9:09 PM | Permalink

    Woo hoo! Your first DOS attack! Congratulations, you must have struck a nerve! Keep up the good work.

    CBone

  7. Bob K
    Posted Oct 26, 2006 at 10:03 PM | Permalink

    Congratulations Steve.

    Probably done by a student who can’t stand being shown their king has no clothes.

    Too much time on their hands and too little consideration for others.

  8. Wm. L. Hyde
    Posted Oct 26, 2006 at 10:06 PM | Permalink

    I tried all day to get on. That was the first thing I thought of. Dirty tricks!….theoldhogger

  9. Dave Dardinger
    Posted Oct 26, 2006 at 10:47 PM | Permalink

    I note that on the pages I get from CA tonight, below the line for Spam Karma there’s a WordPress database error:

    WordPress database error: [Can't open file: 'smc_ss_stats.MYI' (errno: 144)]
    INSERT INTO smc_ss_stats (remote_ip,country, language, domain,referer, resource, user_agent, platform,browser, version,dt) VALUES (‘71.35.67.113′, ‘Indeterminable’, ‘en-us’, ‘climateaudit.org’, ‘http://www.climateaudit.org/’, ‘/?p=874′, ‘Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax)’, ‘Windows 2000′, ‘Netscape’, ‘7.2’, 11619241

    Note I inserted spaces after many commas to make the thing wrap.

  10. Gerhard H. Wrodnigg
    Posted Oct 27, 2006 at 2:18 AM | Permalink

    Steve, there are many (commercial) tools to analyze logfiles, also for searching for attackers.

    I used to do it with PERL (as Nicholas mentioned), but I think also R could work…

    (btw, the WordPress database error is still on bottom of the page)

  11. Stan Palmer
    Posted Oct 27, 2006 at 5:23 AM | Permalink

    For DOS atttacks, are there any voluntary organizations which track them as there are quite a few which track SPAM. Both SPAM and DOS attacks are based on networks of zombie PCs so the problem is basically the same.

  12. KevinUK
    Posted Oct 27, 2006 at 6:09 AM | Permalink

    Steve,

    I assume you’ll want something thats free so try AWStats. Below is the link to the web site

    http://www.awstats.org/

    It runs on Windows or Linux. If you are using Windows, then you’ll need to install ActivePerl first which you can find here

    I use it on my web hosting account and it produces lots of good detailed reports.

    Onc eyou’ve identified who the culprit/culprits (IP address(es) of machines involved in the DOS attack) are let me know as I know a few people who can help trace the b*****ds.

    Best of luck

    KevinUK

  13. Jaye
    Posted Oct 27, 2006 at 6:41 AM | Permalink

    DOS attacks by the “open minded” crowd. Pathetic but typical.

  14. Paul
    Posted Oct 27, 2006 at 6:53 AM | Permalink

    Steve,

    On other thing…at some point, it came up with an error that appeared to have your “regular” email address. You might want to check to be sure you’ve not opened yourself up to a spam attack, too…

  15. Curt
    Posted Oct 27, 2006 at 7:51 AM | Permalink

    1-2 million hits per day? It’s just “natural variation”…

  16. welikerocks
    Posted Oct 27, 2006 at 8:48 AM | Permalink

    #15 Curt
    And isn’t only natural that the Culprit returns to the scene of the Crime?
    Concern Climate Scientist
    Your “C” names aren’t very original.
    Even your real name is a double C, eh?
    Sheesh.

  17. Steve McIntyre
    Posted Oct 27, 2006 at 10:49 AM | Permalink

    Update: the problem seems to be a little different than initially advertised and the server people have re-described the reason for the shutdown. The problem is due to the number of bytes served; it’s not due to a DOS attack. We didn’t get a 1-2 million hit spike after all. The problem is that we’re running at over 1 GB being served every day. For example, two weeks ago the weekly total from the stat sheet was 7 487 504 903 bytes. It seems like a lot for a blog.

  18. Chris H
    Posted Oct 27, 2006 at 11:08 AM | Permalink

    #17 I had a quick look at a couple of pages and a few images and as a rough estimate, I’d say the average page size, including images is around 100,000 bytes. At 1,000,000,000 bytes per day, that would be between 10,000 page downloads per day.

  19. Chris H
    Posted Oct 27, 2006 at 11:10 AM | Permalink

    Aaargh! between

  20. Michael J
    Posted Oct 27, 2006 at 11:28 AM | Permalink

    I bet Al Gore could have helped you resolve this Steve, After all, he did invent the Internet.

  21. P-Dog
    Posted Oct 27, 2006 at 11:48 AM | Permalink

    Why do people make claims of “eco-terrorism” without any evidence? Turns out it’s not even a DOS attack. Pony up, haters.

    And what dumba$$ still is attributing a quote that Gore didn’t even make to Gore? Pshhh.

  22. Gary
    Posted Oct 27, 2006 at 12:04 PM | Permalink

    Its interesting how some have jumped to the conclusion that the problem was a DOS attack before all the data were in. This ought to be a lesson for us.

    Yeah, I know even the paranoid can be right about having enemies, but let’s do the audit before we start blasting at shadows.

  23. Steve Sadlov
    Posted Oct 27, 2006 at 12:14 PM | Permalink

    I’ve witnessed way more embarassing things like this which involved large multinationals. We all have our “DOH!” moments. Good work with the root cause analysis guys! Not bad for a true “non profit!” (“anti profit” is probably the appropriate term, thanks to Steve M’s personal financing of all this!)

  24. Steve McIntyre
    Posted Oct 27, 2006 at 12:32 PM | Permalink

    #18. That’s a pretty plausible logical explanation. There are a lot of images on the site and that would be what’s doing it. It does result in a LOT of server volume relative to most blogs although realclimate would presumably have similar or greater server volume.

    #21. I do not purport to be a computer expert and relied on information that I was given during a shutdown when we were trying to resolve things as quickly as possible. Dealing with the service provider is like pushing on a string to get information. In order to diagnose things, I proceeded to try to trace the information back to original data and in the process detrermined that I had been given some inaccurate information and we arrived at a correct diagnosis. That’s why you check things.

  25. welikerocks
    Posted Oct 27, 2006 at 1:05 PM | Permalink

    “P-dog” Friend of Curt? ;)

    So yep, it’s official! You can call Mrs.Rocks, WeAreParanoid from now on.
    I’ll own it. I am. I am. Skeptial, suspicious, and paranoid officially now I guess. Might be a good screen name after all.

    Especially when my husband, the environmental scientist asks what are the definitive papers re GW and AGW and we can’t seem to find one without errors and fudging. That’s just sad and disturbing. Especially when work like CA is doing gets ignored or not done in the review process or by the authors themselves. ( yay AGU!)

    A hater? Not us, not me. Hate is a useless-awful emotion. That word is thrown around much too easily these days, IMHO. I’ve seen it, who uses it and who it comes from. And I’ve made up my mind about it. This also is-my opinion.

    I am glad the board glip wasn’t sinsister as I thought it was!!
    Sincerely! :)

Follow

Get every new post delivered to your Inbox.

Join 3,203 other followers

%d bloggers like this: