Wiki Spam Report

Discussion in 'Ruby' started by Jim Weirich, Dec 13, 2004.

  1. Jim Weirich

    Jim Weirich Guest

    Wiki Spam Report
    ----------------

    I thought I would take some time and report on the wiki spam situation
    on RubyGarden. As I hope you have noticed, the wiki has been
    remarkably spam free. This email will tell you what measures we have
    taken to get to this point.

    But first ...

    Some Numbers
    ------------

    Over the past 10 days, we have had:

    93 updates to the wiki page, all (AFAICT) spam free.
    (although I might have missed spotting some).

    46 updates to the wiki tarpit. Of those, we had ...
    3 innocent updates
    2 questionable updates
    1 update by me
    40 spams

    The Mechanism
    -------------

    Spammers are automatically routed to a wiki tarpit. The tarpit is an
    (almost) exact copy of the real RubyGarden wiki. Making changes to
    the tarpit looks as if you are making changes to the real wiki. And
    since spammers get their pages from the wiki, it looks like (to them)
    that they have successfully spammed our site.

    However, everyone else never gets to see the spam.

    By tricking the spammers into thinking they are successful, they don't
    put any additional effort into bypassing our spam detection criteria.
    This is important! When we explicitly denied them access to the wiki,
    then went to great lengths to figure out how to get around the
    restrictions. I haven't seen any of that kind of probing with the
    tarpit.

    Detecting Spammers
    ------------------

    The current spammer detection logic is based on two observations:

    (1) Spammers almost never use an IP address that has reverse lookup
    enabled. This effectively means that it appears (to the wiki
    software) that your host name looks like a numeric IP address.

    (2) Spammers almost never set user preferences on the wiki.

    So if both of these conditions are true, we treat the access as a spammer
    and send it to the tarpit.

    Now this isn't perfect, but that's OK. We also have a explicit ban
    list for spammers who pass one of (1) or (2) above. And we have an
    explicit allow list that overrides the automatic spammer detection.

    Innocent Users
    --------------

    Can innocent users get trapped by the Tapit? The short answer is yes.
    However, we are monitoring the tarpit and will attempt to rescue such
    users.

    In the past 10 days, there were at least 3 page updates that were from
    innocent users. One guy (bless his heart) even removed some spam from
    the tarpit for us.

    When I see innocents trapped in the tarpit, I add their IP address to
    the allow list and manually update the wiki with their changes (if
    they are significant).

    Detecting the Tarpit?
    ---------------------

    The tarpit is deliberately designed to look like the original wiki, so
    it is sometimes difficult to tell when you are trapped. Here's some
    suggestions.

    You are probably in the Tarpit when:

    * there are a lot of recent updates made with numeric IP addresses
    rather than host names.

    * a lot of the pages have spam.

    Although neither of these suggestions are foolproof. I refresh the tarpit
    from the real wiki occasionally (to keep it looking realistic).
    Immediately after a refresh it is /very/ difficult to tell the difference.

    If you think you are trapped by the tarpit, send me
    () an email with your IP address and I will check
    the logs. If you are trapped, we can add your IP address to the allow
    list.

    If you are worried about getting caught in the tarpit, just make sure you
    have your user preferences set when accessing the tarpit (click on the
    preferences link from any wiki page).

    Summary
    -------

    I am pretty happy with the current wiki situation. In fact, the
    tarpit has been so successful, that I am considering lifting the ban
    on lower case http. The ban currently isn't buying us any benefits
    and is rather annoying (I'll make it so both upper and lower case
    work).

    Thanks for your time.

    --
    -- Jim Weirich http://onestepback.org
    -----------------------------------------------------------------
    "Beware of bugs in the above code; I have only proved it correct,
    not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
    Jim Weirich, Dec 13, 2004
    #1
    1. Advertising

  2. Jim Weirich

    martinus Guest

    That is a very cool idea! But I am afraid this posting is the reason
    why http://www.rubygarden.org/ruby currently is under attack from a lot
    of spammers.

    martinus
    martinus, Dec 13, 2004
    #2
    1. Advertising

  3. On Tue, Dec 14, 2004 at 06:27:18AM +0900, martinus scribed:
    > That is a very cool idea! But I am afraid this posting is the reason
    > why http://www.rubygarden.org/ruby currently is under attack from a lot
    > of spammers.


    Are you sure you didn't slip into the tarpit? :) It looks fine
    from here.

    -Dave wonders if it might not be nice to have multiple
    spammer / legitimate user detection heuristics, though.
    David G. Andersen, Dec 13, 2004
    #3
  4. On Tue, 14 Dec 2004 03:21:02 +0900, Jim Weirich <> wrote:
    > Wiki Spam Report
    > ----------------
    >
    > I thought I would take some time and report on the wiki spam situation
    > on RubyGarden. As I hope you have noticed, the wiki has been
    > remarkably spam free. This email will tell you what measures we have
    > taken to get to this point.
    >


    What does spam look like on a wiki site?

    Thanks,
    Ed
    --
    Pretty women make us buy beer, ugly women make us drink beer
    Edgardo Hames, Dec 14, 2004
    #4
  5. Jim Weirich

    James Britt Guest

    Edgardo Hames wrote:
    > On Tue, 14 Dec 2004 03:21:02 +0900, Jim Weirich <> wrote:
    >
    >>Wiki Spam Report
    >>----------------
    >>
    >>I thought I would take some time and report on the wiki spam situation
    >>on RubyGarden. As I hope you have noticed, the wiki has been
    >>remarkably spam free. This email will tell you what measures we have
    >>taken to get to this point.
    >>

    >
    >
    > What does spam look like on a wiki site?



    The junk I'm accustomed to seeing are pages devoid of any Ruby content,
    but filled with links to sites apparently hawking Natural Nigerian Rolex
    Enhancements.

    James
    James Britt, Dec 14, 2004
    #5
  6. Jim Weirich

    Jim Weirich Guest

    On Monday 13 December 2004 10:30 pm, Edgardo Hames wrote:
    > On Tue, 14 Dec 2004 03:21:02 +0900, Jim Weirich <>

    wrote:
    > > Wiki Spam Report
    > > ----------------
    > >
    > > I thought I would take some time and report on the wiki spam situation
    > > on RubyGarden. As I hope you have noticed, the wiki has been
    > > remarkably spam free. This email will tell you what measures we have
    > > taken to get to this point.

    >
    > What does spam look like on a wiki site?


    You asked ... Here's a link to an old page on the RubyGems wiki. Scroll down
    to the bottom of the page.

    http://rubygems.rubyforge.org/wiki/wiki.pl?action=browse&id=DeveloperGuide&revision=78

    --
    -- Jim Weirich http://onestepback.org
    -----------------------------------------------------------------
    "Beware of bugs in the above code; I have only proved it correct,
    not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
    Jim Weirich, Dec 14, 2004
    #6
  7. Jim Weirich

    Curt Sampson Guest

    On Tue, 14 Dec 2004, Jim Weirich wrote:

    > ...


    It looks like the sandbox is not working for 221.197.18.150. If you look
    at RecentChanges, he's changed a lot of pages to add what seems to be
    mostly Chinese links to chinese websites. I've set preferences, and I'm
    still seeing it.

    cjs
    --
    Curt Sampson <> +81 90 7737 2974 http://www.NetBSD.org
    Make up enjoying your city life...produced by BIC CAMERA
    Curt Sampson, Dec 14, 2004
    #7
  8. Jim Weirich

    Jim Weirich Guest

    On Monday 13 December 2004 11:46 pm, Curt Sampson wrote:
    > It looks like the sandbox is not working for 221.197.18.150. If you look
    > at RecentChanges, he's changed a lot of pages to add what seems to be
    > mostly Chinese links to chinese websites. I've set preferences, and I'm
    > still seeing it.


    Curt is looking at the RubyGems wiki hosted by RubyForge (which has little to
    no spam protection ... hopefully that will change in the near future).

    The tarpit is for the RubyGarden wiki.

    --
    -- Jim Weirich http://onestepback.org
    -----------------------------------------------------------------
    "Beware of bugs in the above code; I have only proved it correct,
    not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
    Jim Weirich, Dec 14, 2004
    #8
  9. Jim Weirich

    martinus Guest

    It was fine again after about 5 minutes, it sees that the system really
    works
    And no, i didn't slip into the tarpit :)

    martinus
    martinus, Dec 14, 2004
    #9
  10. Jim Weirich <> wrote:
    > 46 updates to the wiki tarpit. Of those, we had ...
    > 3 innocent updates
    > 2 questionable updates
    > 1 update by me
    > 40 spams


    Very gratifying :)

    martin
    Martin DeMello, Dec 14, 2004
    #10
  11. Jim Weirich

    slonik AZ Guest

    Each act of applying a change to a Wiki can be seen as a "message".
    One can use email spam filter (with some modifications, of course) as
    a first line of defense. If a proposed wiki change looks as spam to
    "modified email spam filter" a user is confronted with a set of
    challenges such as read letters from a distorted bitmap, answer silly
    questions like Who invented Ruby?... etc.
    Alternatively, spam filter decides to silently forward the change to
    the Tarpit and also notify Wiki admin of the change if the probability
    of this message to be a legitimate one is higher that certain
    threshold.

    --Leo--


    On Tue, 14 Dec 2004 20:17:23 +0900, Martin DeMello
    <> wrote:
    > Jim Weirich <> wrote:
    > > 46 updates to the wiki tarpit. Of those, we had ...
    > > 3 innocent updates
    > > 2 questionable updates
    > > 1 update by me
    > > 40 spams

    >
    > Very gratifying :)
    >
    > martin
    >
    >
    slonik AZ, Dec 14, 2004
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. chris
    Replies:
    4
    Views:
    873
    chris
    May 17, 2004
  2. Eric @ Zomething

    Python Wiki & wiki Hosting?

    Eric @ Zomething, Jun 5, 2004, in forum: Python
    Replies:
    2
    Views:
    513
  3. Eric S. Johansson

    Re: Python Wiki & wiki Hosting?

    Eric S. Johansson, Jun 6, 2004, in forum: Python
    Replies:
    6
    Views:
    484
    Paul Boddie
    Jun 8, 2004
  4. Mahrt, Dallas

    RE: Python Wiki & wiki Hosting?

    Mahrt, Dallas, Jun 7, 2004, in forum: Python
    Replies:
    0
    Views:
    409
    Mahrt, Dallas
    Jun 7, 2004
  5. Replies:
    0
    Views:
    310
Loading...

Share This Page