Referrer Spam

[Roy Schestowitz]

Judging by the statistics of 3 domains, it seems like in the past month or
two spammers found a new method of aggravating society. It becomes hard to
analyse Web log because every second predominant entry is a link trying to
sell you cigarettes, Viagra and pr0n.

Is it just me, or do others experience the same problem? I am talking about
a real surge here and not a moderate rise. I can no longer see my referrers
because it's hard to distinguish spam and real citations. Is there a way to
prevent this? (I doubt it)

Roy

 

[David Dorward]

I can no longer see my
referrers because it's hard to distinguish spam and real citations. Is
there a way to prevent this? (I doubt it)

Write a bot (I'd use Perl with LWP) to visit each referer and look for a
link to your site, dump each entry in to the log to /dev/null or a new file
depending on the result. Not 100% reliable, but it should help.

 

[Ken]

Hi Roy -

Judging by the statistics of 3 domains, it seems like in the past month or
two spammers found a new method of aggravating society. It becomes hard to
analyse Web log because every second predominant entry is a link trying to
sell you cigarettes, Viagra and pr0n.

Is it just me, or do others experience the same problem? I am talking about
a real surge here and not a moderate rise. I can no longer see my referrers
because it's hard to distinguish spam and real citations. Is there a way to
prevent this? (I doubt it)

I'm seeing the same thing in my logs. I don't know what the sites are
selling because I refuse to visit such sites. (I know by the
documents being requested and the number of entries all right together
from widely scattered IP addresses that it isn't legitimate referrer
information.)

When I get a round tuit, I'm going to update my httpd.conf file to
watch for those referrers and do something other than returning the
requested documents. The documents are fairly large. Whatever I do
will be much smaller and will return a code other than 200.

 

[Roy Schestowitz]

Write a bot (I'd use Perl with LWP) to visit each referer and look for a
link to your site, dump each entry in to the log to /dev/null or a new
file depending on the result. Not 100% reliable, but it should help.

Is there something I can re-use? Do you have anything for this job yourself?
It seems like a fair bit of trouble for an unimportant convenience.

I am concerned about this phenomenon because it appears to grow rapidly,
from just a few 2 months ago, to dozens last month and hundreds this
month... some of them have deceiving names and a high number of referrals
(>50), so they are hard to ignore and just not follow. This only encourages
the spammers to proceed with this technique.

One would say this is rather like E-mail spam. You can no longer find your
referrer among the 'noise'. What makes it worse is the fact that there is
no message body, subject line, etc. to judge the source by...

Roy

 

[lostinspace]

----- Original Message -----
From: "Roy Schestowitz" <>
Newsgroups: alt.html
Sent: Saturday, March 19, 2005 12:20 PM
Subject: Re: Referrer Spam

Is there something I can re-use? Do you have anything for this job
yourself?
It seems like a fair bit of trouble for an unimportant convenience.

I am concerned about this phenomenon because it appears to grow rapidly,
from just a few 2 months ago, to dozens last month and hundreds this
month... some of them have deceiving names and a high number of referrals
(>50), so they are hard to ignore and just not follow. This only
encourages
the spammers to proceed with this technique.

One would say this is rather like E-mail spam. You can no longer find your
referrer among the 'noise'. What makes it worse is the fact that there is
no message body, subject line, etc. to judge the source by...

Roy

--

Roy,
There's not much you can do to eliminate log spamming. You may
however reduce it over time.
The old default of referrals being included is getting less and less, even
from genuine visitors who use software's which blank out the referrer field.
I've used referrer's in the past to assist in expanding content that
visitors where looking for, some times, even on the fly.

"jdmorgan" in the Webmaster World forums offers a solution for a server
rather than a hosted web site. (I failed to record the reference.)
Jim moderates a forum:
http://www.webmasterworld.com/forum92/
He also occasionally pops in to:
http://www.webmasterworld.com/forum11/index.htm

Jim stays quite busy and generally will only assist folks who've show some
type of effort in a solution and offer that solution in the forum. I suggest
searching the archives and making inquiries from there.