anti-spam measures

Discussion in 'Java' started by Roedy Green, Jun 24, 2004.

  1. Roedy Green

    Roedy Green Guest

    I had a bit of a fright the other day. I thought for a while I was
    under a email denial of service attack. I wondered if I would ever be
    able to post even a munged public email address ever again.

    Surely there are such people.

    In like manner, I can see how spammers with political, religious,
    pornographic, enfant terrible, or commercial interests will gradually
    make the newsgroups and standard email totally unusable. As my Dad you
    used say all the time, "watch the derivative".

    We can't wait until that happens before taking action.

    I see a multi-pronged approach will be necessary:

    1. legal means. However but spammers will be able to hide anywhere on
    earth. Surely some third world country will harbour them just as the
    Cayman Islands harbours crooked companies. With the net, they can set
    up shop in SomethingIstan and have effective storefronts in every
    country.

    2. boycotts. We must educate people to ensure spammers DON'T get
    whatever it is they want from spamming.

    3. technology. I see a new email delivery system evolving to
    completely replace POP3/SMTP. It will have a number of features.

    a. automatic encryption, compression, digital signing.
    b. full use of the 8-bit channels.
    c. a sender pays receiver system so any spam that does leak through
    still costs the spammer.
    d. the best anti-spam thinking that is built in, suitable for
    technopeasants.
    e. suitable for exchanging large files, and common files.
    f. ways to protect against denial of service attacks.
    g. designed from the ground up for technopeasants. Everything is
    automatic.

    The original email system was cooked up overnight as a demo. The
    author surely never dreamed his system would be used almost unmodified
    for planetary email scheme. It needs a major overhaul.



    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Jun 24, 2004
    #1
    1. Advertising

  2. Roedy Green

    Job Numbers Guest

    "Roedy Green" <> wrote in message
    news:...
    >I had a bit of a fright the other day. I thought for a while I was
    > under a email denial of service attack. I wondered if I would ever be
    > able to post even a munged public email address ever again.
    >
    > Surely there are such people.
    >
    > In like manner, I can see how spammers with political, religious,
    > pornographic, enfant terrible, or commercial interests will gradually
    > make the newsgroups and standard email totally unusable. As my Dad you
    > used say all the time, "watch the derivative".
    >
    > We can't wait until that happens before taking action.
    >
    > I see a multi-pronged approach will be necessary:
    >
    > 1. legal means. However but spammers will be able to hide anywhere on
    > earth. Surely some third world country will harbour them just as the
    > Cayman Islands harbours crooked companies. With the net, they can set
    > up shop in SomethingIstan and have effective storefronts in every
    > country.
    >
    > 2. boycotts. We must educate people to ensure spammers DON'T get
    > whatever it is they want from spamming.
    >
    > 3. technology. I see a new email delivery system evolving to
    > completely replace POP3/SMTP. It will have a number of features.
    >
    > a. automatic encryption, compression, digital signing.
    > b. full use of the 8-bit channels.
    > c. a sender pays receiver system so any spam that does leak through
    > still costs the spammer.
    > d. the best anti-spam thinking that is built in, suitable for
    > technopeasants.
    > e. suitable for exchanging large files, and common files.
    > f. ways to protect against denial of service attacks.
    > g. designed from the ground up for technopeasants. Everything is
    > automatic.
    >
    > The original email system was cooked up overnight as a demo. The
    > author surely never dreamed his system would be used almost unmodified
    > for planetary email scheme. It needs a major overhaul.


    Almost all of the original protocols need overhauls, but it doesn't look
    like it's going to happen anytime soon. Back then, we started with http,
    ftp, telnet, smtp.. and we still use http, ftp, telnet and smtp.

    The only solution is to standardize the small efforts in fighting spam and
    bring them all together so that every client and server is using them for
    free. Make it so ubiqitous and backwards compatible that you really can't
    find a reason to use the old stuff. While solutions do exist already, they
    aren't free (open-source) and the common public doesn't know them like they
    know apache http server, sendmail or ms outlook. That's the real problem.
     
    Job Numbers, Jun 24, 2004
    #2
    1. Advertising

  3. Job Numbers wrote:


    >
    > The only solution is to standardize the small efforts in fighting spam and
    > bring them all together so that every client and server is using them for
    > free. Make it so ubiqitous and backwards compatible that you really can't
    > find a reason to use the old stuff. While solutions do exist already,
    > they aren't free (open-source) and the common public doesn't know them
    > like they
    > know apache http server, sendmail or ms outlook. That's the real problem.


    SpamAssassin is free (www.spamasasin.org).
    This is an efficient antispam mail filter which is also used inside some
    commercial antispam solutions.

    Frederic
     
    =?ISO-8859-15?Q?Fr=E9d=E9ric_G=E9din?=, Jun 24, 2004
    #3
  4. Roedy Green

    Roedy Green Guest

    On Thu, 24 Jun 2004 05:56:27 +0200, Frédéric Gédin <>
    wrote or quoted :

    >SpamAssassin is free (www.spamasasin.org).
    >This is an efficient antispam mail filter which is also used inside some
    >commercial antispam solutions.


    I think SpamAssassin and the like has to be incorporated into
    MailServers to work, even if they are configured by the clients. One
    problem I am having is giant spams that take many minutes to download.
    Even if my client software filters them, they still tie up my Internet
    connection.

    To deal with them we need to detect spam on the fly and derail it
    before it even gets to the server.

    SpamAssasin is incorporated into the all Java James mailserver where I
    work, but I still have not seen that big a drop off in spam. Perhaps
    it takes a while to warm up. I have not yet been asked to configure
    anything or mark anything.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Jun 24, 2004
    #4
  5. Roedy Green <> wrote:

    > I think SpamAssassin and the like has to be incorporated into
    > MailServers to work, even if they are configured by the clients. One
    > problem I am having is giant spams that take many minutes to download.
    > Even if my client software filters them, they still tie up my Internet
    > connection.


    I'm working towards giving people 100% control of Spamassassin over the
    web. It already works on my users' accounts and we can shuffle spam off
    into a temporary folder to be double-checked later, or delete it, without
    the user having to download anything. But we're getting WAY offtopic here.

    --
    JustThe.net Internet & New Media Services, http://JustThe.net/
    Steven J. Sobol, Geek In Charge / 888.480.4NET (4638) /
    PGP Key available from your friendly local key server (0xE3AE35ED)
    Apple Valley, California Nothing scares me anymore. I have three kids.
     
    Steven J Sobol, Jun 24, 2004
    #5
  6. Roedy Green

    Roedy Green Guest

    On Thu, 24 Jun 2004 00:38:30 -0500, Steven J Sobol
    <> wrote or quoted :

    >
    >I'm working towards giving people 100% control of Spamassassin over the
    >web. It already works on my users' accounts and we can shuffle spam off
    >into a temporary folder to be double-checked later, or delete it, without
    >the user having to download anything. But we're getting WAY offtopic here.


    It is on topic is that we are talking about what will have to done to
    assure we can continue our Java discussions. I we are like frogs in
    boiling water we will be shut out of even a place to plan a counter
    attack.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Jun 24, 2004
    #6
  7. Re: OT: anti-spam measures

    On Thu, 24 Jun 2004 00:56:06 +0000, Roedy Green wrote:

    > I had a bit of a fright the other day. I thought for a while I was under
    > a email denial of service attack. I wondered if I would ever be able to
    > post even a munged public email address ever again.


    My biggest complaint against current spam filters is the number of false
    positives. Amongst the hundreds of messages marked as SPAM by my filter,
    there are always 5 or 6 legit emails. Likewise 10 or so SPAM get through
    my filter on a daily basis.

    La'ie Techie
     
    =?UTF-8?b?TMSByrtpZSBUZWNoaWU=?=, Jun 24, 2004
    #7
  8. Roedy Green

    Andy Fish Guest


    >
    > The only solution is to standardize the small efforts in fighting spam and
    > bring them all together so that every client and server is using them for
    > free. Make it so ubiqitous and backwards compatible that you really can't
    > find a reason to use the old stuff. While solutions do exist already,

    they
    > aren't free (open-source) and the common public doesn't know them like

    they
    > know apache http server, sendmail or ms outlook. That's the real problem.
    >
    >


    IMHO the only sensible way to solve the spam problem long-term is to
    introduce proper authentication for senders. Once you have that, you have a
    foundation to build on.

    I know it won't be easy and I can't see a simple way of making it backward
    compatible at the protocol level but I think reckon things will just get
    worse and worse until we decide we have no choice.
     
    Andy Fish, Jun 24, 2004
    #8
  9. Roedy Green

    Tim Ward Guest

    "Frédéric Gédin" <> wrote in message
    news:40da50ea$0$19036$...
    >
    > SpamAssassin is free (www.spamasasin.org).
    > This is an efficient antispam mail filter which is also used inside some
    > commercial antispam solutions.


    So? It's no use to me if I can't persuade my ISP to run it, which so far I
    haven't been able to.

    --
    Tim Ward
    Brett Ward Limited - www.brettward.co.uk
     
    Tim Ward, Jun 24, 2004
    #9
  10. Roedy Green

    Tim Ward Guest

    Re: OT: anti-spam measures

    "La?ie Techie" <laie@win_remove_get_nospam_solutions.com> wrote in message
    news:...
    >
    > My biggest complaint against current spam filters is the number of false
    > positives.


    I am aware of 0 false positives on one email system I use.

    This is because I don't look at the spam folder, I just delete the entire
    contents unexamined.

    --
    Tim Ward
    Brett Ward Limited - www.brettward.co.uk
     
    Tim Ward, Jun 24, 2004
    #10
  11. Roedy Green

    Alex Hunsley Guest

    Roedy Green wrote:

    > I had a bit of a fright the other day. I thought for a while I was
    > under a email denial of service attack. I wondered if I would ever be
    > able to post even a munged public email address ever again.


    [snippo]

    > a. automatic encryption, compression, digital signing.
    > b. full use of the 8-bit channels.
    > c. a sender pays receiver system so any spam that does leak through
    > still costs the spammer.


    I quite like the look of hash cash:

    http://www.cypherspace.org/adam/hashcash/

    (scroll down a little to see the meat of the page)

    Doesn't cost real money, just processor time, and hence makes large scale
    spamming unachievable, unless you have a supercomputer... I personally would
    welcome a wait of a second when sending an email if it would kill spam.
    The biggest problem is going to be widespread adoption of a new technology for
    email!
     
    Alex Hunsley, Jun 24, 2004
    #11
  12. Roedy Green

    Alex Hunsley Guest

    Re: OT: anti-spam measures

    LÄÊ»ie Techie wrote:

    > On Thu, 24 Jun 2004 00:56:06 +0000, Roedy Green wrote:
    >
    >
    >>I had a bit of a fright the other day. I thought for a while I was under
    >>a email denial of service attack. I wondered if I would ever be able to
    >>post even a munged public email address ever again.

    >
    >
    > My biggest complaint against current spam filters is the number of false
    > positives. Amongst the hundreds of messages marked as SPAM by my filter,
    > there are always 5 or 6 legit emails. Likewise 10 or so SPAM get through
    > my filter on a daily basis.


    I use k9 and I've never had a false positive (and I do check). I get false
    negatives - but this is about 5 emails out of 100 that have to be manually
    marked as spam. A quick scan of the senders/subject lines is enough to weed out
    the false negatives.

    alex
     
    Alex Hunsley, Jun 24, 2004
    #12
  13. Roedy Green

    Alex Hunsley Guest

    Re: OT: anti-spam measures

    Tim Ward wrote:

    > "La?ie Techie" <laie@win_remove_get_nospam_solutions.com> wrote in message
    > news:...
    >
    >>My biggest complaint against current spam filters is the number of false
    >>positives.

    >
    >
    > I am aware of 0 false positives on one email system I use.
    >
    > This is because I don't look at the spam folder, I just delete the entire
    > contents unexamined.
    >

    btw, follow on from my last msg: k9 is here...

    http://keir.net/k9.html
     
    Alex Hunsley, Jun 24, 2004
    #13
  14. Roedy Green

    Nigel Wade Guest

    On Thu, 24 Jun 2004 00:56:06 +0000, Roedy Green wrote:

    > I had a bit of a fright the other day. I thought for a while I was
    > under a email denial of service attack. I wondered if I would ever be
    > able to post even a munged public email address ever again.
    >
    > Surely there are such people.
    >
    > In like manner, I can see how spammers with political, religious,
    > pornographic, enfant terrible, or commercial interests will gradually
    > make the newsgroups and standard email totally unusable. As my Dad you
    > used say all the time, "watch the derivative".
    >
    > We can't wait until that happens before taking action.
    >
    > I see a multi-pronged approach will be necessary:
    >
    > 1. legal means. However but spammers will be able to hide anywhere on
    > earth. Surely some third world country will harbour them just as the
    > Cayman Islands harbours crooked companies. With the net, they can set up
    > shop in SomethingIstan and have effective storefronts in every country.


    Like you say, this won't work. Just look at the emaciated laws brought in
    the US and Europe. The commercial interests of the direct marketing
    companies have much greater lobbying power with the politicians than the
    inconvenience of their voters.


    > 2. boycotts. We must educate people to ensure spammers DON'T get
    > whatever it is they want from spamming.


    Definitely. I have blacklisted several commercial organizations who would
    not take any notice of the requests I sent to them to stop the UCE. They
    now get a rejection message from our mail server pointing out that they
    are spammers, and that we don't accept mail from them.


    > 3. technology. I see a new email delivery system evolving to completely
    > replace POP3/SMTP. It will have a number of features.


    This is the only mechanism which truly has any chance or working.


    > a. automatic encryption, compression, digital signing.


    I can't see automatic encryption working as many governments wouldn't
    allow encrypted email, and email has to be universal to be of any real
    use. Digital signing is a way forward, but to work reliably requires some
    form of escrow body to hold the public keys.

    > b. full use of the 8-bit channels.
    > c. a sender pays receiver system so any spam that does leak through
    > still costs the spammer.


    How would you get the sender to pay?

    > d. the best anti-spam thinking that is built in, suitable for
    > technopeasants.
    > e. suitable for exchanging large files, and common files. f. ways to
    > protect against denial of service attacks. g. designed from the ground
    > up for technopeasants. Everything is automatic.


    The thing we need to avoid is knee-jerk reactions and the resulting poorly
    thought out "solutions". All the current attempts to block spam that I
    know of also block some form of legitimate email. For example, they've
    just introduced a system here at work which is supposed to prevent
    incoming mail with the sender forged as an internal mail address. Whilst
    this may block some spam, it's easy to circumvent and the spam will evolve
    to do this (spammers are not all stupid). It also blocks legitimate mail
    from people who are working from home and want to email colleagues using
    their work email address to identify themselves. So, eventually, this
    measure will probably block only the legitimate external mail, while
    allowing in the spam.


    > The original email system was cooked up overnight as a demo. The author
    > surely never dreamed his system would be used almost unmodified for
    > planetary email scheme. It needs a major overhaul.


    That's unjustified. The current email system was developed with a
    great deal of thought over many years. It was developed as a means of
    communication between cooperating and consenting parties. The blame for
    spam lies entirely with commercialism.

    Don't shoot the messenger.


    The single most useful weapon currently is anti-virus measures. The
    largest source of spam is virus infected machines acting as spam
    relays. If users would clean and protect their systems a huge proportion
    of spam could be got rid of over night.

    --
    Nigel Wade, System Administrator, Space Plasma Physics Group,
    University of Leicester, Leicester, LE1 7RH, UK
    E-mail :
    Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555
     
    Nigel Wade, Jun 24, 2004
    #14
  15. Roedy Green

    flupke Guest

    Roedy Green wrote:
    > On Thu, 24 Jun 2004 05:56:27 +0200, Frédéric Gédin <>
    > wrote or quoted :
    >
    >
    >>SpamAssassin is free (www.spamasasin.org).
    >>This is an efficient antispam mail filter which is also used inside some
    >>commercial antispam solutions.

    >
    >
    > I think SpamAssassin and the like has to be incorporated into
    > MailServers to work, even if they are configured by the clients. One
    > problem I am having is giant spams that take many minutes to download.
    > Even if my client software filters them, they still tie up my Internet
    > connection.
    >
    > To deal with them we need to detect spam on the fly and derail it
    > before it even gets to the server.
    >
    > SpamAssasin is incorporated into the all Java James mailserver where I
    > work, but I still have not seen that big a drop off in spam. Perhaps
    > it takes a while to warm up. I have not yet been asked to configure
    > anything or mark anything.
    >


    I have installed Debian Gnu/Linux on an old pc and some programs to
    build an email server. Fetchmail downloads the mail from my mailbox,
    then passes it to exim which passes it on to spamassassin and clamav and
    after it's checked for virusses and spam it ends up in my emailbox.
    I then connect via a client and imap to the server to collect my mail.
    Almost no spam ends up in my email box. I get between 40-60 spam mails
    per day.

    On windows there are also free smtp server that you can use to
    accomplish the same.

    Benedict
     
    flupke, Jun 24, 2004
    #15
  16. Roedy Green

    Phillip Lord Guest

    >>>>> "Nigel" == Nigel Wade <> writes:


    >> 2. boycotts. We must educate people to ensure spammers DON'T get
    >> whatever it is they want from spamming.


    Nigel> Definitely. I have blacklisted several commercial
    Nigel> organizations who would not take any notice of the requests I
    Nigel> sent to them to stop the UCE. They now get a rejection
    Nigel> message from our mail server pointing out that they are
    Nigel> spammers, and that we don't accept mail from them.

    I hope this doesn't mean you sent bounce messages. About half of the
    spam I get are bounce messages telling me what a bad boy I am, for
    having my email address spoofed by a virus.


    >> The original email system was cooked up overnight as a demo. The
    >> author surely never dreamed his system would be used almost
    >> unmodified for planetary email scheme. It needs a major overhaul.


    Nigel> That's unjustified. The current email system was developed
    Nigel> with a great deal of thought over many years. It was
    Nigel> developed as a means of communication between cooperating and
    Nigel> consenting parties. The blame for spam lies entirely with
    Nigel> commercialism.


    No, its more legacy. The system could have been made improved since it
    was produced, but its too hard to get everyone to update their
    systems. Even if we invent new technology this will still cause a
    difficulty.

    Cheers

    Phil
     
    Phillip Lord, Jun 24, 2004
    #16
  17. Roedy Green

    Bryce Guest

    Re: OT: anti-spam measures

    On Thu, 24 Jun 2004 06:34:39 GMT, L??ie Techie
    <laie@win_remove_get_nospam_solutions.com> wrote:

    >My biggest complaint against current spam filters is the number of false
    >positives. Amongst the hundreds of messages marked as SPAM by my filter,
    >there are always 5 or 6 legit emails. Likewise 10 or so SPAM get through
    >my filter on a daily basis.


    I block, at the server, all email from China, and any that are on Open
    Relay lists. No false positives, very few false negatives...

    --
    now with more cowbell
     
    Bryce, Jun 24, 2004
    #17
  18. Roedy Green

    Roedy Green Guest

    On Thu, 24 Jun 2004 06:50:18 GMT, "Andy Fish"
    <> wrote or quoted :

    >IMHO the only sensible way to solve the spam problem long-term is to
    >introduce proper authentication for senders. Once you have that, you have a
    >foundation to build on.


    Another way it could be built is by issuing time-limited keys. So for
    example when you post in a newsgroup, you issue a public key to send
    you email good for 8 hours after the post, perhaps that has to answer
    a skill testing question usually relevant to the post, that the
    unskilled general public could NOT answer. At worst then, the spam
    harvester can only get through your initial defense for 8 hours.

    There might be various types of cert, in grades of just how
    permanently they identify you. It is up to you as recipient to set
    your bar depending on how beleaguered you are.

    The basic problem is I WANT mail from people I have never met. I just
    want it truly directed to me.

    It is just that once I have decided I want no more posts or emails
    from some twit I would like him to STAY out of my life. He should not
    be able to mutate to avoid my filter block. Current name-based
    filtering in newsgroups is useless because the spammer mutates with
    every message.

    With permanent, unforgeable ids, there can be reputation banks of
    spam/twit/crank history to also help you automatically decide whom to
    let through.

    The biggie though is to make spam COST. If it cost 50 cents to send a
    message, and you got 49 cents every time you received one, that would
    go a great distance both toward stopping spam, and taking the pain out
    of it. Most people would balance out or make money.

    When people pestered you with a question, they at least have to invest
    50 cents, whether you answer or not.

    The problem then comes from people who solicit mail then refuse to
    respond. Perhaps your ratio of send/receive should be public
    knowledge.


    If we don't solve this, we will end up with a parallel purely private
    mail system, where everyone has unlisted numbers, and a scheme to only
    let people through on your white list, verified with digital
    signatures.


    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Jun 24, 2004
    #18
  19. Roedy Green

    Roedy Green Guest

    On Thu, 24 Jun 2004 14:44:45 +0100, Nigel Wade <>
    wrote or quoted :

    >How would you get the sender to pay?


    Let's say IBM decided to tackle the problem. If you want to use the
    new mail scheme, you have to agree to their terms. It would be run on
    their servers initially, and eventually franchised out. Advertisers
    will still want to get at this lucrative market, just the way they use
    snail mail. Since you send fewer emails than you receive, most people
    will be ahead. They benefit from sender-pays. This is free money paid
    for by the advertisers. You are under no obligation to READ what comes
    in, no more that junk snail mail.

    Business at some point going to be against the wall. EMAIL will become
    unusable at some point. NOW WHAT? Business is totally dependent on
    it.

    The reason spam is such a problem is that it costs almost nothing to
    send. Even a tiny cost would shut down most of it. Amateur
    fruitcakes could not afford it.

    For public forums, voting schemes could financially reward those who
    post good material and penalise spammers and triflers.

    You could actually earn a living answering questions in public forums.


    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Jun 24, 2004
    #19
  20. Roedy Green

    Roedy Green Guest

    On Thu, 24 Jun 2004 00:56:06 GMT, Roedy Green
    <> wrote or quoted :

    >a. automatic encryption, compression, digital signing.


    One possible way this might be implemented is with anti-spam clubs.

    Members police each other, and kick out offenders. Members get a time
    limited digital certificate signed by the group. Then you can create
    bulk whitelists based on the group reputation.

    Members have to be picky about who they let in, or they destroy their
    own reputations.

    You can then be anonymous, uniquely identifiable, but still have a
    public reputation.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Jun 24, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark Rae
    Replies:
    0
    Views:
    630
    Mark Rae
    Mar 7, 2006
  2. Roedy Green

    Yet another anti spam approach

    Roedy Green, May 11, 2004, in forum: Java
    Replies:
    16
    Views:
    657
    Roedy Green
    May 13, 2004
  3. Alan Kennedy
    Replies:
    6
    Views:
    343
    Alan Kennedy
    Sep 1, 2005
  4. Tony Meyer
    Replies:
    2
    Views:
    318
    Mike Meyer
    Sep 2, 2005
  5. zax75
    Replies:
    1
    Views:
    1,109
Loading...

Share This Page