[SpamCop-List] Re: Open Proxy SCBL Rules

[John E. Malmberg]

In article <ctq0vj$3mj$1 at news.spamcop.net>, "K. Crocker" <nobody at spamcop.net> writes:
> John E. Malmberg wrote:
>
>> K. Crocker wrote:
>>
>>
>> There is no reason for spamcop.net to duplicate the function of the open
>> proxy lists.
>
> I suppose it depends on SpamCop's charter and how accurate the
> determination of "open proxy" is. My ISP hasn't revealed the algorithm
> it uses, except to say that they are using SCBL. Every additional list
> each ISP uses consumes that much more bandwidth, multiplied by each
> piece of email (spam and valid) flowing through the internet.

Not exactly true.  Several of the blocklists are relatively static, and
large mailserver operations routinely download local copies using tools
like rsync which only transfer the changes.

DHCP lists are an example of a DNSBL that is likely to only change on a daily
basis or even longer.

A local copy of a good DHCP blocking list will probably reject over 50%
of the spam delivery attempts with out additional bandwidth use.

The sbl.spamhaus.org list is also pretty stable for keeping a cached copy.
It will have less effect, as the smart spammers have figured out that it is
useless to send spam from any I.P. address listed in spamhaus.org.

The spews.org lists are also only distributed as files.  Some dnsbl operators
will provide access to them through their servers.

And it is quite likely that your mail server operator can use local blocking
lists for the spam that gets through all their checks.

I know of one postmaster that for certain countries, seems to locally block at
least the /22 surrounding the I.P. address of any spam that gets through
on them. In several years, that technique has not resulted in any reported real
e-mail being rejected.

The spamcop.net blocking list is not suitable for being the main blocking
list of a mail server for several reasons.

1. It tries to identify the injection point of the spam, and your mail
   server can usually only use it against the last hop.

2. Long term spam sources may drop off the spamcop.net blocking list
   because the source I.P. is already on a more conservative list.

3. A more conservative list may have determined that a whole netblock
   is controlled by spammers and blocked the whole thing, while the
   spammer jumping around in it evades the spamcop.net algorithm.

4. The spamcop.net algorithm is aggressive and will list real mail servers,
   and many times this is from spam reporters not noticing that a parser
   error is reporting their own mail server.

> Logistically, it could be argued that the perfect block list should add
> blackhat addresses ASAP and keep them there ALAP, commensurate with a
> totally automatic system.

The blocking lists are specialized because of how they determine a listing.

And there are several of them that are aggregated to simplify lookups.

opm.blitzed.org only tests for open proxies known to be used to abuse IRC
networks.  The mail server protection is a side effect.

cbl.abuseat.org has spamtraps that are content filtered to remove "bounce"
backscatter so it tends to only list spam sources and viruses.  The
cbl.abuseat.org is very good at catching sources of direct to MX viruses or
spammers that use harvested e-mail addresses.

By querying the xbl.spamhaus.org, you get a lookup of the opm.blitzed.org and
the cbl.abuseat.org at the same time.

By querying the sbl-xbl.spamhaus.org, you get a lookup of the sbl.spamhaus.org,
and the xbl.spamhaus.org.

Combine the sbl-xbl.spamhaus.org with a good dhcp blocking list, and you will
find that will catch lot of the spam, and as pointed out above, most
of the data can be locally cached efficiently.

The list.dsbl.org only lists I.P. addresses that have sent it a specially
formatted listme message.

That message is sent by special software by trusted volunteers that knows how
to scan for many security vulnerabilities.

The njabl.org runs proxy tests.

There seems to be a high overlap in what njabl.org and dsbl.org list.

> I think you missed my point. I understand what you are saying. I've
> done both parsing and checking to see if an IP address was on the SCBL
> on numerous occasions. My intent was to foster a discussion, perhaps
> observed by a deputy, to get open proxy addresses added ASAP to the
> SCBL, rather than waiting for corroborative evidence.

That would place control of a spamcop.net listing under the control of
an entity that has no affiliation with spamcop.net.

Spamcop.net keeps evidence of spam being sent.  The evidence used by the
open proxy listing service may not be available for a deputy to determine
why the open proxy service is listing it.

By the time you submit the spam from a POP3 account, it may already be on
the spamcop.net blocking list, or your report may be the one that puts it
over the edge.

>>> My POP3 service uses the SCBL, so any spam I receive is usually from
>>> sources not on the SCBL. A large proportion of that spam appears to be
>>> coming from open proxies, hence the interest. Thanks for your comments!
>>
>>
>> It is probably is a case that your mail server operators are using an
>> open proxy list, yet at the time your mail server operator accepted the
>> e-mail, that I.P. address was not yet on either the open proxy lists
>> that they use, or on the spamcop.net list either.
>
> I would guess that my ISP is *not* using an open proxy list, or, at
> least, not the one SC uses. I've parsed spam literally seconds old that
> shows up open proxy, yet was admitted through my ISP.

There are several services that will check lots of blocking lists to see
where an I.P. address is listed.

By taking the I.P. address that your mail server accepted the spam from
and putting it in those lists, you can determine which ones your mail server
operator is likely using or not using.

It would be a big surprise for a mail server operator to use the spamcop.net
blocking list with out using the other ones, especially the open proxy
lists or the spamhaus.org lists.


The biggest argument that I have heard against using an open proxy list
is that there is a high concern it will block real e-mails.

This is from mail server operators that use open relay lists as their
primary anti-spam defense.

Their lack of understanding of why their logic is faulty is amazing, and
it is always amazing that they can not be convinced of their error.
Such mis-understandings usually translate too - "I barely got this mail
server thing working, and if I change anything, it will probably break,
and my boss will discover I have no clue of what I am doing".

The simple issues are:

An open relay is usually a real mail server that is misconfigured, so blocking
open relays is probably going to have a measurable chance of causing a real
e-mail to be blocked.

An open proxy is usually a computer that is not intentionally a mail server,
so blocking an open proxy has a much lower chance of blocking a real e-mail,
than the open relay lists that the mail server operator is already using.

Now is there any way to make it clearer that anyone using an open relay
list, but not using an open proxy list, clearly does not have a good
technical understanding of what they are doing?

>> Statistics from one of my mail server operators show that the
>> spamcop.net blocking list is only catching 3% of the spam.  The majority
>>  of spam is removed by more conservative blocking lists.
>
> I think you meant liberal. SCBL would be considered conservative, since
> one of it's aims is to block as little valid email as possible. Pardon
> the nit picking...

No, I mean conservative.  The other blocking lists try not to list production
mail servers unless there is either a documented security problem with them
or that the mail server operator has through action or inaction allowed the
mail server to be freely used by spammers.

Spamcop.net will list real mail servers and has a much higher chance of
causing collateral damage than the conservative lists.

Using the spamcop.net list to reject e-mail will only block a small
percentage more of the spam sources that the conservative lists will block,
but is more likley to reject a real e-mail.

The spamcop.net blocking list is more useful on a scoring system where
additional tests can usually confirm that an item is spam, where in many
cases, many of those tests by them selves could cause false positives.

>> Other statistics that I am seeing indicate that the bulk of the spam is
>> coming from dynamic pools, which many mail server operators block.
>
> <dynamic pool discussion> Thanks for the info!
>
>> But do not submit I.P. addresses for listing in a dynamic pool unless
>> you have strong evidence that the I.P. address is dynamic, as the
>> processing of them is completely manual.
>
> Ah, if I had the kung fu (time + effort) to do this! I once kept track
> of some of the IP addresses used by one spammer as they sent one
> particular email campaign. I recorded well over 100 different addresses
> before I got tired, many from vastly differing blocks, none reused.
> This has nothing to do with the open proxy issue, but just to say that
> spammers have the "whack-a-mole" game down pat.

If you are getting that volume of spam, it indicates that there is a hole
in your ISP's spam defences.

I kept track for almost a year of spam from DHCP pools which basically
proved that a comercial DHCP pool listing service was missing many very
large and very well known DHCP pools.

That mail server operator switched to using the SORB dhcp pool list, and
that made a significant reduction in the spam leakage.

One of the results of the tests showed that the spammers were apparently
assuming that the dhcp address block that they spammed from was probably
blocked for about two months, and then they would recycle it.

Of course that could be the time that I.P. address was sitting in one
of the open proxy lists that age out their listings.

> If the open proxy determination was simple and bullet proof, I don't
> see a reason why it shouldn't be used to prevent known chronic repeat
> offenders from moving back into my neighborhood, to borrow from a
> different analogy.

Too many mail server operators or ISP operators do not have a clue of what
they are doing.

Too many of them are trying to do spam filtering by content analysis
instead of source I.P blocking, because that is what most of the commercial
spam filtering companies offer.

Too many clueless media reporters mis-report the spam issue.  Most media
reports I have read present following statements as fact, with no data
at all to back them up.

    1. DNSbls are evil and will regularly cause real e-mail to be lost.
    2. Content filtering from their (potential) advertisers is state of the art.
    3. Spammers make big money from people buying spamvertized items.
       (It seems that the big money is selling spamming kits, not spamming,
        most of the actuall spammers seem to never make back even a fraction
	of what they spent to get started - It's is just a pyramid scam)

They also omit the following information:

    1. That blocking of e-mail and other packets has shown to be
       the only way to motivate a large number of network operators to
       do anything at all about abuse coming from their systems.

    2. That once those blocks become noticed by a critical paying
       customer, the ISP allowing spam to be sent seems to be able
       clean up the problem almost instaneously, even though up to
       that point they were making excuses about how hard the job is,
       and how much time it will take.

    3. Never ask one of the blocked ISP's why they are providing
       services for a web site advertising illegal items?

    4. Never ask one of the blocked ISP's why they are keeping a customer
       that can be verified to be attempting to spam through open proxies?

    5. Omit disclosing that they hope to sell advertising to the network
       operators that permit spam to be sent.

    6. Ignore all tests that show that the DNSbls are more accurate both
       at detecting spam and real e-mail than any of the commercial
       content filters.

    7. Never point out that large mail server operators pay a metered
       rate for their connection, so that to use content filtering
       greatly increaes their cost.

Too many ISP users do not realize what the state of the art is in spam
blocking, so they do not realize that all their ISP is offering is a placebo
for spam filtering so that they can claim that they care, while just
passing on the extra charges for doing an incompetent job.

If an ISP wanted to really hurt the spammers, they would use the
sbl.spamhaus.org list at their border routers to block access to the spammer's
web pages.  (or spews, if they really wanted to be a BOFH).

This would make it obvious to the most of the spammers that no one at that
ISP could even visit their web site to order the product should their spew
get through their filters.

Also too many people are not bugging their elective officials to hold ISP's
corporate officers criminally responsible for not taking action against
customers that are still using their services after one business day that
the ISP should have received a notification.  And make sure that the law
indicates that the ISP is still liable if their abuse or postmaster e-mail
address rejected or deleted the notification, or if they where a day behind
in processing abuse/postmaster issues.

-John
wb8tyw at qsl.network
Personal Opinion Only