[SC-Help] Re: Parser missed URLs in spam..

[Mike Easter]

Duncan Hill wrote:
>www.spamcop.net/sc?id=z760605444zbd6d8a5bfd704c6bbd6a3ac143b3a035z
>
> The parser misses the URL that's about 30 lines down in the body.

Finds 2 urls but can't resolve them [and spends a very long time trying
to do so].
Cannot resolve http://middleweight.klipsandbracks.com/tx
Cannot resolve http://makeup.klipsandbracks.com/index.html

It resolved promptly for me and didn't appear to have gross nameservice
problems at dnsstuff, so maybe its nameservice is blocking SC's
resolver.

> www.spamcop.net/sc?id=z760605443z7ec08a7b1611eae20442b7d8b1aa9b3fz
>
> Parser misses the URL that's in the plain text body, and alive and
> kicking (and just happens to be a redirector), and also in the HTML
> part.

Looks at both parts:

Finding links in message body
Recurse multipart:
   Parsing text part
   Parsing HTML part

Resolving link obfuscation
http://midland.myabsinth.com/nothanks.php
http://all.myabsinth.com/575r.html
http://contrariety.myabsinth.com/nothanks.php
http://bide.myabsinth.com/575r.html

Those are from both html and plaintext sections, I forget which is
which.  Sometimes it will resolve them, sometimes it won't.  And when it
doesn't, it doesn't spend any time at all messing with trying to resolve
them.  It just 'passes' on the resolving part without delay.

IMO the parser is 'rigged' to sometimes 'bypass' body url resolving.  "I
don't feel like doing that right now;  I'm doing some other stuff."

If I feed one of those into the parser naked, it resolves it OK

Parsing input: http://midland.myabsinth.com/nothanks.php
Routing details for 202.99.172.149
Using postmaster#cnc-noc.net at devnull.spamcop.net for statistical
tracking.

www.spamcop.net/sc?id=z760594773z37451b0b3b5559c426a6365e5f3c02e1z
>
> has the same URL tld as the first missed report.

Like the first, SC spends a great deal of time not resolving any of the
links it finds:

Resolving link obfuscation
http://cliffhang.klipsandbracks.com/index.html
   host cliffhang.klipsandbracks.com (checking ip) ip not found ;
cliffhang.klipsandbracks.com discarded as fake.
http://continent.klipsandbracks.com/index.html
   host continent.klipsandbracks.com (checking ip) ip not found ;
continent.klipsandbracks.com discarded as fake.
http://shudder.klipsandbracks.com/index.html
   host shudder.klipsandbracks.com (checking ip) ip not found ;
shudder.klipsandbracks.com discarded as fake.
http://indeed.klipsandbracks.com/tx
   host indeed.klipsandbracks.com (checking ip) ip not found ;
indeed.klipsandbracks.com discarded as fake.
http://dugout.klipsandbracks.com/tx
   host dugout.klipsandbracks.com (checking ip) ip not found ;
dugout.klipsandbracks.com discarded as fake.

> Mails are fowarded as attachment from KMail, and the parser has
> picked up other URLs before with no problem.

It isn't related to kmail;  those are common behaviors and situations
with spamvertiser nonresolving problems.


-- 
Mike Easter
kibitzer, not SC admin