Certain Spamassassin rules do not seem to be firing all of the time

[Duncan, Brian M.]

spamassassin-3.3.1-3.el5.rf
mailscanner-4.83.5-1

Looking for some help here, it looks like sometimes Mailscanner is causing SpamAssassin to not use some rules. (Not exactly sure on this I assume it is Mailscanner based on the behavior I am seeing)

I receive the message and it is not tagged as Spam and winds up in my inbox. The headers show on this example:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,RP_MATCHES_RCVD -0.00)

I then take that message and drag it into a separate mailbox I had setup on our Exchange server, then pull it down to my Sendmail/Mailscanner/SpamAssassin box through imap in rfc822 format.

I then run the same message through Spamassassin with -test-mode locally from my mail server I get different scoring on, it looks like I am missing some of the checks because now it defiantly shows as Spam:

Content analysis details: (17.3 hits, 6.5 required)
5.0 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist
[URIs: eelefs.net]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.5050]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
0.0 LOTS_OF_MONEY Huge... sums of money

------ End of SpamAssassin results, Original message follows --------

So I was wondering if it had to with my MailScanner.conf having this line: SpamAssassin Local State Dir = # /var/lib/spamassassin

But based on my debug of MailScanner, it does not matter if the # is present or not, MailScanner seems to think it knows where all the rules are. The below output is with SpamAssassin Local State Dir = /var/lib/spamassassin

In Debugging mode, not forking...
Trying to setlogsock(unix)
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: adding facilities: all
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: logging level is DBG
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: SpamAssassin version 3.3.1
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: Perl 5.008008, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/mail/spamassassin, LOCAL_STATE_DIR=/var/lib/spam
assassin
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: timing enabled
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: score set 0 chosen.
15:54:01 Jun 12 15:54:01.477 [32352] dbg: util: running in taint mode? no
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: is Net::DNS::Resolver available? yes
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: Net::DNS version: 0.65
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/init.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v310.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v312.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v320.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v330.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/var/lib/spamassassin/3.003001" for sys rules pre files
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/var/lib/spamassassin/3.003001" for default rules dir
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org.cf
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: using "/etc/mail/spamassassin" for site rules dir
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum1.cf
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/70_sare_unsub.cf
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/chickenpox.cf
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/local.cf
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/mailscanner.cf
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: using "/root/.spamassassin/user_prefs" for user prefs file
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: read file /root/.spamassassin/user_prefs
15:54:01 Jun 12 15:54:01.484 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
15:54:01 Jun 12 15:54:01.488 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
15:54:01 Jun 12 15:54:01.491 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
15:54:01 Jun 12 15:54:01.494 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
15:54:01 Jun 12 15:54:01.496 [32352] dbg: pyzor: network tests on, attempting Pyzor

The odd thing here to me, is if I search my maillog for some of the hits from above, Like URIBL_DBL_SPAM, I am seeing many hits on this.. It just seems to be skipping some of the rules for a certain messages. I looked through

Anyone have any ideas where I can start to figure this one out? I checked my rules, but since some of the rules are firing I assumed it can't have anything to do with that..

Here is the complete output from the message I give as an example from above: (minus the spammy body)

Received: from CHI-US-HT-01.us.kmz.com (10.18.17.28) by
CHI-US-CAS-1B.us.kmz.com (10.125.15.2) with Microsoft SMTP Server (TLS) id
14.3.123.3; Wed, 12 Jun 2013 15:44:04 -0500
Received: from chi-us-vwall-01.us.kmz.com (10.18.16.181) by
CHI-US-HT-01.us.kmz.com (10.18.17.28) with Microsoft SMTP Server id
14.3.123.3; Wed, 12 Jun 2013 15:44:03 -0500
Received: from venus.kattenlaw.com ([10.18.3.33]) by us.kmz.com
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id 844d8c4f001d4ac4
; Wed, 12 Jun 2013 15:44:01 -0500
Received: from smtp1.eelefs.net (smtp1.eelefs.net [66.197.143.105]) by
venus.kattenlaw.com (8.13.8/8.13.4) with ESMTP id r5CKi0H8028960 for
<brian.duncan at kmzr.com>; Wed, 12 Jun 2013 15:44:03 -0500
From: 2013 Models <Jorge.Mendoza at eelefs.net>
To: "Duncan, Brian M." <brian.duncan at kattenlaw.com>
Subject: *Reduction Information* 2013's for thousands less
Thread-Topic: *Reduction Information* 2013's for thousands less
Thread-Index: AQHOZ62T+0z+e2LgwkiBidggfWeC0A==
Date: Wed, 12 Jun 2013 15:43:58 -0500
Message-ID: <29295056e3e7741908e644022e5f0220 at smtp1.eelefs.net>
Reply-To: "Jorge.Mendoza at eelefs.net" <Jorge.Mendoza at eelefs.net>
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-AuthSource: CHI-US-HT-01.us.kmz.com
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailscanner-from: jorgemendoza at smtp1.eelefs.net
x-mailscanner-spamcheck: not spam, SpamAssassin (cached, score=0.8, required
6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00, RP_MATCHES_RCVD -0.00)
x-kattenlaw-mailscanner-information:
x-mailscanner-spam: no
x-kattenlaw-mailscanner-id: r5CKi0H8028960
x-tm-imss-message-id: <844d8c4f001d4ac4 at us.kmz.com>
x-kattenlaw: NS
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8737EB66163E6F4DA060748F2D862AD0 at kattenlaw.com>
MIME-Version: 1.0

Thanks for any help.


===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
CONFIDENTIALITY NOTICE:
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130612/690ca823/attachment.html

[Martin Hepworth]

Are you running the tests against the same user MailScanner runs as to make
sure any .spamassassin directory settings arent overriding

in both headers you're getting spamassassin cache hits which is a
mailscanner option. You might want to stop MailScanner, delete the
spamassassin cache file ans retry. Could be the cache file has got corrupt
somehow.

martin
--
Martin Hepworth, CISSP
Oxford, UK

spamassassin-3.3.1-3.el5.rf****
mailscanner-4.83.5-1****
** **
Looking for some help here, it looks like sometimes Mailscanner is causing
SpamAssassin to not use some rules. (Not exactly sure on this I assume it
is Mailscanner based on the behavior I am seeing)****
** **
I receive the message and it is not tagged as Spam and winds up in my
inbox. The headers show on this example:****
** **
X-MailScanner-SpamCheck: not spam, SpamAssassin (cached,
score=0.8,required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,RP_MATCHES_RCVD
-0.00)****
** **
I then take that message and drag it into a separate mailbox I had setup
on our Exchange server, then pull it down to my
Sendmail/Mailscanner/SpamAssassin box through imap in rfc822 format.****
** **
I then run the same message through Spamassassin with ?test-mode locally
from my mail server I get different scoring on, it looks like I am missing
some of the checks because now it defiantly shows as Spam:****
** **
Content analysis details: (17.3 hits, 6.5 required)****
5.0 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist***
*
[URIs: eelefs.net]****
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain****
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%****
[score: 0.5050]****
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level****
above 50%****
[cf: 100]****
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)****
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%****
[cf: 100]****
0.0 LOTS_OF_MONEY Huge... sums of money****
** **
------ End of SpamAssassin results, Original message follows --------****
** **
SpamAssassin Local State Dir = # /var/lib/spamassassin****
** **
But based on my debug of MailScanner, it does not matter if the # is
present or not, MailScanner seems to think it knows where all the rules
are. The below output is with SpamAssassin Local State Dir =
/var/lib/spamassassin****
** **
In Debugging mode, not forking...****
Trying to setlogsock(unix)****
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: adding facilities: all**
**
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: logging level is DBG****
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: SpamAssassin version
3.3.1****
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: Perl 5.008008,
PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin,
LOCAL_RULES_DIR=/etc/mail/spamassassin, LOCAL_STATE_DIR=/var/lib/spam****
assassin****
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: timing enabled****
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: score set 0 chosen.****
15:54:01 Jun 12 15:54:01.477 [32352] dbg: util: running in taint mode? no*
***
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: is Net::DNS::Resolver
available? yes****
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: Net::DNS version: 0.65****
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using
"/etc/mail/spamassassin" for site rules pre files****
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file
/etc/mail/spamassassin/init.pre****
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file
/etc/mail/spamassassin/v310.pre****
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file
/etc/mail/spamassassin/v312.pre****
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file
/etc/mail/spamassassin/v320.pre****
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file
/etc/mail/spamassassin/v330.pre****
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using
"/var/lib/spamassassin/3.003001" for sys rules pre files****
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using
"/var/lib/spamassassin/3.003001" for default rules dir****
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file
/var/lib/spamassassin/3.003001/updates_spamassassin_org.cf****
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: using
"/etc/mail/spamassassin" for site rules dir****
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file
/etc/mail/spamassassin/70_sare_evilnum1.cf****
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file
/etc/mail/spamassassin/70_sare_unsub.cf****
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file
/etc/mail/spamassassin/chickenpox.cf****
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file
/etc/mail/spamassassin/local.cf****
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file
/etc/mail/spamassassin/mailscanner.cf****
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: using
"/root/.spamassassin/user_prefs" for user prefs file****
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: read file
/root/.spamassassin/user_prefs****
15:54:01 Jun 12 15:54:01.484 [32352] dbg: plugin: loading
15:54:01 Jun 12 15:54:01.488 [32352] dbg: plugin: loading
15:54:01 Jun 12 15:54:01.491 [32352] dbg: plugin: loading
15:54:01 Jun 12 15:54:01.494 [32352] dbg: plugin: loading
15:54:01 Jun 12 15:54:01.496 [32352] dbg: pyzor: network tests on,
attempting Pyzor****
** **
The odd thing here to me, is if I search my maillog for some of the hits
from above, Like URIBL_DBL_SPAM, I am seeing many hits on this.. It just
seems to be skipping some of the rules for a certain messages. I looked
through****
** **
Anyone have any ideas where I can start to figure this one out? I checked
my rules, but since some of the rules are firing I assumed it can?t have
anything to do with that..****
** **
Here is the complete output from the message I give as an example from
above: (minus the spammy body)****
** **
Received: from CHI-US-HT-01.us.kmz.com (10.18.17.28) by****
CHI-US-CAS-1B.us.kmz.com (10.125.15.2) with Microsoft SMTP Server (TLS)
id****
14.3.123.3; Wed, 12 Jun 2013 15:44:04 -0500****
Received: from chi-us-vwall-01.us.kmz.com (10.18.16.181) by****
CHI-US-HT-01.us.kmz.com (10.18.17.28) with Microsoft SMTP Server id****
14.3.123.3; Wed, 12 Jun 2013 15:44:03 -0500****
Received: from venus.kattenlaw.com ([10.18.3.33]) by us.kmz.com****
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id
844d8c4f001d4ac4****
; Wed, 12 Jun 2013 15:44:01 -0500****
Received: from smtp1.eelefs.net (smtp1.eelefs.net [66.197.143.105]) by
****
venus.kattenlaw.com (8.13.8/8.13.4) with ESMTP id r5CKi0H8028960
for****
<brian.duncan at kmzr.com>; Wed, 12 Jun 2013 15:44:03 -0500****
From: 2013 Models <Jorge.Mendoza at eelefs.net>****
To: "Duncan, Brian M." <brian.duncan at kattenlaw.com>****
Subject: *Reduction Information* 2013's for thousands less****
Thread-Topic: *Reduction Information* 2013's for thousands less****
Thread-Index: AQHOZ62T+0z+e2LgwkiBidggfWeC0A==****
Date: Wed, 12 Jun 2013 15:43:58 -0500****
Message-ID: <29295056e3e7741908e644022e5f0220 at smtp1.eelefs.net>****
Reply-To: "Jorge.Mendoza at eelefs.net" <Jorge.Mendoza at eelefs.net>****
Content-Language: en-US****
X-MS-Exchange-Organization-AuthAs: Anonymous****
X-MS-Exchange-Organization-AuthSource: CHI-US-HT-01.us.kmz.com****
X-MS-Has-Attach:****
X-MS-TNEF-Correlator:****
x-mailscanner-from: jorgemendoza at smtp1.eelefs.net****
x-mailscanner-spamcheck: not spam, SpamAssassin (cached, score=0.8,
required****
6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00, RP_MATCHES_RCVD -0.00)****
x-kattenlaw-mailscanner-information:****
x-mailscanner-spam: no****
x-kattenlaw-mailscanner-id: r5CKi0H8028960****
x-tm-imss-message-id: <844d8c4f001d4ac4 at us.kmz.com>****
x-kattenlaw: NS****
Content-Type: text/plain; charset="us-ascii"****
Content-ID: <8737EB66163E6F4DA060748F2D862AD0 at kattenlaw.com>****
MIME-Version: 1.0****
** **
Thanks for any help.****
** **
===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130613/803b5982/attachment.html

[Duncan, Brian M.]

Thanks for the suggestions Martin.

I don't have any specific user that I run as:

Run As User =

So I assume it is running as root? My tests with -test-mode were run as root.. I do have the .spamassassin dir in root that has bayes db's that are the ones that get updated, and I did confirm there was nothing there causing problems.

I took your advice and started by stopping Mailscanner and deleting the cache and any orphaned files in the directories, hopefully that will have a positive impact.

I assume it must be something odd like that, these messages started coming through last week. I have to believe if all my rules were not firing since I built that box a year or so ago I would have noticed this sooner.

One thing I noticed after taking other messages that failed due to body checks that actually wind up tagged as Spam, most seem to have more rules that fire off when I run them locally as root with -test-mode then what they have in my mail client after they have come through.

I do see hits on messages for rules that ONLY exist in some of the rules in the /var/lib/spamassassin/3.003001/updates_spamassassin_org directory. So I know it is accessing those files, just not all of them for some reason at certain times..

I just took a message that made it through today for me:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,
required 6.5, BAYES_50 0.80, DIET_1 0.00, RP_MATCHES_RCVD -0.00)

When I check this message on my MailScanner box with Spamassassin as root I get:

Content analysis details: (30.1 hits, 6.5 required)
6.5 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
[URIs: nthjus.com]
0.0 DIET_1 BODY: Lose Weight Spam
1.3 RCVD_IN_RP_RNBL RBL: Relay in RNBL,
https://senderscore.org/blacklistlookup/
[64.191.19.228 listed in bl.score.senderscore.com]
10 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: nthjus.com]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.5001]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]

It seems to be all the rules that don't fire are the ones where it would actually be looking something up, right? Through DNS?


BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com


From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Thursday, June 13, 2013 1:51 AM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

Are you running the tests against the same user MailScanner runs as to make sure any .spamassassin directory settings arent overriding
in both headers you're getting spamassassin cache hits which is a mailscanner option. You might want to stop MailScanner, delete the spamassassin cache file ans retry. Could be the cache file has got corrupt somehow.

martin

--
Martin Hepworth, CISSP
Oxford, UK

On 12 June 2013 22:05, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
spamassassin-3.3.1-3.el5.rf
mailscanner-4.83.5-1

Looking for some help here, it looks like sometimes Mailscanner is causing SpamAssassin to not use some rules. (Not exactly sure on this I assume it is Mailscanner based on the behavior I am seeing)

I receive the message and it is not tagged as Spam and winds up in my inbox. The headers show on this example:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,RP_MATCHES_RCVD -0.00)

I then take that message and drag it into a separate mailbox I had setup on our Exchange server, then pull it down to my Sendmail/Mailscanner/SpamAssassin box through imap in rfc822 format.

I then run the same message through Spamassassin with -test-mode locally from my mail server I get different scoring on, it looks like I am missing some of the checks because now it defiantly shows as Spam:

Content analysis details: (17.3 hits, 6.5 required)
5.0 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist
[URIs: eelefs.net<http://eelefs.net>]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.5050]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
0.0 LOTS_OF_MONEY Huge... sums of money

------ End of SpamAssassin results, Original message follows --------

So I was wondering if it had to with my MailScanner.conf having this line: SpamAssassin Local State Dir = # /var/lib/spamassassin

But based on my debug of MailScanner, it does not matter if the # is present or not, MailScanner seems to think it knows where all the rules are. The below output is with SpamAssassin Local State Dir = /var/lib/spamassassin

In Debugging mode, not forking...
Trying to setlogsock(unix)
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: adding facilities: all
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: logging level is DBG
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: SpamAssassin version 3.3.1
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: Perl 5.008008, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/mail/spamassassin, LOCAL_STATE_DIR=/var/lib/spam
assassin
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: timing enabled
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: score set 0 chosen.
15:54:01 Jun 12 15:54:01.477 [32352] dbg: util: running in taint mode? no
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: is Net::DNS::Resolver available? yes
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: Net::DNS version: 0.65
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/init.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v310.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v312.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v320.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v330.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/var/lib/spamassassin/3.003001" for sys rules pre files
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/var/lib/spamassassin/3.003001" for default rules dir
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org.cf<http://updates_spamassassin_org.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: using "/etc/mail/spamassassin" for site rules dir
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum1.cf<http://70_sare_evilnum1.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/70_sare_unsub.cf<http://70_sare_unsub.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/chickenpox.cf<http://chickenpox.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/local.cf<http://local.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/mailscanner.cf<http://mailscanner.cf>
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: using "/root/.spamassassin/user_prefs" for user prefs file
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: read file /root/.spamassassin/user_prefs
15:54:01 Jun 12 15:54:01.484 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
15:54:01 Jun 12 15:54:01.488 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
15:54:01 Jun 12 15:54:01.491 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
15:54:01 Jun 12 15:54:01.494 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
15:54:01 Jun 12 15:54:01.496 [32352] dbg: pyzor: network tests on, attempting Pyzor

The odd thing here to me, is if I search my maillog for some of the hits from above, Like URIBL_DBL_SPAM, I am seeing many hits on this.. It just seems to be skipping some of the rules for a certain messages. I looked through

Anyone have any ideas where I can start to figure this one out? I checked my rules, but since some of the rules are firing I assumed it can't have anything to do with that..

Here is the complete output from the message I give as an example from above: (minus the spammy body)

Received: from CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com> (10.18.17.28) by
CHI-US-CAS-1B.us.kmz.com<http://CHI-US-CAS-1B.us.kmz.com> (10.125.15.2) with Microsoft SMTP Server (TLS) id
14.3.123.3; Wed, 12 Jun 2013 15:44:04 -0500
Received: from chi-us-vwall-01.us.kmz.com<http://chi-us-vwall-01.us.kmz.com> (10.18.16.181) by
CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com> (10.18.17.28) with Microsoft SMTP Server id
14.3.123.3; Wed, 12 Jun 2013 15:44:03 -0500
Received: from venus.kattenlaw.com<http://venus.kattenlaw.com> ([10.18.3.33]) by us.kmz.com<http://us.kmz.com>
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id 844d8c4f001d4ac4
; Wed, 12 Jun 2013 15:44:01 -0500
Received: from smtp1.eelefs.net<http://smtp1.eelefs.net> (smtp1.eelefs.net<http://smtp1.eelefs.net> [66.197.143.105]) by
venus.kattenlaw.com<http://venus.kattenlaw.com> (8.13.8/8.13.4) with ESMTP id r5CKi0H8028960 for
<brian.duncan at kmzr.com<mailto:brian.duncan at kmzr.com>>; Wed, 12 Jun 2013 15:44:03 -0500
From: 2013 Models <Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>>
To: "Duncan, Brian M." <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>>
Subject: *Reduction Information* 2013's for thousands less
Thread-Topic: *Reduction Information* 2013's for thousands less
Thread-Index: AQHOZ62T+0z+e2LgwkiBidggfWeC0A==
Date: Wed, 12 Jun 2013 15:43:58 -0500
Message-ID: <29295056e3e7741908e644022e5f0220 at smtp1.eelefs.net<mailto:29295056e3e7741908e644022e5f0220 at smtp1.eelefs.net>>
Reply-To: "Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>" <Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>>
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-AuthSource: CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailscanner-from: jorgemendoza at smtp1.eelefs.net<mailto:jorgemendoza at smtp1.eelefs.net>
x-mailscanner-spamcheck: not spam, SpamAssassin (cached, score=0.8, required
6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00, RP_MATCHES_RCVD -0.00)
x-kattenlaw-mailscanner-information:
x-mailscanner-spam: no
x-kattenlaw-mailscanner-id: r5CKi0H8028960
x-tm-imss-message-id: <844d8c4f001d4ac4 at us.kmz.com<mailto:844d8c4f001d4ac4 at us.kmz.com>>
x-kattenlaw: NS
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8737EB66163E6F4DA060748F2D862AD0 at kattenlaw.com<mailto:8737EB66163E6F4DA060748F2D862AD0 at kattenlaw.com>>
MIME-Version: 1.0

Thanks for any help.


===========================================================

CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue

Service, any tax advice contained herein is not intended or written to be used and cannot be used

by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.

===========================================================

CONFIDENTIALITY NOTICE:

This electronic mail message and any attached files contain information intended for the exclusive

use of the individual or entity to whom it is addressed and may contain information that is

proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you

are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or

distribution of this information may be subject to legal restriction or sanction. Please notify

the sender, by electronic mail or telephone, of any unintended recipients and delete the original

message without making any copies.

===========================================================

NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has

elected to be governed by the Illinois Uniform Partnership Act (1997).

===========================================================


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130614/81a13109/attachment.html

[Duncan, Brian M.]

Looks like deleting the spamassassin cache made no difference.

This morning I received another spam that made it through.

This is what it scored when passed through Mailscanner/Spamassassin:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,
required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,
RP_MATCHES_RCVD -0.00)

I moved it over to my mailscanner/spamassassin box within 30 seconds of receiving it and this is what it scored on my Mailscanner box from the command line doing spamassassin -test-mode < message.txt:

Content analysis details: (14.6 hits, 6.5 required)
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
3.0 BAYES_60 BODY: Bayes spam probability is 60 to 80%
[score: 0.6460]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
0.0 LOTS_OF_MONEY Huge... sums of money
0.1 FROM_12LTRDOM From a 12-letter domain

------ End of SpamAssassin results, Original message follows --------

The really odd thing, is if I take the body and subject from the spam message above and send it through a hotmail account I have (which I white list, which is why that shows in the below results), this is what it scores when passed through Mailscanner/Spamassassin:

X-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached,
score=20.146, required 6.5, autolearn=spam, AWL -13.90,
BAYES_50 0.80, FREEMAIL_ENVFROM_END_DIGIT 0.25, FREEMAIL_FROM 0.00,
HTML_MESSAGE 0.00, LOTS_OF_MONEY 0.00, RAZOR2_CF_RANGE_51_100 0.50,
RAZOR2_CF_RANGE_E8_51_100 2.50, RAZOR2_CHECK 8.50,
RCVD_IN_DNSWL_NONE -0.00, RP_MATCHES_RCVD -0.00, SPF_PASS -0.00,
URIBL_BLACK 10.00, URIBL_DBL_SPAM 5.00, URIBL_JP_SURBL 6.50)

This makes no sense to me, it's almost like this specific Spammer has figured out a way to get Mailscanner to stop scanning portions of its message.

I am going to turn off caching of spamassassin results next in my mailscanner conf to see if that has any impact.

If anyone has any ideas please let me know.

Brian


BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com


From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Duncan, Brian M.
Sent: Thursday, June 13, 2013 7:35 PM
To: MailScanner discussion
Subject: RE: Certain Spamassassin rules do not seem to be firing all of the time

Thanks for the suggestions Martin.

I don't have any specific user that I run as:

Run As User =

So I assume it is running as root? My tests with -test-mode were run as root.. I do have the .spamassassin dir in root that has bayes db's that are the ones that get updated, and I did confirm there was nothing there causing problems.

I took your advice and started by stopping Mailscanner and deleting the cache and any orphaned files in the directories, hopefully that will have a positive impact.

I assume it must be something odd like that, these messages started coming through last week. I have to believe if all my rules were not firing since I built that box a year or so ago I would have noticed this sooner.

One thing I noticed after taking other messages that failed due to body checks that actually wind up tagged as Spam, most seem to have more rules that fire off when I run them locally as root with -test-mode then what they have in my mail client after they have come through.

I do see hits on messages for rules that ONLY exist in some of the rules in the /var/lib/spamassassin/3.003001/updates_spamassassin_org directory. So I know it is accessing those files, just not all of them for some reason at certain times..

I just took a message that made it through today for me:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,
required 6.5, BAYES_50 0.80, DIET_1 0.00, RP_MATCHES_RCVD -0.00)

When I check this message on my MailScanner box with Spamassassin as root I get:

Content analysis details: (30.1 hits, 6.5 required)
6.5 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
[URIs: nthjus.com]
0.0 DIET_1 BODY: Lose Weight Spam
1.3 RCVD_IN_RP_RNBL RBL: Relay in RNBL,
https://senderscore.org/blacklistlookup/
[64.191.19.228 listed in bl.score.senderscore.com]
10 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: nthjus.com]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.5001]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]

It seems to be all the rules that don't fire are the ones where it would actually be looking something up, right? Through DNS?


BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com


From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Thursday, June 13, 2013 1:51 AM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

Are you running the tests against the same user MailScanner runs as to make sure any .spamassassin directory settings arent overriding
in both headers you're getting spamassassin cache hits which is a mailscanner option. You might want to stop MailScanner, delete the spamassassin cache file ans retry. Could be the cache file has got corrupt somehow.

martin

--
Martin Hepworth, CISSP
Oxford, UK

On 12 June 2013 22:05, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
spamassassin-3.3.1-3.el5.rf
mailscanner-4.83.5-1

Looking for some help here, it looks like sometimes Mailscanner is causing SpamAssassin to not use some rules. (Not exactly sure on this I assume it is Mailscanner based on the behavior I am seeing)

I receive the message and it is not tagged as Spam and winds up in my inbox. The headers show on this example:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,RP_MATCHES_RCVD -0.00)

I then take that message and drag it into a separate mailbox I had setup on our Exchange server, then pull it down to my Sendmail/Mailscanner/SpamAssassin box through imap in rfc822 format.

I then run the same message through Spamassassin with -test-mode locally from my mail server I get different scoring on, it looks like I am missing some of the checks because now it defiantly shows as Spam:

Content analysis details: (17.3 hits, 6.5 required)
5.0 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist
[URIs: eelefs.net<http://eelefs.net>]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.5050]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
0.0 LOTS_OF_MONEY Huge... sums of money

------ End of SpamAssassin results, Original message follows --------

So I was wondering if it had to with my MailScanner.conf having this line: SpamAssassin Local State Dir = # /var/lib/spamassassin

But based on my debug of MailScanner, it does not matter if the # is present or not, MailScanner seems to think it knows where all the rules are. The below output is with SpamAssassin Local State Dir = /var/lib/spamassassin

In Debugging mode, not forking...
Trying to setlogsock(unix)
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: adding facilities: all
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: logging level is DBG
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: SpamAssassin version 3.3.1
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: Perl 5.008008, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/mail/spamassassin, LOCAL_STATE_DIR=/var/lib/spam
assassin
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: timing enabled
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: score set 0 chosen.
15:54:01 Jun 12 15:54:01.477 [32352] dbg: util: running in taint mode? no
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: is Net::DNS::Resolver available? yes
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: Net::DNS version: 0.65
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/init.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v310.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v312.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v320.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v330.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/var/lib/spamassassin/3.003001" for sys rules pre files
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/var/lib/spamassassin/3.003001" for default rules dir
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org.cf<http://updates_spamassassin_org.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: using "/etc/mail/spamassassin" for site rules dir
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum1.cf<http://70_sare_evilnum1.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/70_sare_unsub.cf<http://70_sare_unsub.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/chickenpox.cf<http://chickenpox.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/local.cf<http://local.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/mailscanner.cf<http://mailscanner.cf>
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: using "/root/.spamassassin/user_prefs" for user prefs file
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: read file /root/.spamassassin/user_prefs
15:54:01 Jun 12 15:54:01.484 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
15:54:01 Jun 12 15:54:01.488 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
15:54:01 Jun 12 15:54:01.491 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
15:54:01 Jun 12 15:54:01.494 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
15:54:01 Jun 12 15:54:01.496 [32352] dbg: pyzor: network tests on, attempting Pyzor

The odd thing here to me, is if I search my maillog for some of the hits from above, Like URIBL_DBL_SPAM, I am seeing many hits on this.. It just seems to be skipping some of the rules for a certain messages. I looked through

Anyone have any ideas where I can start to figure this one out? I checked my rules, but since some of the rules are firing I assumed it can't have anything to do with that..

Here is the complete output from the message I give as an example from above: (minus the spammy body)

Received: from CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com> (10.18.17.28) by
CHI-US-CAS-1B.us.kmz.com<http://CHI-US-CAS-1B.us.kmz.com> (10.125.15.2) with Microsoft SMTP Server (TLS) id
14.3.123.3; Wed, 12 Jun 2013 15:44:04 -0500
Received: from chi-us-vwall-01.us.kmz.com<http://chi-us-vwall-01.us.kmz.com> (10.18.16.181) by
CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com> (10.18.17.28) with Microsoft SMTP Server id
14.3.123.3; Wed, 12 Jun 2013 15:44:03 -0500
Received: from venus.kattenlaw.com<http://venus.kattenlaw.com> ([10.18.3.33]) by us.kmz.com<http://us.kmz.com>
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id 844d8c4f001d4ac4
; Wed, 12 Jun 2013 15:44:01 -0500
Received: from smtp1.eelefs.net<http://smtp1.eelefs.net> (smtp1.eelefs.net<http://smtp1.eelefs.net> [66.197.143.105]) by
venus.kattenlaw.com<http://venus.kattenlaw.com> (8.13.8/8.13.4) with ESMTP id r5CKi0H8028960 for
<brian.duncan at kmzr.com<mailto:brian.duncan at kmzr.com>>; Wed, 12 Jun 2013 15:44:03 -0500
From: 2013 Models <Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>>
To: "Duncan, Brian M." <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>>
Subject: *Reduction Information* 2013's for thousands less
Thread-Topic: *Reduction Information* 2013's for thousands less
Thread-Index: AQHOZ62T+0z+e2LgwkiBidggfWeC0A==
Date: Wed, 12 Jun 2013 15:43:58 -0500
Message-ID: <29295056e3e7741908e644022e5f0220 at smtp1.eelefs.net<mailto:29295056e3e7741908e644022e5f0220 at smtp1.eelefs.net>>
Reply-To: "Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>" <Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>>
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-AuthSource: CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailscanner-from: jorgemendoza at smtp1.eelefs.net<mailto:jorgemendoza at smtp1.eelefs.net>
x-mailscanner-spamcheck: not spam, SpamAssassin (cached, score=0.8, required
6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00, RP_MATCHES_RCVD -0.00)
x-kattenlaw-mailscanner-information:
x-mailscanner-spam: no
x-kattenlaw-mailscanner-id: r5CKi0H8028960
x-tm-imss-message-id: <844d8c4f001d4ac4 at us.kmz.com<mailto:844d8c4f001d4ac4 at us.kmz.com>>
x-kattenlaw: NS
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8737EB66163E6F4DA060748F2D862AD0 at kattenlaw.com<mailto:8737EB66163E6F4DA060748F2D862AD0 at kattenlaw.com>>
MIME-Version: 1.0

Thanks for any help.


===========================================================

CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue

Service, any tax advice contained herein is not intended or written to be used and cannot be used

by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.

===========================================================

CONFIDENTIALITY NOTICE:

This electronic mail message and any attached files contain information intended for the exclusive

use of the individual or entity to whom it is addressed and may contain information that is

proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you

are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or

distribution of this information may be subject to legal restriction or sanction. Please notify

the sender, by electronic mail or telephone, of any unintended recipients and delete the original

message without making any copies.

===========================================================

NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has

elected to be governed by the Illinois Uniform Partnership Act (1997).

===========================================================


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130614/413d766f/attachment.html

[Martin Hepworth]

Very odd can u pastebin the raw email and drop the pastebin link so we can
run it over our systems to compare

Looks like deleting the spamassassin cache made no difference.****
** **
This morning I received another spam that made it through.****
** **
This is what it scored when passed through Mailscanner/Spamassassin:****
** **
X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,****
required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,****
RP_MATCHES_RCVD -0.00)****
** **
I moved it over to my mailscanner/spamassassin box within 30 seconds of
receiving it and this is what it scored on my Mailscanner box from the
command line doing spamassassin ?test-mode < message.txt:****
** **
Content analysis details: (14.6 hits, 6.5 required)****
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain****
3.0 BAYES_60 BODY: Bayes spam probability is 60 to 80%****
[score: 0.6460]****
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level****
above 50%****
[cf: 100]****
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)****
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%****
[cf: 100]****
0.0 LOTS_OF_MONEY Huge... sums of money****
0.1 FROM_12LTRDOM From a 12-letter domain****
** **
------ End of SpamAssassin results, Original message follows --------****
** **
The really odd thing, is if I take the body and subject from the spam
message above and send it through a hotmail account I have (which I white
list, which is why that shows in the below results), this is what it scores
when passed through Mailscanner/Spamassassin:****
** **
X-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached,
****
score=20.146, required 6.5, autolearn=spam, AWL -13.90,***
*
BAYES_50 0.80, FREEMAIL_ENVFROM_END_DIGIT 0.25,
FREEMAIL_FROM 0.00,****
HTML_MESSAGE 0.00, LOTS_OF_MONEY 0.00,
RAZOR2_CF_RANGE_51_100 0.50,****
RAZOR2_CF_RANGE_E8_51_100 2.50, RAZOR2_CHECK 8.50,****
RCVD_IN_DNSWL_NONE -0.00, RP_MATCHES_RCVD -0.00, SPF_PASS
-0.00,****
URIBL_BLACK 10.00, URIBL_DBL_SPAM 5.00, URIBL_JP_SURBL
6.50)****
** **
This makes no sense to me, it?s almost like this specific Spammer has
figured out a way to get Mailscanner to stop scanning portions of its
message. ****
** **
I am going to turn off caching of spamassassin results next in my
mailscanner conf to see if that has any impact. ****
** **
If anyone has any ideas please let me know.****
** **
Brian****
** **
** **
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com <javascript:_e({}, 'cvml',
'brian.duncan at kattenlaw.com');> / www.kattenlaw.com
****
** **
*From:* mailscanner-bounces at lists.mailscanner.info <javascript:_e({},
mailscanner-bounces at lists.mailscanner.info <javascript:_e({}, 'cvml',
'mailscanner-bounces at lists.mailscanner.info');>] *On Behalf Of *Duncan,
Brian M.
*Sent:* Thursday, June 13, 2013 7:35 PM
*To:* MailScanner discussion
*Subject:* RE: Certain Spamassassin rules do not seem to be firing all of
the time****
** **
Thanks for the suggestions Martin.****
** **
I don?t have any specific user that I run as:****
** **
Run As User =****
** **
So I assume it is running as root? My tests with ?test-mode were run as
root.. I do have the .spamassassin dir in root that has bayes db?s that are
the ones that get updated, and I did confirm there was nothing there
causing problems.****
** **
I took your advice and started by stopping Mailscanner and deleting the
cache and any orphaned files in the directories, hopefully that will have a
positive impact.****
** **
I assume it must be something odd like that, these messages started
coming through last week. I have to believe if all my rules were not
firing since I built that box a year or so ago I would have noticed this
sooner.****
** **
One thing I noticed after taking other messages that failed due to body
checks that actually wind up tagged as Spam, most seem to have more rules
that fire off when I run them locally as root with ?test-mode then what
they have in my mail client after they have come through.****
** **
I do see hits on messages for rules that ONLY exist in some of the rules
in the /var/lib/spamassassin/3.003001/updates_spamassassin_org directory.
So I know it is accessing those files, just not all of them for some reason
at certain times..****
** **
I just took a message that made it through today for me: ****
** **
X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,****
required 6.5, BAYES_50 0.80, DIET_1 0.00, RP_MATCHES_RCVD
-0.00)****
** **
When I check this message on my MailScanner box with Spamassassin as root
I get:****
** **
Content analysis details: (30.1 hits,

[Duncan, Brian M.]

http://pastebin.com/VQs2FSxK

I also tried disabling caching with SpamAssassin in my Mailscanner.conf today. I don't think it made a difference.. I don't have many examples today, it seems as if this specific spammer is only sending out a few today.

The above example just came in within the last 15 minutes. It did manage to get classified as Spam, but when I compare what rules it hit on going through MailScanner/Spamassassin vs using the above text and scanning with -test-mode, some of the rules are not hitting when going through MailScanner/Spamassassin.

The rules it hits on for me through Mailscanner:

X-MailScanner-SpamCheck: spam, SpamAssassin (score=9.3, required 6.5,
BAYES_50 0.80, LOTS_OF_MONEY 0.00, RAZOR2_CHECK 8.50,
RP_MATCHES_RCVD -0.00)

The rules it hits on according to spamassassin -test-mode:

Content analysis details: (28.1 hits, 6.5 required)
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%
[score: 1.0000]
6.5 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
[URIs: eldmil.com]
5.0 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist
[URIs: eldmil.com]
10 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: eldmil.com]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.0 LOTS_OF_MONEY Huge... sums of money
-8.4 AWL AWL: From: address is in the auto white-list

------ End of SpamAssassin results, Original message follows --------

Thanks for your assistance.

Brian

BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com


From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Friday, June 14, 2013 12:15 PM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

Very odd can u pastebin the raw email and drop the pastebin link so we can run it over our systems to compare

On Friday, 14 June 2013, Duncan, Brian M. wrote:
Looks like deleting the spamassassin cache made no difference.

This morning I received another spam that made it through.

This is what it scored when passed through Mailscanner/Spamassassin:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,
required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,
RP_MATCHES_RCVD -0.00)

I moved it over to my mailscanner/spamassassin box within 30 seconds of receiving it and this is what it scored on my Mailscanner box from the command line doing spamassassin -test-mode < message.txt:

Content analysis details: (14.6 hits, 6.5 required)
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
3.0 BAYES_60 BODY: Bayes spam probability is 60 to 80%
[score: 0.6460]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
0.0 LOTS_OF_MONEY Huge... sums of money
0.1 FROM_12LTRDOM From a 12-letter domain

------ End of SpamAssassin results, Original message follows --------

The really odd thing, is if I take the body and subject from the spam message above and send it through a hotmail account I have (which I white list, which is why that shows in the below results), this is what it scores when passed through Mailscanner/Spamassassin:

X-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached,
score=20.146, required 6.5, autolearn=spam, AWL -13.90,
BAYES_50 0.80, FREEMAIL_ENVFROM_END_DIGIT 0.25, FREEMAIL_FROM 0.00,
HTML_MESSAGE 0.00, LOTS_OF_MONEY 0.00, RAZOR2_CF_RANGE_51_100 0.50,
RAZOR2_CF_RANGE_E8_51_100 2.50, RAZOR2_CHECK 8.50,
RCVD_IN_DNSWL_NONE -0.00, RP_MATCHES_RCVD -0.00, SPF_PASS -0.00,
URIBL_BLACK 10.00, URIBL_DBL_SPAM 5.00, URIBL_JP_SURBL 6.50)

This makes no sense to me, it's almost like this specific Spammer has figured out a way to get Mailscanner to stop scanning portions of its message.

I am going to turn off caching of spamassassin results next in my mailscanner conf to see if that has any impact.

If anyone has any ideas please let me know.

Brian


BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com<javascript:_e(%7b%7d,%20'cvml',%20'brian.duncan at kattenlaw.com');> / www.kattenlaw.com<http://www.kattenlaw.com>


From: mailscanner-bounces at lists.mailscanner.info<javascript:_e(%7b%7d,%20'cvml',%20'mailscanner-bounces at lists.mailscanner.info');> [mailto:mailscanner-bounces at lists.mailscanner.info<javascript:_e(%7b%7d,%20'cvml',%20'mailscanner-bounces at lists.mailscanner.info');>] On Behalf Of Duncan, Brian M.
Sent: Thursday, June 13, 2013 7:35 PM
To: MailScanner discussion
Subject: RE: Certain Spamassassin rules do not seem to be firing all of the time



Thanks for the suggestions Martin.



I don't have any specific user that I run as:



Run As User =



So I assume it is running as root? My tests with -test-mode were run as root.. I do have the .spamassassin dir in root that has bayes db's that are the ones that get updated, and I did confirm there was nothing there causing problems.



I took your advice and started by stopping Mailscanner and deleting the cache and any orphaned files in the directories, hopefully that will have a positive impact.



I assume it must be something odd like that, these messages started coming through last week. I have to believe if all my rules were not firing since I built that box a year or so ago I would have noticed this sooner.



One thing I noticed after taking other messages that failed due to body checks that actually wind up tagged as Spam, most seem to have more rules that fire off when I run them locally as root with -test-mode then what they have in my mail client after they have come through.



I do see hits on messages for rules that ONLY exist in some of the rules in the /var/lib/spamassassin/3.003001/updates_spamassassin_org directory. So I know it is accessing those files, just not all of them for some reason at certain times..



I just took a message that made it through today for me:



X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,

required 6.5, BAYES_50 0.80, DIET_1 0.00, RP_MATCHES_RCVD -0.00)



When I check this message on my MailScanner box with Spamassassin as root I get:



Content analysis details: (30.1 hits,


--
--
Martin Hepworth, CISSP
Oxford, UK

===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
CONFIDENTIALITY NOTICE:
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130614/5a315a0a/attachment-0001.html

[Duncan, Brian M.]

Here is one more that just came in to me and was not tagged as Spam:

http://pastebin.com/w8SJk660


Mailscanner/Spamassassin results:

X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)


--test-mode results:

Content analysis details: (10.5 hits, 6.5 required)
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%
[score: 1.0000]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
-7.5 AWL AWL: From: address is in the auto white-list

------ End of SpamAssassin results, Original message follows --------


===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
CONFIDENTIALITY NOTICE:
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130614/3da6d488/attachment.html

[Martin Hepworth]

Hmm most if the extra rules youre hitting are dns based
I'd check youre running a local caching dns server on the scanning box and
that youre not timing out the network checks in sa too quickly

Martin

Here is one more that just came in to me and was not tagged as Spam:****
** **
http://pastebin.com/w8SJk660****
** **
** **
Mailscanner/Spamassassin results:****
** **
X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
****
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)****
** **
** **
--test-mode results:****
** **
Content analysis details: (10.5 hits, 6.5 required)****
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%****
[score: 1.0000]****
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain****
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level****
above 50%****
[cf: 100]****
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)****
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%****
[cf: 100]****
-7.5 AWL AWL: From: address is in the auto white-list**
**
** **
------ End of SpamAssassin results, Original message follows --------****
** **
===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================

[Duncan, Brian M.]

Thanks, yes I noticed that, they all do seem to be the DNS rules. I do have a caching DNS server but it is on the local network. I will try and see if the behavior changes at all by running one locally on the box itself.

When you say "that youre not timing out the network checks in sa too quickly" I have not changed anything in the defaults of Mailscanner or included any directives that would lower whatever time limits are set by default.

I took a look at the last example I put on pastebin, and it looks like it took 3 seconds to go from my Mailscanner box to my next gateway.

Received: from venus.kattenlaw.com ([10.18.3.33]) by us.kmz.com
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id 8e3c2381002025b2
; Fri, 14 Jun 2013 14:01:09 -0500
Received: from a.loselit.net (a.loselit.net [66.96.254.156]) by
venus.kattenlaw.com (8.13.8/8.13.4) with ESMTP id r5EJ13oK014449 for
<brian.duncan at kmzr.com>; Fri, 14 Jun 2013 14:01:06 -0500

I am assuming the 3 seconds going from my incoming mail server Venus, to the next hop in my environment includes the time it took for the Spammer to send me the message.

I also don't see anything in my maillogs related to Spam Assassin timing out for anything.. I recall many years ago when we used to run systems with much less CPU power (10+) seeing Spam Assassin time outs.

Which BTW, at the peak of activity today the lowest idle %idle was 91.00 and that is because I turned off caching of SpamAssassin in Mailscanner to see if that had any impact.

I also looked at the local caching DNS server that is on the same switch as this box, and it was peaking at like 30 Kilobytes per second on UDP 53 requests from anything that uses it locally according to iptraf.

It also seems to be these messages from the same Spammer, as I said before if I take any of these message bodies and send them in myself I seem to get the DNS Spam Assassin hits then.

Really odd one..

Thanks for your help



BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com


From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Friday, June 14, 2013 4:16 PM
To: MailScanner discussion
Subject: Certain Spamassassin rules do not seem to be firing all of the time

Hmm most if the extra rules youre hitting are dns based
I'd check youre running a local caching dns server on the scanning box and that youre not timing out the network checks in sa too quickly

Martin

On Friday, 14 June 2013, Duncan, Brian M. wrote:
Here is one more that just came in to me and was not tagged as Spam:

http://pastebin.com/w8SJk660


Mailscanner/Spamassassin results:

X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)


--test-mode results:

Content analysis details: (10.5 hits, 6.5 required)
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%
[score: 1.0000]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
-7.5 AWL AWL: From: address is in the auto white-list

------ End of SpamAssassin results, Original message follows --------


===========================================================

CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue

Service, any tax advice contained herein is not intended or written to be used and cannot be used

by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.

===========================================================

CONFIDENTIALITY NOTICE:

This electronic mail message and any attached files contain information intended for the exclusive

use of the individual or entity to whom it is addressed and may contain information that is

proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you

are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or

distribution of this information may be subject to legal restriction or sanction. Please notify

the sender, by electronic mail or telephone, of any unintended recipients and delete the original

message without making any copies.

===========================================================

NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has

elected to be governed by the Illinois Uniform Partnership Act (1997).

===========================================================



--
--
Martin Hepworth, CISSP
Oxford, UK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130615/75f6ec22/attachment.html

[Martin Hepworth]

Ok, you really need to put a local DNS server on the MailScanner box,
doesn't matter if the DNS resolver is next to the server in the switch
port, DNS is actually quite heavy on network traffic and hitting this all
the time makes a huge difference. It can forward to the current machine,
but the time this saves is actually quite noticable.

that three seconds for the pass across seems very quick to me, esp as it's
got all the DNS requests to process.I normally remove most of the RBL's
from being scanned in Spamassassin by giving most of them a zero score (see
50_scores.cf <http://spamassassin.apache.org/dist/rules/50_scores.cf> in
the DNSEval section). also make sure you're updating sa rules regularly. In
fact its almost as if you've got skip-rbl-checks set to 1 in a spamassassin
.cf or mailscanner,conf file somewhere.

I'd double check all the setup to make sure everythings OK, as it's really
odd that you're getting DNS based hits in test mode but not in test mode.
Check the MailScanner.conf setttings and any site MailScanner.conf, and
also get rid of any .spamassassin dirs esp if there's anything in root's
home dir (so i presume ther MTA is sendmail?) to make sure that isnt
overriding any settings. Check you've got one MailScanner.conf and not
multiple ones, sometimes some distributions put the active file in
'non-standard' places.
--
Martin Hepworth, CISSP
Oxford, UK

Post by Duncan, Brian M.
Thanks, yes I noticed that, they all do seem to be the DNS rules. I do
have a caching DNS server but it is on the local network. I will try and
see if the behavior changes at all by running one locally on the box itself.
****
** **
When you say ?that youre not timing out the network checks in sa too
quickly? I have not changed anything in the defaults of Mailscanner or
included any directives that would lower whatever time limits are set by
default.****
** **
I took a look at the last example I put on pastebin, and it looks like it
took 3 seconds to go from my Mailscanner box to my next gateway. ****
** **
Received: from venus.kattenlaw.com ([10.18.3.33]) by us.kmz.com****
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id
8e3c2381002025b2****
; Fri, 14 Jun 2013 14:01:09 -0500****
Received: from a.loselit.net (a.loselit.net [66.96.254.156]) by****
venus.kattenlaw.com (8.13.8/8.13.4) with ESMTP id r5EJ13oK014449
for****
<brian.duncan at kmzr.com>; Fri, 14 Jun 2013 14:01:06 -0500****
** **
I am assuming the 3 seconds going from my incoming mail server Venus, to
the next hop in my environment includes the time it took for the Spammer to
send me the message.****
** **
I also don?t see anything in my maillogs related to Spam Assassin timing
out for anything.. I recall many years ago when we used to run systems with
much less CPU power (10+) seeing Spam Assassin time outs.****
** **
Which BTW, at the peak of activity today the lowest idle %idle was 91.00
and that is because I turned off caching of SpamAssassin in Mailscanner to
see if that had any impact.****
** **
I also looked at the local caching DNS server that is on the same switch
as this box, and it was peaking at like 30 Kilobytes per second on UDP 53
requests from anything that uses it locally according to iptraf.****
** **
It also seems to be these messages from the same Spammer, as I said before
if I take any of these message bodies and send them in myself I seem to get
the DNS Spam Assassin hits then. ****
** **
Really odd one..****
** **
Thanks for your help****
** **
** **
** **
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com
****
** **
mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Martin Hepworth
*Sent:* Friday, June 14, 2013 4:16 PM
*To:* MailScanner discussion
*Subject:* Certain Spamassassin rules do not seem to be firing all of the
time****
** **
Hmm most if the extra rules youre hitting are dns based****
I'd check youre running a local caching dns server on the scanning box and
that youre not timing out the network checks in sa too quickly****
** **
Martin
On Friday, 14 June 2013, Duncan, Brian M. wrote:****
Here is one more that just came in to me and was not tagged as Spam:****
****
http://pastebin.com/w8SJk660****
****
****
Mailscanner/Spamassassin results:****
****
X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
****
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)****
****
****
--test-mode results:****
****
Content analysis details: (10.5 hits, 6.5 required)****
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%****
[score: 1.0000]****
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain****
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level****
above 50%****
[cf: 100]****
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)****
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%****
[cf: 100]****
-7.5 AWL AWL: From: address is in the auto white-list**
**
****
------ End of SpamAssassin results, Original message follows --------****
****
===========================================================****
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue****
Service, any tax advice contained herein is not intended or written to be used and cannot be used****
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.****
===========================================================****
CONFIDENTIALITY NOTICE:****
This electronic mail message and any attached files contain information intended for the exclusive****
use of the individual or entity to whom it is addressed and may contain information that is****
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you****
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or ****
distribution of this information may be subject to legal restriction or sanction. Please notify****
the sender, by electronic mail or telephone, of any unintended recipients and delete the original ****
message without making any copies.****
===========================================================****
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has****
elected to be governed by the Illinois Uniform Partnership Act (1997).****
===========================================================****
--
--
Martin Hepworth, CISSP
Oxford, UK****
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130615/99eef3d6/attachment.html

[Duncan, Brian M.]

Thanks for the recommendations Martin.

The way I have it setup in Mailscanner is if the sending mail server is on a RBL, (If it on at least 1 of the 4 RBLS we use with MailScanner) It becomes high scoring spam and is tagged and moved on, it does not get scanned by Spamassassin then.

The one thing I never considered before was if Spamassassin is scanning the same sending mail server IP for being listed when it does not get caught by MailScanner as being on any of the 4 RB:'s I use. Not that it is causing my problem now, but that it is not very efficient if it is doing it again. (I would guess 80% of my mail never gets scanned by SpamAssassin each day because the sending mail gateway is blacklisted and it is marked as Spam and moves on)

When you say: " I normally remove most of the RBL's from being scanned in Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf> in the DNSEval section)."

I don't think I follow, are you saying Spamassassin is scanning the sending mail host again against the RBL's? So by giving them a zero score you are avoiding the double effort? This section has nothing to do with the URL/URI scanning that is happening? I assumed the rules that I have that are NOT hitting when it goes through MailScanner/Spamassassin have all been based on the URI/URL's in the body of the message.


When I take these specific Spam messages that make it into my inbox, I am noticing they never hit on the same URIBL hits I get when I move the message locally to the box, if I take one of these URI based RBL checking rules like for example URIBL_BLACK, I have never seen that rule hit on ANY of these ones making it into my inbox. If I search my maillog from yesterday for every message that wound up being scanned by Spamassassin, I see that there were 1014 times that rule is listed on detected Spam.

Last night I first tried setting up a caching bind server local to the box. Made no difference.

I tried upgrading to MailScanner 4.84.5-3 after and updating to SpamAssassin 3.3.2-1 to see if that would make a difference, I even looked at the Perl modules that come with MailScanner, one of them was perl-Net-DNS-0.65-2, I was running
perl-Net-DNS-0.65-1, was hoping that had something to do with this so I updated to .65-2 of that perl modules.. the rest all seemed to be the same version that comes with 4.84.5-3 (I was running 4.83.5-1 before I updated)

I went over all my configs for both MailScanner and SpamAssassin, nothing seems wrong or set to low that would create the situation I am seeing. I did find I had the pyzor plugin loading in SpamAssassin but no exe, so I just disabled pyzor and verified in the -debug-sa that everything looks fine.

I waited and sure enough it happened again today. We get less mail on the weekends so it took awhile waiting..

I have posted my MailScanner -debug-sa to pastebin if anyone can take a look and give me a recommendation of where to look next. I am almost out of things to try.

Here is one -debug-sa:

http://pastebin.com/C2XPs7D2

Then I kept running with -debug-sa till it caught one with a DNS based rule like URIBL_* rules.

This one hits on those URIBL rules that are DNS based and it looks like everything is OK as far as I can tell.. This is really the first time I have tried to debug a debug log from MailScanner/Spamassassin before..

http://pastebin.com/iWMnJqf3


Thanks



BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com


From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Saturday, June 15, 2013 8:46 AM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

Ok, you really need to put a local DNS server on the MailScanner box, doesn't matter if the DNS resolver is next to the server in the switch port, DNS is actually quite heavy on network traffic and hitting this all the time makes a huge difference. It can forward to the current machine, but the time this saves is actually quite noticable.
that three seconds for the pass across seems very quick to me, esp as it's got all the DNS requests to process.I normally remove most of the RBL's from being scanned in Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf> in the DNSEval section). also make sure you're updating sa rules regularly. In fact its almost as if you've got skip-rbl-checks set to 1 in a spamassassin .cf or mailscanner,conf file somewhere.
I'd double check all the setup to make sure everythings OK, as it's really odd that you're getting DNS based hits in test mode but not in test mode. Check the MailScanner.conf setttings and any site MailScanner.conf, and also get rid of any .spamassassin dirs esp if there's anything in root's home dir (so i presume ther MTA is sendmail?) to make sure that isnt overriding any settings. Check you've got one MailScanner.conf and not multiple ones, sometimes some distributions put the active file in 'non-standard' places.


--
Martin Hepworth, CISSP
Oxford, UK

On 15 June 2013 03:22, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
Thanks, yes I noticed that, they all do seem to be the DNS rules. I do have a caching DNS server but it is on the local network. I will try and see if the behavior changes at all by running one locally on the box itself.

When you say "that youre not timing out the network checks in sa too quickly" I have not changed anything in the defaults of Mailscanner or included any directives that would lower whatever time limits are set by default.

I took a look at the last example I put on pastebin, and it looks like it took 3 seconds to go from my Mailscanner box to my next gateway.

Received: from venus.kattenlaw.com<http://venus.kattenlaw.com> ([10.18.3.33]) by us.kmz.com<http://us.kmz.com>
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id 8e3c2381002025b2
; Fri, 14 Jun 2013 14:01:09 -0500
Received: from a.loselit.net<http://a.loselit.net> (a.loselit.net<http://a.loselit.net> [66.96.254.156]) by
venus.kattenlaw.com<http://venus.kattenlaw.com> (8.13.8/8.13.4) with ESMTP id r5EJ13oK014449 for
<brian.duncan at kmzr.com<mailto:brian.duncan at kmzr.com>>; Fri, 14 Jun 2013 14:01:06 -0500

I am assuming the 3 seconds going from my incoming mail server Venus, to the next hop in my environment includes the time it took for the Spammer to send me the message.

I also don't see anything in my maillogs related to Spam Assassin timing out for anything.. I recall many years ago when we used to run systems with much less CPU power (10+) seeing Spam Assassin time outs.

Which BTW, at the peak of activity today the lowest idle %idle was 91.00 and that is because I turned off caching of SpamAssassin in Mailscanner to see if that had any impact.

I also looked at the local caching DNS server that is on the same switch as this box, and it was peaking at like 30 Kilobytes per second on UDP 53 requests from anything that uses it locally according to iptraf.

It also seems to be these messages from the same Spammer, as I said before if I take any of these message bodies and send them in myself I seem to get the DNS Spam Assassin hits then.

Really odd one..

Thanks for your help



BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com> / www.kattenlaw.com<http://www.kattenlaw.com>


From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Martin Hepworth
Sent: Friday, June 14, 2013 4:16 PM
To: MailScanner discussion
Subject: Certain Spamassassin rules do not seem to be firing all of the time

Hmm most if the extra rules youre hitting are dns based
I'd check youre running a local caching dns server on the scanning box and that youre not timing out the network checks in sa too quickly

Martin

On Friday, 14 June 2013, Duncan, Brian M. wrote:
Here is one more that just came in to me and was not tagged as Spam:

http://pastebin.com/w8SJk660


Mailscanner/Spamassassin results:

X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)


--test-mode results:

Content analysis details: (10.5 hits, 6.5 required)
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%
[score: 1.0000]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
-7.5 AWL AWL: From: address is in the auto white-list

------ End of SpamAssassin results, Original message follows --------


===========================================================

CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue

Service, any tax advice contained herein is not intended or written to be used and cannot be used

by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.

===========================================================

CONFIDENTIALITY NOTICE:

This electronic mail message and any attached files contain information intended for the exclusive

use of the individual or entity to whom it is addressed and may contain information that is

proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you

are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or

distribution of this information may be subject to legal restriction or sanction. Please notify

the sender, by electronic mail or telephone, of any unintended recipients and delete the original

message without making any copies.

===========================================================

NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has

elected to be governed by the Illinois Uniform Partnership Act (1997).

===========================================================



--
--
Martin Hepworth, CISSP
Oxford, UK

--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130615/841ea31f/attachment.html

[Martin Hepworth]

really odd, seems to be suffering with network based rules, not just the
URI ones but razor as well.
Personally I always put all the RBL checks into SA rather than letting
MailScanner do it by itself. that way no 1 rbl can false postive and email
and the RBL just add to the overall score.

What MTA are you running? and is there a .spamassassin directory in root's
home dir?
--
Martin Hepworth, CISSP
Oxford, UK

Thanks for the recommendations Martin.****
** **
The way I have it setup in Mailscanner is if the sending mail server is on
a RBL, (If it on at least 1 of the 4 RBLS we use with MailScanner) It
becomes high scoring spam and is tagged and moved on, it does not get
scanned by Spamassassin then.****
** **
The one thing I never considered before was if Spamassassin is scanning
the same sending mail server IP for being listed when it does not get
caught by MailScanner as being on any of the 4 RB:?s I use. Not that it
is causing my problem now, but that it is not very efficient if it is doing
it again. (I would guess 80% of my mail never gets scanned by SpamAssassin
each day because the sending mail gateway is blacklisted and it is marked
as Spam and moves on)****
** **
When you say: ? I normally remove most of the RBL's from being scanned in
Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf>in the DNSEval section).?
****
** **
I don?t think I follow, are you saying Spamassassin is scanning the
sending mail host again against the RBL?s? So by giving them a zero score
you are avoiding the double effort? This section has nothing to do with the
URL/URI scanning that is happening? I assumed the rules that I have that
are NOT hitting when it goes through MailScanner/Spamassassin have all been
based on the URI/URL?s in the body of the message. ****
** **
** **
When I take these specific Spam messages that make it into my inbox, I am
noticing they never hit on the same URIBL hits I get when I move the
message locally to the box, if I take one of these URI based RBL checking
rules like for example URIBL_BLACK, I have never seen that rule hit on ANY
of these ones making it into my inbox. If I search my maillog from
yesterday for every message that wound up being scanned by Spamassassin, I
see that there were 1014 times that rule is listed on detected Spam. ****
** **
Last night I first tried setting up a caching bind server local to the
box. Made no difference.****
** **
I tried upgrading to MailScanner 4.84.5-3 after and updating to
SpamAssassin 3.3.2-1 to see if that would make a difference, I even looked
at the Perl modules that come with MailScanner, one of them was
perl-Net-DNS-0.65-2, I was running****
perl-Net-DNS-0.65-1, was hoping that had something to do with this so I
updated to .65-2 of that perl modules.. the rest all seemed to be the same
version that comes with 4.84.5-3 (I was running 4.83.5-1 before I updated)
****
** **
I went over all my configs for both MailScanner and SpamAssassin, nothing
seems wrong or set to low that would create the situation I am seeing. I
did find I had the pyzor plugin loading in SpamAssassin but no exe, so I
just disabled pyzor and verified in the ?debug-sa that everything looks
fine.****
** **
I waited and sure enough it happened again today. We get less mail on the
weekends so it took awhile waiting..****
** **
I have posted my MailScanner ?debug-sa to pastebin if anyone can take a
look and give me a recommendation of where to look next. I am almost out
of things to try.****
** **
Here is one ?debug-sa:****
** **
http://pastebin.com/C2XPs7D2****
** **
Then I kept running with ?debug-sa till it caught one with a DNS based
rule like URIBL_* rules.****
** **
This one hits on those URIBL rules that are DNS based and it looks like
everything is OK as far as I can tell.. This is really the first time I
have tried to debug a debug log from MailScanner/Spamassassin before..****
** **
http://pastebin.com/iWMnJqf3****
** **
** **
Thanks****
** **
** **
** **
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com
****
** **
mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Martin Hepworth
*Sent:* Saturday, June 15, 2013 8:46 AM
*To:* MailScanner discussion
*Subject:* Re: Certain Spamassassin rules do not seem to be firing all of
the time****
** **
Ok, you really need to put a local DNS server on the MailScanner box,
doesn't matter if the DNS resolver is next to the server in the switch
port, DNS is actually quite heavy on network traffic and hitting this all
the time makes a huge difference. It can forward to the current machine,
but the time this saves is actually quite noticable.****
that three seconds for the pass across seems very quick to me, esp as it's
got all the DNS requests to process.I normally remove most of the RBL's
from being scanned in Spamassassin by giving most of them a zero score (see
50_scores.cf <http://spamassassin.apache.org/dist/rules/50_scores.cf> in
the DNSEval section). also make sure you're updating sa rules regularly. In
fact its almost as if you've got skip-rbl-checks set to 1 in a spamassassin
.cf or mailscanner,conf file somewhere.****
I'd double check all the setup to make sure everythings OK, as it's really
odd that you're getting DNS based hits in test mode but not in test mode.
Check the MailScanner.conf setttings and any site MailScanner.conf, and
also get rid of any .spamassassin dirs esp if there's anything in root's
home dir (so i presume ther MTA is sendmail?) to make sure that isnt
overriding any settings. Check you've got one MailScanner.conf and not
multiple ones, sometimes some distributions put the active file in
'non-standard' places.
****
****
--
Martin Hepworth, CISSP
Oxford, UK****
** **
On 15 June 2013 03:22, Duncan, Brian M. <brian.duncan at kattenlaw.com>
wrote:****
Thanks, yes I noticed that, they all do seem to be the DNS rules. I do
have a caching DNS server but it is on the local network. I will try and
see if the behavior changes at all by running one locally on the box itself.
****
****
When you say ?that youre not timing out the network checks in sa too
quickly? I have not changed anything in the defaults of Mailscanner or
included any directives that would lower whatever time limits are set by
default.****
****
I took a look at the last example I put on pastebin, and it looks like it
took 3 seconds to go from my Mailscanner box to my next gateway. ****
****
Received: from venus.kattenlaw.com ([10.18.3.33]) by us.kmz.com****
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id
8e3c2381002025b2****
; Fri, 14 Jun 2013 14:01:09 -0500****
Received: from a.loselit.net (a.loselit.net [66.96.254.156]) by****
venus.kattenlaw.com (8.13.8/8.13.4) with ESMTP id r5EJ13oK014449
for****
<brian.duncan at kmzr.com>; Fri, 14 Jun 2013 14:01:06 -0500****
****
I am assuming the 3 seconds going from my incoming mail server Venus, to
the next hop in my environment includes the time it took for the Spammer to
send me the message.****
****
I also don?t see anything in my maillogs related to Spam Assassin timing
out for anything.. I recall many years ago when we used to run systems with
much less CPU power (10+) seeing Spam Assassin time outs.****
****
Which BTW, at the peak of activity today the lowest idle %idle was 91.00
and that is because I turned off caching of SpamAssassin in Mailscanner to
see if that had any impact.****
****
I also looked at the local caching DNS server that is on the same switch
as this box, and it was peaking at like 30 Kilobytes per second on UDP 53
requests from anything that uses it locally according to iptraf.****
****
It also seems to be these messages from the same Spammer, as I said before
if I take any of these message bodies and send them in myself I seem to get
the DNS Spam Assassin hits then. ****
****
Really odd one..****
****
Thanks for your help****
****
****
****
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com
****
****
mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Martin Hepworth
*Sent:* Friday, June 14, 2013 4:16 PM
*To:* MailScanner discussion
*Subject:* Certain Spamassassin rules do not seem to be firing all of the
time****
****
Hmm most if the extra rules youre hitting are dns based****
I'd check youre running a local caching dns server on the scanning box and
that youre not timing out the network checks in sa too quickly****
****
Martin
On Friday, 14 June 2013, Duncan, Brian M. wrote:****
Here is one more that just came in to me and was not tagged as Spam:****
****
http://pastebin.com/w8SJk660****
****
****
Mailscanner/Spamassassin results:****
****
X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
****
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)****
****
****
--test-mode results:****
****
Content analysis details: (10.5 hits, 6.5 required)****
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%****
[score: 1.0000]****
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain****
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level****
above 50%****
[cf: 100]****
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)****
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%****
[cf: 100]****
-7.5 AWL AWL: From: address is in the auto white-list**
**
****
------ End of SpamAssassin results, Original message follows --------****
****
===========================================================****
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue****
Service, any tax advice contained herein is not intended or written to be used and cannot be used****
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.****
===========================================================****
CONFIDENTIALITY NOTICE:****
This electronic mail message and any attached files contain information intended for the exclusive****
use of the individual or entity to whom it is addressed and may contain information that is****
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you****
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or ****
distribution of this information may be subject to legal restriction or sanction. Please notify****
the sender, by electronic mail or telephone, of any unintended recipients and delete the original ****
message without making any copies.****
===========================================================****
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has****
elected to be governed by the Illinois Uniform Partnership Act (1997).****
===========================================================****
--
--
Martin Hepworth, CISSP
Oxford, UK****
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!****
** **
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130618/31a09fe7/attachment.html

[Duncan, Brian M.]

Yeah I know it's very weird and I can't track it down.

Yesterday, I tried removing the NET:DNS perl module (.65 is what MailScanner (and I believe SpamAssassin use by default) and compiling 0.72 in the hopes that it had something to do with that. Nope, still happening today. Fortunately it only seems to be letting a few Spam in overall. It just happens when there is a black listed domain that is used in a URL that is sent by a non-blacklisted gateway where I get caught by this issue.

I am using Sendmail. Yes there is a .spamassassin directory in root, where the bayes db's are located and autowhitelist db's (I have autowhite list disabled for the moment) The user_prefs file has no directives set in it, they are all #'ed out.

I don't specify a run as user in my MailScanner.conf, and according to ps all the MailScanner processes are running as root, and my spamassassin -test-mode I have run as root.

I turned on skip_rbl_checks 1 yesterday, since I detect RBL'ed hosts using MailScanner I figured it was kind of pointless to do it again with SpamAssassin..

I also tried tweaking rbl_timeout to 60 seconds instead of 30, what is was before. Because I did find someone else reporting a similar issue to mine.. back in 2007 someone was reporting this same behavior that rules were not hitting when using Amavis with Spamassassin, but then when you ran them through Spamassassin they worked, and I believe it was the same types of rules I am not hitting on through MailScanner. And the issue wound up being Net:DNS.

http://www.gossamer-threads.com/lists/spamassassin/users/102307?do=post_view_threaded#102307

If I can't figure this out, I might attempt a fresh install of Cent OS 6.4 and fresh install of MailScanner and SpamAssassin. Just don't have the time for a full re-install right now.

Yeah I prefer doing the RBL's with MailScanner because it uses such little CPU to perform those tests. I mark as Spam probably as high as 80% of my incoming mail just on the RBL checks through MailScanner. Spamasssassin only has to process like 20% of my Spam mail then. Since we pass everything through and assign the Microsoft SCL score based on if it failed Spam checks, the users can whitelist if someone winds up on an RBL or they want what most consider Spam.


BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com


From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Tuesday, June 18, 2013 10:09 AM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

really odd, seems to be suffering with network based rules, not just the URI ones but razor as well.
Personally I always put all the RBL checks into SA rather than letting MailScanner do it by itself. that way no 1 rbl can false postive and email and the RBL just add to the overall score.

What MTA are you running? and is there a .spamassassin directory in root's home dir?

--
Martin Hepworth, CISSP
Oxford, UK

On 15 June 2013 21:08, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
Thanks for the recommendations Martin.

The way I have it setup in Mailscanner is if the sending mail server is on a RBL, (If it on at least 1 of the 4 RBLS we use with MailScanner) It becomes high scoring spam and is tagged and moved on, it does not get scanned by Spamassassin then.

The one thing I never considered before was if Spamassassin is scanning the same sending mail server IP for being listed when it does not get caught by MailScanner as being on any of the 4 RB:'s I use. Not that it is causing my problem now, but that it is not very efficient if it is doing it again. (I would guess 80% of my mail never gets scanned by SpamAssassin each day because the sending mail gateway is blacklisted and it is marked as Spam and moves on)

When you say: " I normally remove most of the RBL's from being scanned in Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf> in the DNSEval section)."

I don't think I follow, are you saying Spamassassin is scanning the sending mail host again against the RBL's? So by giving them a zero score you are avoiding the double effort? This section has nothing to do with the URL/URI scanning that is happening? I assumed the rules that I have that are NOT hitting when it goes through MailScanner/Spamassassin have all been based on the URI/URL's in the body of the message.


When I take these specific Spam messages that make it into my inbox, I am noticing they never hit on the same URIBL hits I get when I move the message locally to the box, if I take one of these URI based RBL checking rules like for example URIBL_BLACK, I have never seen that rule hit on ANY of these ones making it into my inbox. If I search my maillog from yesterday for every message that wound up being scanned by Spamassassin, I see that there were 1014 times that rule is listed on detected Spam.

Last night I first tried setting up a caching bind server local to the box. Made no difference.

I tried upgrading to MailScanner 4.84.5-3 after and updating to SpamAssassin 3.3.2-1 to see if that would make a difference, I even looked at the Perl modules that come with MailScanner, one of them was perl-Net-DNS-0.65-2, I was running
perl-Net-DNS-0.65-1, was hoping that had something to do with this so I updated to .65-2 of that perl modules.. the rest all seemed to be the same version that comes with 4.84.5-3 (I was running 4.83.5-1 before I updated)

I went over all my configs for both MailScanner and SpamAssassin, nothing seems wrong or set to low that would create the situation I am seeing. I did find I had the pyzor plugin loading in SpamAssassin but no exe, so I just disabled pyzor and verified in the -debug-sa that everything looks fine.

I waited and sure enough it happened again today. We get less mail on the weekends so it took awhile waiting..

I have posted my MailScanner -debug-sa to pastebin if anyone can take a look and give me a recommendation of where to look next. I am almost out of things to try.

Here is one -debug-sa:

http://pastebin.com/C2XPs7D2

Then I kept running with -debug-sa till it caught one with a DNS based rule like URIBL_* rules.

This one hits on those URIBL rules that are DNS based and it looks like everything is OK as far as I can tell.. This is really the first time I have tried to debug a debug log from MailScanner/Spamassassin before..

http://pastebin.com/iWMnJqf3


Thanks



BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com> / www.kattenlaw.com<http://www.kattenlaw.com>


From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Martin Hepworth
Sent: Saturday, June 15, 2013 8:46 AM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

Ok, you really need to put a local DNS server on the MailScanner box, doesn't matter if the DNS resolver is next to the server in the switch port, DNS is actually quite heavy on network traffic and hitting this all the time makes a huge difference. It can forward to the current machine, but the time this saves is actually quite noticable.
that three seconds for the pass across seems very quick to me, esp as it's got all the DNS requests to process.I normally remove most of the RBL's from being scanned in Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf> in the DNSEval section). also make sure you're updating sa rules regularly. In fact its almost as if you've got skip-rbl-checks set to 1 in a spamassassin .cf or mailscanner,conf file somewhere.
I'd double check all the setup to make sure everythings OK, as it's really odd that you're getting DNS based hits in test mode but not in test mode. Check the MailScanner.conf setttings and any site MailScanner.conf, and also get rid of any .spamassassin dirs esp if there's anything in root's home dir (so i presume ther MTA is sendmail?) to make sure that isnt overriding any settings. Check you've got one MailScanner.conf and not multiple ones, sometimes some distributions put the active file in 'non-standard' places.

--
Martin Hepworth, CISSP
Oxford, UK

On 15 June 2013 03:22, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
Thanks, yes I noticed that, they all do seem to be the DNS rules. I do have a caching DNS server but it is on the local network. I will try and see if the behavior changes at all by running one locally on the box itself.

When you say "that youre not timing out the network checks in sa too quickly" I have not changed anything in the defaults of Mailscanner or included any directives that would lower whatever time limits are set by default.

I took a look at the last example I put on pastebin, and it looks like it took 3 seconds to go from my Mailscanner box to my next gateway.

Received: from venus.kattenlaw.com<http://venus.kattenlaw.com> ([10.18.3.33]) by us.kmz.com<http://us.kmz.com>
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id 8e3c2381002025b2
; Fri, 14 Jun 2013 14:01:09 -0500
Received: from a.loselit.net<http://a.loselit.net> (a.loselit.net<http://a.loselit.net> [66.96.254.156]) by
venus.kattenlaw.com<http://venus.kattenlaw.com> (8.13.8/8.13.4) with ESMTP id r5EJ13oK014449 for
<brian.duncan at kmzr.com<mailto:brian.duncan at kmzr.com>>; Fri, 14 Jun 2013 14:01:06 -0500

I am assuming the 3 seconds going from my incoming mail server Venus, to the next hop in my environment includes the time it took for the Spammer to send me the message.

I also don't see anything in my maillogs related to Spam Assassin timing out for anything.. I recall many years ago when we used to run systems with much less CPU power (10+) seeing Spam Assassin time outs.

Which BTW, at the peak of activity today the lowest idle %idle was 91.00 and that is because I turned off caching of SpamAssassin in Mailscanner to see if that had any impact.

I also looked at the local caching DNS server that is on the same switch as this box, and it was peaking at like 30 Kilobytes per second on UDP 53 requests from anything that uses it locally according to iptraf.

It also seems to be these messages from the same Spammer, as I said before if I take any of these message bodies and send them in myself I seem to get the DNS Spam Assassin hits then.

Really odd one..

Thanks for your help



BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com> / www.kattenlaw.com<http://www.kattenlaw.com>


From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Martin Hepworth
Sent: Friday, June 14, 2013 4:16 PM
To: MailScanner discussion
Subject: Certain Spamassassin rules do not seem to be firing all of the time

Hmm most if the extra rules youre hitting are dns based
I'd check youre running a local caching dns server on the scanning box and that youre not timing out the network checks in sa too quickly

Martin

On Friday, 14 June 2013, Duncan, Brian M. wrote:
Here is one more that just came in to me and was not tagged as Spam:

http://pastebin.com/w8SJk660


Mailscanner/Spamassassin results:

X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)


--test-mode results:

Content analysis details: (10.5 hits, 6.5 required)
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%
[score: 1.0000]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
-7.5 AWL AWL: From: address is in the auto white-list

------ End of SpamAssassin results, Original message follows --------


===========================================================

CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue

Service, any tax advice contained herein is not intended or written to be used and cannot be used

by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.

===========================================================

CONFIDENTIALITY NOTICE:

This electronic mail message and any attached files contain information intended for the exclusive

use of the individual or entity to whom it is addressed and may contain information that is

proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you

are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or

distribution of this information may be subject to legal restriction or sanction. Please notify

the sender, by electronic mail or telephone, of any unintended recipients and delete the original

message without making any copies.

===========================================================

NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has

elected to be governed by the Illinois Uniform Partnership Act (1997).

===========================================================



--
--
Martin Hepworth, CISSP
Oxford, UK

--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130618/6c414f14/attachment-0001.html

[Martin Hepworth]

maybe you can use sendmail to call-out for valid recipients first, I find
this drops HUGE amounts of traffic dead before it gets anywhere near
MailScanner, easily 50% and maybe higher

http://wiki.mailscanner.info/doku.php?id=documentation:configuration:mta:sendmail:how_to:reject_non_existent_users
--
Martin Hepworth, CISSP
Oxford, UK

Yeah I know it?s very weird and I can?t track it down. ****
** **
Yesterday, I tried removing the NET:DNS perl module (.65 is what
MailScanner (and I believe SpamAssassin use by default) and compiling 0.72
in the hopes that it had something to do with that. Nope, still happening
today. Fortunately it only seems to be letting a few Spam in overall.
It just happens when there is a black listed domain that is used in a
URL that is sent by a non-blacklisted gateway where I get caught by this
issue.****
** **
I am using Sendmail. Yes there is a .spamassassin directory in root, where
the bayes db?s are located and autowhitelist db?s (I have autowhite list
disabled for the moment) The user_prefs file has no directives set in it,
they are all #?ed out.****
** **
I don?t specify a run as user in my MailScanner.conf, and according to ps
all the MailScanner processes are running as root, and my spamassassin
?test-mode I have run as root.****
** **
I turned on skip_rbl_checks 1 yesterday, since I detect RBL?ed hosts using
MailScanner I figured it was kind of pointless to do it again with
SpamAssassin.. ****
** **
I also tried tweaking rbl_timeout to 60 seconds instead of 30, what is was
before. Because I did find someone else reporting a similar issue to
mine.. back in 2007 someone was reporting this same behavior that rules
were not hitting when using Amavis with Spamassassin, but then when you ran
them through Spamassassin they worked, and I believe it was the same types
of rules I am not hitting on through MailScanner. And the issue wound up
being Net:DNS.****
** **
http://www.gossamer-threads.com/lists/spamassassin/users/102307?do=post_view_threaded#102307
****
** **
If I can?t figure this out, I might attempt a fresh install of Cent OS 6.4
and fresh install of MailScanner and SpamAssassin. Just don?t have the
time for a full re-install right now.****
** **
Yeah I prefer doing the RBL?s with MailScanner because it uses such little
CPU to perform those tests. I mark as Spam probably as high as 80% of my
incoming mail just on the RBL checks through MailScanner. Spamasssassin
only has to process like 20% of my Spam mail then. Since we pass
everything through and assign the Microsoft SCL score based on if it failed
Spam checks, the users can whitelist if someone winds up on an RBL or they
want what most consider Spam.****
** **
** **
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com
****
** **
mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Martin Hepworth
*Sent:* Tuesday, June 18, 2013 10:09 AM
*To:* MailScanner discussion
*Subject:* Re: Certain Spamassassin rules do not seem to be firing all of
the time****
** **
really odd, seems to be suffering with network based rules, not just the
URI ones but razor as well.****
Personally I always put all the RBL checks into SA rather than letting
MailScanner do it by itself. that way no 1 rbl can false postive and email
and the RBL just add to the overall score.****
** **
What MTA are you running? and is there a .spamassassin directory in root's
home dir?****
****
--
Martin Hepworth, CISSP
Oxford, UK****
** **
On 15 June 2013 21:08, Duncan, Brian M. <brian.duncan at kattenlaw.com>
wrote:****
Thanks for the recommendations Martin.****
****
The way I have it setup in Mailscanner is if the sending mail server is on
a RBL, (If it on at least 1 of the 4 RBLS we use with MailScanner) It
becomes high scoring spam and is tagged and moved on, it does not get
scanned by Spamassassin then.****
****
The one thing I never considered before was if Spamassassin is scanning
the same sending mail server IP for being listed when it does not get
caught by MailScanner as being on any of the 4 RB:?s I use. Not that it
is causing my problem now, but that it is not very efficient if it is doing
it again. (I would guess 80% of my mail never gets scanned by SpamAssassin
each day because the sending mail gateway is blacklisted and it is marked
as Spam and moves on)****
****
When you say: ? I normally remove most of the RBL's from being scanned in
Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf>in the DNSEval section).?
****
****
I don?t think I follow, are you saying Spamassassin is scanning the
sending mail host again against the RBL?s? So by giving them a zero score
you are avoiding the double effort? This section has nothing to do with the
URL/URI scanning that is happening? I assumed the rules that I have that
are NOT hitting when it goes through MailScanner/Spamassassin have all been
based on the URI/URL?s in the body of the message. ****
****
****
When I take these specific Spam messages that make it into my inbox, I am
noticing they never hit on the same URIBL hits I get when I move the
message locally to the box, if I take one of these URI based RBL checking
rules like for example URIBL_BLACK, I have never seen that rule hit on ANY
of these ones making it into my inbox. If I search my maillog from
yesterday for every message that wound up being scanned by Spamassassin, I
see that there were 1014 times that rule is listed on detected Spam. ****
****
Last night I first tried setting up a caching bind server local to the
box. Made no difference.****
****
I tried upgrading to MailScanner 4.84.5-3 after and updating to
SpamAssassin 3.3.2-1 to see if that would make a difference, I even looked
at the Perl modules that come with MailScanner, one of them was
perl-Net-DNS-0.65-2, I was running****
perl-Net-DNS-0.65-1, was hoping that had something to do with this so I
updated to .65-2 of that perl modules.. the rest all seemed to be the same
version that comes with 4.84.5-3 (I was running 4.83.5-1 before I updated)
****
****
I went over all my configs for both MailScanner and SpamAssassin, nothing
seems wrong or set to low that would create the situation I am seeing. I
did find I had the pyzor plugin loading in SpamAssassin but no exe, so I
just disabled pyzor and verified in the ?debug-sa that everything looks
fine.****
****
I waited and sure enough it happened again today. We get less mail on the
weekends so it took awhile waiting..****
****
I have posted my MailScanner ?debug-sa to pastebin if anyone can take a
look and give me a recommendation of where to look next. I am almost out
of things to try.****
****
Here is one ?debug-sa:****
****
http://pastebin.com/C2XPs7D2****
****
Then I kept running with ?debug-sa till it caught one with a DNS based
rule like URIBL_* rules.****
****
This one hits on those URIBL rules that are DNS based and it looks like
everything is OK as far as I can tell.. This is really the first time I
have tried to debug a debug log from MailScanner/Spamassassin before..****
****
http://pastebin.com/iWMnJqf3****
****
****
Thanks****
****
****
****
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com
****
****
mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Martin Hepworth
*Sent:* Saturday, June 15, 2013 8:46 AM
*To:* MailScanner discussion
*Subject:* Re: Certain Spamassassin rules do not seem to be firing all of
the time****
****
Ok, you really need to put a local DNS server on the MailScanner box,
doesn't matter if the DNS resolver is next to the server in the switch
port, DNS is actually quite heavy on network traffic and hitting this all
the time makes a huge difference. It can forward to the current machine,
but the time this saves is actually quite noticable.****
that three seconds for the pass across seems very quick to me, esp as it's
got all the DNS requests to process.I normally remove most of the RBL's
from being scanned in Spamassassin by giving most of them a zero score (see
50_scores.cf <http://spamassassin.apache.org/dist/rules/50_scores.cf> in
the DNSEval section). also make sure you're updating sa rules regularly. In
fact its almost as if you've got skip-rbl-checks set to 1 in a spamassassin
.cf or mailscanner,conf file somewhere.****
I'd double check all the setup to make sure everythings OK, as it's really
odd that you're getting DNS based hits in test mode but not in test mode.
Check the MailScanner.conf setttings and any site MailScanner.conf, and
also get rid of any .spamassassin dirs esp if there's anything in root's
home dir (so i presume ther MTA is sendmail?) to make sure that isnt
overriding any settings. Check you've got one MailScanner.conf and not
multiple ones, sometimes some distributions put the active file in
'non-standard' places.****
****
--
Martin Hepworth, CISSP
Oxford, UK****
****
On 15 June 2013 03:22, Duncan, Brian M. <brian.duncan at kattenlaw.com>
wrote:****
Thanks, yes I noticed that, they all do seem to be the DNS rules. I do
have a caching DNS server but it is on the local network. I will try and
see if the behavior changes at all by running one locally on the box itself.
****
****
When you say ?that youre not timing out the network checks in sa too
quickly? I have not changed anything in the defaults of Mailscanner or
included any directives that would lower whatever time limits are set by
default.****
****
I took a look at the last example I put on pastebin, and it looks like it
took 3 seconds to go from my Mailscanner box to my next gateway. ****
****
Received: from venus.kattenlaw.com ([10.18.3.33]) by us.kmz.com****
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id
8e3c2381002025b2****
; Fri, 14 Jun 2013 14:01:09 -0500****
Received: from a.loselit.net (a.loselit.net [66.96.254.156]) by****
venus.kattenlaw.com (8.13.8/8.13.4) with ESMTP id r5EJ13oK014449
for****
<brian.duncan at kmzr.com>; Fri, 14 Jun 2013 14:01:06 -0500****
****
I am assuming the 3 seconds going from my incoming mail server Venus, to
the next hop in my environment includes the time it took for the Spammer to
send me the message.****
****
I also don?t see anything in my maillogs related to Spam Assassin timing
out for anything.. I recall many years ago when we used to run systems with
much less CPU power (10+) seeing Spam Assassin time outs.****
****
Which BTW, at the peak of activity today the lowest idle %idle was 91.00
and that is because I turned off caching of SpamAssassin in Mailscanner to
see if that had any impact.****
****
I also looked at the local caching DNS server that is on the same switch
as this box, and it was peaking at like 30 Kilobytes per second on UDP 53
requests from anything that uses it locally according to iptraf.****
****
It also seems to be these messages from the same Spammer, as I said before
if I take any of these message bodies and send them in myself I seem to get
the DNS Spam Assassin hits then. ****
****
Really odd one..****
****
Thanks for your help****
****
****
****
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com
****
****
mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Martin Hepworth
*Sent:* Friday, June 14, 2013 4:16 PM
*To:* MailScanner discussion
*Subject:* Certain Spamassassin rules do not seem to be firing all of the
time****
****
Hmm most if the extra rules youre hitting are dns based****
I'd check youre running a local caching dns server on the scanning box and
that youre not timing out the network checks in sa too quickly****
****
Martin
On Friday, 14 June 2013, Duncan, Brian M. wrote:****
Here is one more that just came in to me and was not tagged as Spam:****
****
http://pastebin.com/w8SJk660****
****
****
Mailscanner/Spamassassin results:****
****
X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
****
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)****
****
****
--test-mode results:****
****
Content analysis details: (10.5 hits, 6.5 required)****
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%****
[score: 1.0000]****
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain****
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level****
above 50%****
[cf: 100]****
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)****
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%****
[cf: 100]****
-7.5 AWL AWL: From: address is in the auto white-list**
**
****
------ End of SpamAssassin results, Original message follows --------****
****
===========================================================****
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue****
Service, any tax advice contained herein is not intended or written to be used and cannot be used****
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.****
===========================================================****
CONFIDENTIALITY NOTICE:****
This electronic mail message and any attached files contain information intended for the exclusive****
use of the individual or entity to whom it is addressed and may contain information that is****
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you****
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or ****
distribution of this information may be subject to legal restriction or sanction. Please notify****
the sender, by electronic mail or telephone, of any unintended recipients and delete the original ****
message without making any copies.****
===========================================================****
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has****
elected to be governed by the Illinois Uniform Partnership Act (1997).****
===========================================================****
--
--
Martin Hepworth, CISSP
Oxford, UK****
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!****
****
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!****
** **
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130619/755f5b52/attachment.html

[Duncan, Brian M.]

I already do that in a sense.. I don't have it call out, but I export all my SMTP aliases from AD and add them to the access file on my sendmail servers and reject all other mail to my domain, so the rest is discarded to non existent users, and it saves with dealing with all the NDR's

Well it looks like I bought myself some time. Even though I have NOT figured out what is going on here, since I disabled auto white listing the other day, it looks like the majority of these Spam messages that were making it through before because they were NOT hitting on these different URIBLS are getting tagged from Bayes hits now. And since the AWL is not factoring into it, 98% of them are getting labeled as Spam.

I am probably just going to rebuild my primary mail server and re-install Mailscanner and Spamassassin in a few weeks and see if this problem goes away.

I still think there is something unique with these particular Spam emails. These messages I am talking about, I have NEVER seen URIBL_BLACK ever fire on. (But does fire on it when I manually scan with spamassassin-test-mode)

Yesterday I had plenty of other emails where it does fire on that rule:

[root at venus log]# cat maillog.1 | grep -i "URIBL_BLACK" | wc -l
2971


BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com / www.kattenlaw.com


From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Wednesday, June 19, 2013 5:30 AM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

maybe you can use sendmail to call-out for valid recipients first, I find this drops HUGE amounts of traffic dead before it gets anywhere near MailScanner, easily 50% and maybe higher

http://wiki.mailscanner.info/doku.php?id=documentation:configuration:mta:sendmail:how_to:reject_non_existent_users

--
Martin Hepworth, CISSP
Oxford, UK

On 18 June 2013 19:40, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
Yeah I know it's very weird and I can't track it down.

Yesterday, I tried removing the NET:DNS perl module (.65 is what MailScanner (and I believe SpamAssassin use by default) and compiling 0.72 in the hopes that it had something to do with that. Nope, still happening today. Fortunately it only seems to be letting a few Spam in overall. It just happens when there is a black listed domain that is used in a URL that is sent by a non-blacklisted gateway where I get caught by this issue.

I am using Sendmail. Yes there is a .spamassassin directory in root, where the bayes db's are located and autowhitelist db's (I have autowhite list disabled for the moment) The user_prefs file has no directives set in it, they are all #'ed out.

I don't specify a run as user in my MailScanner.conf, and according to ps all the MailScanner processes are running as root, and my spamassassin -test-mode I have run as root.

I turned on skip_rbl_checks 1 yesterday, since I detect RBL'ed hosts using MailScanner I figured it was kind of pointless to do it again with SpamAssassin..

I also tried tweaking rbl_timeout to 60 seconds instead of 30, what is was before. Because I did find someone else reporting a similar issue to mine.. back in 2007 someone was reporting this same behavior that rules were not hitting when using Amavis with Spamassassin, but then when you ran them through Spamassassin they worked, and I believe it was the same types of rules I am not hitting on through MailScanner. And the issue wound up being Net:DNS.

http://www.gossamer-threads.com/lists/spamassassin/users/102307?do=post_view_threaded#102307

If I can't figure this out, I might attempt a fresh install of Cent OS 6.4 and fresh install of MailScanner and SpamAssassin. Just don't have the time for a full re-install right now.

Yeah I prefer doing the RBL's with MailScanner because it uses such little CPU to perform those tests. I mark as Spam probably as high as 80% of my incoming mail just on the RBL checks through MailScanner. Spamasssassin only has to process like 20% of my Spam mail then. Since we pass everything through and assign the Microsoft SCL score based on if it failed Spam checks, the users can whitelist if someone winds up on an RBL or they want what most consider Spam.


BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com> / www.kattenlaw.com<http://www.kattenlaw.com>


From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Martin Hepworth
Sent: Tuesday, June 18, 2013 10:09 AM

To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

really odd, seems to be suffering with network based rules, not just the URI ones but razor as well.
Personally I always put all the RBL checks into SA rather than letting MailScanner do it by itself. that way no 1 rbl can false postive and email and the RBL just add to the overall score.

What MTA are you running? and is there a .spamassassin directory in root's home dir?

--
Martin Hepworth, CISSP
Oxford, UK

On 15 June 2013 21:08, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
Thanks for the recommendations Martin.

The way I have it setup in Mailscanner is if the sending mail server is on a RBL, (If it on at least 1 of the 4 RBLS we use with MailScanner) It becomes high scoring spam and is tagged and moved on, it does not get scanned by Spamassassin then.

The one thing I never considered before was if Spamassassin is scanning the same sending mail server IP for being listed when it does not get caught by MailScanner as being on any of the 4 RB:'s I use. Not that it is causing my problem now, but that it is not very efficient if it is doing it again. (I would guess 80% of my mail never gets scanned by SpamAssassin each day because the sending mail gateway is blacklisted and it is marked as Spam and moves on)

When you say: " I normally remove most of the RBL's from being scanned in Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf> in the DNSEval section)."

I don't think I follow, are you saying Spamassassin is scanning the sending mail host again against the RBL's? So by giving them a zero score you are avoiding the double effort? This section has nothing to do with the URL/URI scanning that is happening? I assumed the rules that I have that are NOT hitting when it goes through MailScanner/Spamassassin have all been based on the URI/URL's in the body of the message.


When I take these specific Spam messages that make it into my inbox, I am noticing they never hit on the same URIBL hits I get when I move the message locally to the box, if I take one of these URI based RBL checking rules like for example URIBL_BLACK, I have never seen that rule hit on ANY of these ones making it into my inbox. If I search my maillog from yesterday for every message that wound up being scanned by Spamassassin, I see that there were 1014 times that rule is listed on detected Spam.

Last night I first tried setting up a caching bind server local to the box. Made no difference.

I tried upgrading to MailScanner 4.84.5-3 after and updating to SpamAssassin 3.3.2-1 to see if that would make a difference, I even looked at the Perl modules that come with MailScanner, one of them was perl-Net-DNS-0.65-2, I was running
perl-Net-DNS-0.65-1, was hoping that had something to do with this so I updated to .65-2 of that perl modules.. the rest all seemed to be the same version that comes with 4.84.5-3 (I was running 4.83.5-1 before I updated)

I went over all my configs for both MailScanner and SpamAssassin, nothing seems wrong or set to low that would create the situation I am seeing. I did find I had the pyzor plugin loading in SpamAssassin but no exe, so I just disabled pyzor and verified in the -debug-sa that everything looks fine.

I waited and sure enough it happened again today. We get less mail on the weekends so it took awhile waiting..

I have posted my MailScanner -debug-sa to pastebin if anyone can take a look and give me a recommendation of where to look next. I am almost out of things to try.

Here is one -debug-sa:

http://pastebin.com/C2XPs7D2

Then I kept running with -debug-sa till it caught one with a DNS based rule like URIBL_* rules.

This one hits on those URIBL rules that are DNS based and it looks like everything is OK as far as I can tell.. This is really the first time I have tried to debug a debug log from MailScanner/Spamassassin before..

http://pastebin.com/iWMnJqf3


Thanks



BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com> / www.kattenlaw.com<http://www.kattenlaw.com>


From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Martin Hepworth
Sent: Saturday, June 15, 2013 8:46 AM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

Ok, you really need to put a local DNS server on the MailScanner box, doesn't matter if the DNS resolver is next to the server in the switch port, DNS is actually quite heavy on network traffic and hitting this all the time makes a huge difference. It can forward to the current machine, but the time this saves is actually quite noticable.
that three seconds for the pass across seems very quick to me, esp as it's got all the DNS requests to process.I normally remove most of the RBL's from being scanned in Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf> in the DNSEval section). also make sure you're updating sa rules regularly. In fact its almost as if you've got skip-rbl-checks set to 1 in a spamassassin .cf or mailscanner,conf file somewhere.
I'd double check all the setup to make sure everythings OK, as it's really odd that you're getting DNS based hits in test mode but not in test mode. Check the MailScanner.conf setttings and any site MailScanner.conf, and also get rid of any .spamassassin dirs esp if there's anything in root's home dir (so i presume ther MTA is sendmail?) to make sure that isnt overriding any settings. Check you've got one MailScanner.conf and not multiple ones, sometimes some distributions put the active file in 'non-standard' places.

--
Martin Hepworth, CISSP
Oxford, UK

On 15 June 2013 03:22, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
Thanks, yes I noticed that, they all do seem to be the DNS rules. I do have a caching DNS server but it is on the local network. I will try and see if the behavior changes at all by running one locally on the box itself.

When you say "that youre not timing out the network checks in sa too quickly" I have not changed anything in the defaults of Mailscanner or included any directives that would lower whatever time limits are set by default.

I took a look at the last example I put on pastebin, and it looks like it took 3 seconds to go from my Mailscanner box to my next gateway.

Received: from venus.kattenlaw.com<http://venus.kattenlaw.com> ([10.18.3.33]) by us.kmz.com<http://us.kmz.com>
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id 8e3c2381002025b2
; Fri, 14 Jun 2013 14:01:09 -0500
Received: from a.loselit.net<http://a.loselit.net> (a.loselit.net<http://a.loselit.net> [66.96.254.156]) by
venus.kattenlaw.com<http://venus.kattenlaw.com> (8.13.8/8.13.4) with ESMTP id r5EJ13oK014449 for
<brian.duncan at kmzr.com<mailto:brian.duncan at kmzr.com>>; Fri, 14 Jun 2013 14:01:06 -0500

I am assuming the 3 seconds going from my incoming mail server Venus, to the next hop in my environment includes the time it took for the Spammer to send me the message.

I also don't see anything in my maillogs related to Spam Assassin timing out for anything.. I recall many years ago when we used to run systems with much less CPU power (10+) seeing Spam Assassin time outs.

Which BTW, at the peak of activity today the lowest idle %idle was 91.00 and that is because I turned off caching of SpamAssassin in Mailscanner to see if that had any impact.

I also looked at the local caching DNS server that is on the same switch as this box, and it was peaking at like 30 Kilobytes per second on UDP 53 requests from anything that uses it locally according to iptraf.

It also seems to be these messages from the same Spammer, as I said before if I take any of these message bodies and send them in myself I seem to get the DNS Spam Assassin hits then.

Really odd one..

Thanks for your help



BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com> / www.kattenlaw.com<http://www.kattenlaw.com>


From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Martin Hepworth
Sent: Friday, June 14, 2013 4:16 PM
To: MailScanner discussion
Subject: Certain Spamassassin rules do not seem to be firing all of the time

Hmm most if the extra rules youre hitting are dns based
I'd check youre running a local caching dns server on the scanning box and that youre not timing out the network checks in sa too quickly

Martin

On Friday, 14 June 2013, Duncan, Brian M. wrote:
Here is one more that just came in to me and was not tagged as Spam:

http://pastebin.com/w8SJk660


Mailscanner/Spamassassin results:

X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
BAYES_60 3.00, RP_MATCHES_RCVD -0.00)


--test-mode results:

Content analysis details: (10.5 hits, 6.5 required)
6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%
[score: 1.0000]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
-7.5 AWL AWL: From: address is in the auto white-list

------ End of SpamAssassin results, Original message follows --------


===========================================================

CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue

Service, any tax advice contained herein is not intended or written to be used and cannot be used

by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.

===========================================================

CONFIDENTIALITY NOTICE:

This electronic mail message and any attached files contain information intended for the exclusive

use of the individual or entity to whom it is addressed and may contain information that is

proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you

are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or

distribution of this information may be subject to legal restriction or sanction. Please notify

the sender, by electronic mail or telephone, of any unintended recipients and delete the original

message without making any copies.

===========================================================

NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has

elected to be governed by the Illinois Uniform Partnership Act (1997).

===========================================================



--
--
Martin Hepworth, CISSP
Oxford, UK

--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130620/d15749a0/attachment.html

[Martin Hepworth]

I find awl not very good when used in multiuser configs.. May be better in
a user soecific env but never works very well for me using a standard ms
setup

I already do that in a sense.. I don?t have it call out, but I export
all my SMTP aliases from AD and add them to the access file on my sendmail
servers and reject all other mail to my domain, so the rest is discarded to
non existent users, and it saves with dealing with all the NDR?s****
** **
Well it looks like I bought myself some time. Even though I have NOT
figured out what is going on here, since I disabled auto white listing the
other day, it looks like the majority of these Spam messages that were
making it through before because they were NOT hitting on these different
URIBLS are getting tagged from Bayes hits now. And since the AWL is not
factoring into it, 98% of them are getting labeled as Spam.****
** **
I am probably just going to rebuild my primary mail server and re-install
Mailscanner and Spamassassin in a few weeks and see if this problem goes
away. ****
** **
I still think there is something unique with these particular Spam
emails. These messages I am talking about, I have NEVER seen URIBL_BLACK
ever fire on. (But does fire on it when I manually scan with
spamassassin?test-mode)****
** **
Yesterday I had plenty of other emails where it does fire on that rule:***
*
** **
[root at venus log]# cat maillog.1 | grep -i "URIBL_BLACK" | wc -l****
2971****
** **
** **
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / [phone] f / [phone]
brian.duncan at kattenlaw.com <javascript:_e({}, 'cvml',
'brian.duncan at kattenlaw.com');> / www.kattenlaw.com
****
** **
*From:* mailscanner-bounces at lists.mailscanner.info <javascript:_e({},
mailscanner-bounces at lists.mailscanner.info <javascript:_e({}, 'cvml',
'mailscanner-bounces at lists.mailscanner.info');>] *On Behalf Of *Martin
Hepworth
*Sent:* Wednesday, June 19, 2013 5:30 AM
*To:* MailScanner discussion
*Subject:* Re: Certain Spamassassin rules do not seem to be firing all of
the time****
** **
maybe you can use sendmail to call-out for valid recipients first, I find
this drops HUGE amounts of traffic dead before it gets anywhere near
MailScanner, easily 50% and maybe higher
http://wiki.mailscanner.info/doku.php?id=documentation:configuration:mta:sendmail:how_to:reject_non_existent_users
****
****
--
Martin Hepworth, CISSP
Oxford, UK****
** **
On 18 June 2013 19:40, Duncan, Brian M. <brian.duncan at kattenlaw.com>
wrote:****
Yeah I know it?s very weird and I can?t track it down. ****
****
Yesterday, I tried removing the NET:DNS perl module (.65 is what
MailScanner (and I believe SpamAssassin use by default) and compiling 0.72
in the hopes that it had something to do with that. Nope, still happening
today. Fortunately it only seems to be letting a few Spam in overall.
It just happens when there is a black listed domain that is used in a
URL that is sent by a non-blacklisted gateway where I get caught by this
issue.****
****
I am using Sendmail. Yes there is a .spamassassin directory in root, where
the bayes db?s are located and autowhitelist db?s (I have autowhite list
disabled for the moment) The user_prefs file has no directives set in it,
they are all #?ed out.****
****
I don?t specify a run as user in my MailScanner.conf, and according to ps
all the MailScanner processes are running as root, and my spamassassin
?test-mode I have run as root.****
****
I turned on skip_rbl_checks 1 yesterday, since I detect RBL?ed hosts using
MailScanner I figured it was kind of pointless to do it again with
SpamAssassin.. ****
****
I also tried tweaking rbl_timeout to 60 seconds instead of 30, what is was
before. Because I did find someone else reporting a similar issue to
mine.. back in 2007 someone was reporting this same behavior that rules
were not hitting when using Amavis with Spamassassin, but then when you ran
them through Spamassassin they worked, and I believe it was the same types
of rules I am not hitting on through MailScanner. And the issue wound up
being Net:DNS.****
****
http://www.gossamer-threads.com/lists/spamassassin/users/102307?do=post_view_threaded#102307
****
****
If I can?t figure this out, I might attempt a fresh install of Cent OS 6.4
and fresh install of Ma