Discussion:
Has anyone seen when the load on a MailScanner server
Philip Parsons
2014-05-28 01:37:56 UTC
Permalink
Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If you have seen this is there something we can change that changes the process ?

Thank you
P Parsons
Joolee
2014-05-28 06:57:04 UTC
Permalink
You probably hit a timeout somewhere. Can you find out from the logs which
component times out?

On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com> wrote:

> Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If
> you have seen this is there something we can change that changes the
> process ?
>
> Thank you
> P Parsons
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140528/5d398c3a/attachment.html
Jerry Benton
2014-05-28 07:46:53 UTC
Permalink
What is happening is your queue is exceeding the threshold set (800 by
default I think) where MailScanner just starts to process the email without
scanning it. If your server is getting hit by a surge of email and
MailScanner cannot keep up, it stops doing checks.

Of course, I could be wrong. Happens a lot.


On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl> wrote:

> You probably hit a timeout somewhere. Can you find out from the logs which
> component times out?
>
>
> On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com> wrote:
>
>> Gets too high like 5 or 6 that MailScanner marks all messages as spam ?
>> If you have seen this is there something we can change that changes the
>> process ?
>>
>> Thank you
>> P Parsons
>> --
>> MailScanner mailing list
>> mailscanner at lists.mailscanner.info
>> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>>
>> Before posting, read http://wiki.mailscanner.info/posting
>>
>> Support MailScanner development - buy the book off the website!
>>
>
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
>


--

--
Jerry Benton
Mailborder Systems
www.mailborder.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140528/e348e9d2/attachment.html
Denis Beauchemin
2014-05-28 12:20:27 UTC
Permalink
I believe you are wrong. When there are too many emails in the inqueue, MS will still check the email contents it will just not check the mails in the order they were received. It will pick the first N mails it sees in the queue whether they just arrived or have been sitting there for some time. Usually MS picks the oldest mails first.

Denis

De : mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] De la part de Jerry Benton
Envoy? : 28 mai 2014 03:54
? : MailScanner discussion
Objet : Re: Has anyone seen when the load on a MailScanner server

What is happening is your queue is exceeding the threshold set (800 by default I think) where MailScanner just starts to process the email without scanning it. If your server is getting hit by a surge of email and MailScanner cannot keep up, it stops doing checks.
Of course, I could be wrong. Happens a lot.

On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl<mailto:mailscanner at joolee.nl>> wrote:
You probably hit a timeout somewhere. Can you find out from the logs which component times out?

On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com<mailto:pparsons at techeez.com>> wrote:
Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If you have seen this is there something we can change that changes the process ?

Thank you
P Parsons
--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140528/d8891feb/attachment.html
Richard Mealing
2014-05-28 12:22:23 UTC
Permalink
I?m not sure that is correct, but I may be wrong too!

# If more messages are found in the queue than this, then switch to an
# "accelerated" mode of processing messages. This will cause it to stop
# scanning messages in strict date order, but in the order it finds them
# in the queue. If your queue is bigger than this size a lot of the time,
# then some messages could be greatly delayed. So treat this option as
# "in emergency only".
Max Normal Queue Size = 2000

So it just stops in the date order and goes for what is top of the queue. I set mine to 2000 but it never reaches that now I have made all my changes.

You should look into tmpfs, it has brought down the load on my servers considerably. I use it for a few things now ?

Incoming Work Dir = /tmpfs
SpamAssassin Cache Database File = /tmpfs/SpamAssassin.cache.db
SpamAssassin Temporary Dir = /tmpfs

Maybe run some gstat commands and check your disks are behaving. If not, then implement tmpfs.

Thanks!
Rich

From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Jerry Benton
Sent: 28 May 2014 08:47
To: MailScanner discussion
Subject: Re: Has anyone seen when the load on a MailScanner server

What is happening is your queue is exceeding the threshold set (800 by default I think) where MailScanner just starts to process the email without scanning it. If your server is getting hit by a surge of email and MailScanner cannot keep up, it stops doing checks.
Of course, I could be wrong. Happens a lot.

On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl<mailto:mailscanner at joolee.nl>> wrote:
You probably hit a timeout somewhere. Can you find out from the logs which component times out?

On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com<mailto:pparsons at techeez.com>> wrote:
Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If you have seen this is there something we can change that changes the process ?

Thank you
P Parsons
--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140528/b662da2e/attachment.html
Jerry Benton
2014-05-28 13:39:16 UTC
Permalink
See ... my wife was right. I am wrong all the time.

On the same note, I am seeing similar behavior on a lab server I just built
testing out Ubuntu 14.04. I will email results when (if) I have them.


On Wed, May 28, 2014 at 2:22 PM, Richard Mealing <richard at fastnet.co.uk>wrote:

> I?m not sure that is correct, but I may be wrong too!
>
>
>
> # If more messages are found in the queue than this, then switch to an
>
> # "accelerated" mode of processing messages. This will cause it to stop
>
> # scanning messages in strict date order, but in the order it finds them
>
> # in the queue. If your queue is bigger than this size a lot of the time,
>
> # then some messages could be greatly delayed. So treat this option as
>
> # "in emergency only".
>
> Max Normal Queue Size = 2000
>
>
>
> So it just stops in the date order and goes for what is top of the queue.
> I set mine to 2000 but it never reaches that now I have made all my changes.
>
>
>
> You should look into tmpfs, it has brought down the load on my servers
> considerably. I use it for a few things now ?
>
>
>
> Incoming Work Dir = /tmpfs
>
> SpamAssassin Cache Database File = /tmpfs/SpamAssassin.cache.db
>
> SpamAssassin Temporary Dir = /tmpfs
>
>
>
> Maybe run some gstat commands and check your disks are behaving. If not,
> then implement tmpfs.
>
>
>
> Thanks!
>
> Rich
>
>
>
> *From:* mailscanner-bounces at lists.mailscanner.info [mailto:
> mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Jerry Benton
> *Sent:* 28 May 2014 08:47
> *To:* MailScanner discussion
> *Subject:* Re: Has anyone seen when the load on a MailScanner server
>
>
>
> What is happening is your queue is exceeding the threshold set (800 by
> default I think) where MailScanner just starts to process the email without
> scanning it. If your server is getting hit by a surge of email and
> MailScanner cannot keep up, it stops doing checks.
>
> Of course, I could be wrong. Happens a lot.
>
>
>
> On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl> wrote:
>
> You probably hit a timeout somewhere. Can you find out from the logs which
> component times out?
>
>
>
> On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com> wrote:
>
> Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If
> you have seen this is there something we can change that changes the
> process ?
>
> Thank you
> P Parsons
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
>
>
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
>
>
>
> --
>
>
> --
>
> Jerry Benton
>
> Mailborder Systems
> www.mailborder.com
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
>


--

--
Jerry Benton
Mailborder Systems
www.mailborder.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140528/0c30b6b2/attachment.html
Richard Mealing
2014-05-28 14:33:12 UTC
Permalink
Hi Jerry,

When it happens, try deleting the Processing.db and tail the logs. You don?t even need to restart mailscanner. See if that fixes the problem.
I?m curious to see your results..

Thanks,
Rich

From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Jerry Benton
Sent: 28 May 2014 14:39
To: MailScanner discussion
Subject: Re: Has anyone seen when the load on a MailScanner server

See ... my wife was right. I am wrong all the time.
On the same note, I am seeing similar behavior on a lab server I just built testing out Ubuntu 14.04. I will email results when (if) I have them.

On Wed, May 28, 2014 at 2:22 PM, Richard Mealing <richard at fastnet.co.uk<mailto:richard at fastnet.co.uk>> wrote:
I?m not sure that is correct, but I may be wrong too!

# If more messages are found in the queue than this, then switch to an
# "accelerated" mode of processing messages. This will cause it to stop
# scanning messages in strict date order, but in the order it finds them
# in the queue. If your queue is bigger than this size a lot of the time,
# then some messages could be greatly delayed. So treat this option as
# "in emergency only".
Max Normal Queue Size = 2000

So it just stops in the date order and goes for what is top of the queue. I set mine to 2000 but it never reaches that now I have made all my changes.

You should look into tmpfs, it has brought down the load on my servers considerably. I use it for a few things now ?

Incoming Work Dir = /tmpfs
SpamAssassin Cache Database File = /tmpfs/SpamAssassin.cache.db
SpamAssassin Temporary Dir = /tmpfs

Maybe run some gstat commands and check your disks are behaving. If not, then implement tmpfs.

Thanks!
Rich

From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Jerry Benton
Sent: 28 May 2014 08:47
To: MailScanner discussion
Subject: Re: Has anyone seen when the load on a MailScanner server

What is happening is your queue is exceeding the threshold set (800 by default I think) where MailScanner just starts to process the email without scanning it. If your server is getting hit by a surge of email and MailScanner cannot keep up, it stops doing checks.
Of course, I could be wrong. Happens a lot.

On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl<mailto:mailscanner at joolee.nl>> wrote:
You probably hit a timeout somewhere. Can you find out from the logs which component times out?

On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com<mailto:pparsons at techeez.com>> wrote:
Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If you have seen this is there something we can change that changes the process ?

Thank you
P Parsons
--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>

--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140528/bc3e4100/attachment-0001.html
Philip Parsons
2014-05-28 15:11:20 UTC
Permalink
I will look into the queue size setting. But this does not seem to be the issue. What happened is all the emails every single one got marked as spam so where not delivered even the white listed ones.

Thank you
P Parsons

On May 28, 2014, at 7:51 AM, "Jerry Benton" <jerry.benton at mailborder.com<mailto:jerry.benton at mailborder.com>> wrote:

See ... my wife was right. I am wrong all the time.

On the same note, I am seeing similar behavior on a lab server I just built testing out Ubuntu 14.04. I will email results when (if) I have them.


On Wed, May 28, 2014 at 2:22 PM, Richard Mealing <richard at fastnet.co.uk<mailto:richard at fastnet.co.uk>> wrote:
I?m not sure that is correct, but I may be wrong too!

# If more messages are found in the queue than this, then switch to an
# "accelerated" mode of processing messages. This will cause it to stop
# scanning messages in strict date order, but in the order it finds them
# in the queue. If your queue is bigger than this size a lot of the time,
# then some messages could be greatly delayed. So treat this option as
# "in emergency only".
Max Normal Queue Size = 2000

So it just stops in the date order and goes for what is top of the queue. I set mine to 2000 but it never reaches that now I have made all my changes.

You should look into tmpfs, it has brought down the load on my servers considerably. I use it for a few things now ?

Incoming Work Dir = /tmpfs
SpamAssassin Cache Database File = /tmpfs/SpamAssassin.cache.db
SpamAssassin Temporary Dir = /tmpfs

Maybe run some gstat commands and check your disks are behaving. If not, then implement tmpfs.

Thanks!
Rich

From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Jerry Benton
Sent: 28 May 2014 08:47
To: MailScanner discussion
Subject: Re: Has anyone seen when the load on a MailScanner server

What is happening is your queue is exceeding the threshold set (800 by default I think) where MailScanner just starts to process the email without scanning it. If your server is getting hit by a surge of email and MailScanner cannot keep up, it stops doing checks.
Of course, I could be wrong. Happens a lot.

On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl<mailto:mailscanner at joolee.nl>> wrote:
You probably hit a timeout somewhere. Can you find out from the logs which component times out?

On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com<mailto:pparsons at techeez.com>> wrote:
Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If you have seen this is there something we can change that changes the process ?

Thank you
P Parsons
--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>

--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!




--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>

--
This message has been scanned for viruses and
dangerous content by MailScanner<http://www.mailscanner.info/>, and is
believed to be clean.
--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140528/71cb01a0/attachment.html
Martin Hepworth
2014-05-28 16:08:07 UTC
Permalink
why did they get marked as spam? what rules hit?

--
Martin Hepworth, CISSP
Oxford, UK


On 28 May 2014 16:11, Philip Parsons <pparsons at techeez.com> wrote:

> I will look into the queue size setting. But this does not seem to be
> the issue. What happened is all the emails every single one got marked as
> spam so where not delivered even the white listed ones.
>
> Thank you
> P Parsons
>
> On May 28, 2014, at 7:51 AM, "Jerry Benton" <jerry.benton at mailborder.com>
> wrote:
>
> See ... my wife was right. I am wrong all the time.
>
> On the same note, I am seeing similar behavior on a lab server I just
> built testing out Ubuntu 14.04. I will email results when (if) I have them.
>
>
> On Wed, May 28, 2014 at 2:22 PM, Richard Mealing <richard at fastnet.co.uk>wrote:
>
>> I?m not sure that is correct, but I may be wrong too!
>>
>>
>>
>> # If more messages are found in the queue than this, then switch to an
>>
>> # "accelerated" mode of processing messages. This will cause it to stop
>>
>> # scanning messages in strict date order, but in the order it finds them
>>
>> # in the queue. If your queue is bigger than this size a lot of the time,
>>
>> # then some messages could be greatly delayed. So treat this option as
>>
>> # "in emergency only".
>>
>> Max Normal Queue Size = 2000
>>
>>
>>
>> So it just stops in the date order and goes for what is top of the queue.
>> I set mine to 2000 but it never reaches that now I have made all my changes.
>>
>>
>>
>> You should look into tmpfs, it has brought down the load on my servers
>> considerably. I use it for a few things now ?
>>
>>
>>
>> Incoming Work Dir = /tmpfs
>>
>> SpamAssassin Cache Database File = /tmpfs/SpamAssassin.cache.db
>>
>> SpamAssassin Temporary Dir = /tmpfs
>>
>>
>>
>> Maybe run some gstat commands and check your disks are behaving. If not,
>> then implement tmpfs.
>>
>>
>>
>> Thanks!
>>
>> Rich
>>
>>
>>
>> *From:* mailscanner-bounces at lists.mailscanner.info [mailto:
>> mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Jerry Benton
>> *Sent:* 28 May 2014 08:47
>> *To:* MailScanner discussion
>> *Subject:* Re: Has anyone seen when the load on a MailScanner server
>>
>>
>>
>> What is happening is your queue is exceeding the threshold set (800 by
>> default I think) where MailScanner just starts to process the email without
>> scanning it. If your server is getting hit by a surge of email and
>> MailScanner cannot keep up, it stops doing checks.
>>
>> Of course, I could be wrong. Happens a lot.
>>
>>
>>
>> On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl> wrote:
>>
>> You probably hit a timeout somewhere. Can you find out from the logs
>> which component times out?
>>
>>
>>
>> On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com> wrote:
>>
>> Gets too high like 5 or 6 that MailScanner marks all messages as spam ?
>> If you have seen this is there something we can change that changes the
>> process ?
>>
>> Thank you
>> P Parsons
>> --
>> MailScanner mailing list
>> mailscanner at lists.mailscanner.info
>> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>>
>> Before posting, read http://wiki.mailscanner.info/posting
>>
>> Support MailScanner development - buy the book off the website!
>>
>>
>>
>>
>> --
>> MailScanner mailing list
>> mailscanner at lists.mailscanner.info
>> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>>
>> Before posting, read http://wiki.mailscanner.info/posting
>>
>> Support MailScanner development - buy the book off the website!
>>
>>
>>
>>
>> --
>>
>>
>> --
>>
>> Jerry Benton
>>
>> Mailborder Systems
>> www.mailborder.com
>>
>> --
>> MailScanner mailing list
>> mailscanner at lists.mailscanner.info
>> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>>
>> Before posting, read http://wiki.mailscanner.info/posting
>>
>> Support MailScanner development - buy the book off the website!
>>
>>
>
>
> --
>
> --
> Jerry Benton
> Mailborder Systems
> www.mailborder.com
>
> --
> This message has been scanned for viruses and
> dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
> believed to be clean.
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140528/63e66f27/attachment.html
Philip Parsons
2014-05-29 22:54:10 UTC
Permalink
So digging further into this I have found that it is at the same time there are a bunch of these messages in the logs..

May 23 15:06:15 MailScanner[4971]: Commercial scanner clamav timed out!
May 23 15:06:15 MailScanner[4971]: clamav: Failed to complete, timed out
May 23 15:06:15 MailScanner[4971]: Virus Scanning: Denial Of Service attack is in message s4NLTn3v008504

I do not know of a section in Mailscanner.conf related to that ?




From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Jerry Benton
Sent: May-28-14 6:39 AM
To: MailScanner discussion
Subject: Re: Has anyone seen when the load on a MailScanner server

See ... my wife was right. I am wrong all the time.
On the same note, I am seeing similar behavior on a lab server I just built testing out Ubuntu 14.04. I will email results when (if) I have them.

On Wed, May 28, 2014 at 2:22 PM, Richard Mealing <richard at fastnet.co.uk<mailto:richard at fastnet.co.uk>> wrote:
I?m not sure that is correct, but I may be wrong too!

# If more messages are found in the queue than this, then switch to an
# "accelerated" mode of processing messages. This will cause it to stop
# scanning messages in strict date order, but in the order it finds them
# in the queue. If your queue is bigger than this size a lot of the time,
# then some messages could be greatly delayed. So treat this option as
# "in emergency only".
Max Normal Queue Size = 2000

So it just stops in the date order and goes for what is top of the queue. I set mine to 2000 but it never reaches that now I have made all my changes.

You should look into tmpfs, it has brought down the load on my servers considerably. I use it for a few things now ?

Incoming Work Dir = /tmpfs
SpamAssassin Cache Database File = /tmpfs/SpamAssassin.cache.db
SpamAssassin Temporary Dir = /tmpfs

Maybe run some gstat commands and check your disks are behaving. If not, then implement tmpfs.

Thanks!
Rich

From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Jerry Benton
Sent: 28 May 2014 08:47
To: MailScanner discussion
Subject: Re: Has anyone seen when the load on a MailScanner server

What is happening is your queue is exceeding the threshold set (800 by default I think) where MailScanner just starts to process the email without scanning it. If your server is getting hit by a surge of email and MailScanner cannot keep up, it stops doing checks.
Of course, I could be wrong. Happens a lot.

On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl<mailto:mailscanner at joolee.nl>> wrote:
You probably hit a timeout somewhere. Can you find out from the logs which component times out?

On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com<mailto:pparsons at techeez.com>> wrote:
Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If you have seen this is there something we can change that changes the process ?

Thank you
P Parsons
--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>

--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>

--
This message has been scanned for viruses and
dangerous content by MailScanner<http://www.mailscanner.info/>, and is
believed to be clean.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140529/4c8c7a1e/attachment.html
Richard Mealing
2014-05-30 09:49:53 UTC
Permalink
It sounds like you have some high load on your server. Have you looked at your disk i/o activity?

I was getting the same thing with mine some time ago so I had to make some drastic changes to bring the load down.
You should look at things before the mailscanner process, such as RBL?s on your MTA, fail2ban to block malicious senders (fantastic program btw), moving your tmp directories to RAM disk, or adding more servers to compensate for the load.
Thanks,
Rich

From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Philip Parsons
Sent: 29 May 2014 23:54
To: MailScanner discussion
Subject: RE: Has anyone seen when the load on a MailScanner server

So digging further into this I have found that it is at the same time there are a bunch of these messages in the logs..

May 23 15:06:15 MailScanner[4971]: Commercial scanner clamav timed out!
May 23 15:06:15 MailScanner[4971]: clamav: Failed to complete, timed out
May 23 15:06:15 MailScanner[4971]: Virus Scanning: Denial Of Service attack is in message s4NLTn3v008504

I do not know of a section in Mailscanner.conf related to that ?




From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Jerry Benton
Sent: May-28-14 6:39 AM
To: MailScanner discussion
Subject: Re: Has anyone seen when the load on a MailScanner server

See ... my wife was right. I am wrong all the time.
On the same note, I am seeing similar behavior on a lab server I just built testing out Ubuntu 14.04. I will email results when (if) I have them.

On Wed, May 28, 2014 at 2:22 PM, Richard Mealing <richard at fastnet.co.uk<mailto:richard at fastnet.co.uk>> wrote:
I?m not sure that is correct, but I may be wrong too!

# If more messages are found in the queue than this, then switch to an
# "accelerated" mode of processing messages. This will cause it to stop
# scanning messages in strict date order, but in the order it finds them
# in the queue. If your queue is bigger than this size a lot of the time,
# then some messages could be greatly delayed. So treat this option as
# "in emergency only".
Max Normal Queue Size = 2000

So it just stops in the date order and goes for what is top of the queue. I set mine to 2000 but it never reaches that now I have made all my changes.

You should look into tmpfs, it has brought down the load on my servers considerably. I use it for a few things now ?

Incoming Work Dir = /tmpfs
SpamAssassin Cache Database File = /tmpfs/SpamAssassin.cache.db
SpamAssassin Temporary Dir = /tmpfs

Maybe run some gstat commands and check your disks are behaving. If not, then implement tmpfs.

Thanks!
Rich

From: mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info> [mailto:mailscanner-bounces at lists.mailscanner.info<mailto:mailscanner-bounces at lists.mailscanner.info>] On Behalf Of Jerry Benton
Sent: 28 May 2014 08:47
To: MailScanner discussion
Subject: Re: Has anyone seen when the load on a MailScanner server

What is happening is your queue is exceeding the threshold set (800 by default I think) where MailScanner just starts to process the email without scanning it. If your server is getting hit by a surge of email and MailScanner cannot keep up, it stops doing checks.
Of course, I could be wrong. Happens a lot.

On Wed, May 28, 2014 at 8:57 AM, Joolee <mailscanner at joolee.nl<mailto:mailscanner at joolee.nl>> wrote:
You probably hit a timeout somewhere. Can you find out from the logs which component times out?

On 28 May 2014 03:37, Philip Parsons <pparsons at techeez.com<mailto:pparsons at techeez.com>> wrote:
Gets too high like 5 or 6 that MailScanner marks all messages as spam ? If you have seen this is there something we can change that changes the process ?

Thank you
P Parsons
--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>

--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!



--

--
Jerry Benton
Mailborder Systems
www.mailborder.com<http://www.mailborder.com>

--
This message has been scanned for viruses and
dangerous content by MailScanner<http://www.mailscanner.info/>, and is
believed to be clean.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140530/331150f8/attachment.html
Kai Schaetzl
2014-05-30 09:52:06 UTC
Permalink
Philip Parsons wrote on Thu, 29 May 2014 22:54:10 +0000:

> I do not know of a section in Mailscanner.conf related to that ?

https://www.google.de/search?
hl=de&as_q=&as_epq=Denial+Of+Service+attack+is+in+message

Kai

--
Get your web at Conactive Internet Services: http://www.conactive.com
Valentin Laskov
2014-05-30 11:46:54 UTC
Permalink
----- Original Message -----
From: "Philip Parsons" <pparsons at techeez.com>
Sent: Friday, May 30, 2014 1:54 AM


|
| May 23 15:06:15 MailScanner[4971]: Commercial scanner clamav timed out!
| May 23 15:06:15 MailScanner[4971]: clamav: Failed to complete, timed out
| May 23 15:06:15 MailScanner[4971]: Virus Scanning: Denial Of Service attack is in message s4NLTn3v008504
|

First, it's better to configure MailScanner to use clamd (and start clamd :) ) . I have

May 28 14:29:04 mail MailScanner[9271]: ClamD Timed Out During PING Check!
May 28 14:29:04 mail MailScanner[9275]: ClamD Timed Out During PING Check!
May 28 14:29:12 mail MailScanner[8908]: Clamd::ERROR:: CLAM PING TIMED OUT! :: .
May 28 14:29:14 mail MailScanner[8764]: Clamd::ERROR:: CLAM PING TIMED OUT! :: .
and
May 28 14:30:05 mail MailScanner[8764]: Virus Scanning: No virus scanners worked, so message batch was abandoned and re-tried!

This happens just after FreshClam updated virus database and clamd loads new base. Loading virus database takes a while.

Regards
Valentin
Loading...