storing messages

Hello Sandro,

Post by Alessandro Dentella
Hi,
I have several servers where storing messagges just works.
Now i'm fighting to get spam and nonspanm messages stored in a new server
that was not preared by me but seems correct, nevertheless I cant get
messages stored.
Spam Actions = store
High Scoring Spam Actions = store
Spam Actions = store deliver header "
Non Spam Actions = store deliver header "X-Spam-Status: No"

You do reaklize that you have quoted two different "Spam Actions", and
that the secon one is faulty? It should read something like:
Spam Actions = store deliver header "X-Spam-Status: Yes"

... If you do a "MailScanner --lint", that should point you in the
right direction.

Post by Alessandro Dentella
Quarantine dir is /var/spool/MailScanner/quarantine/ and permissions are
postfix.www-data
root at smtp:/etc/MailScanner# find /var/spool/MailScanner/quarantine/ -ls
262148 4 drwxrwxr-x 3 postfix www-data 4096 May 22 06:25 /var/spool/MailScanner/quarantine/
262166 4 drwxrwx--- 4 postfix www-data 4096 May 22 06:53 /var/spool/MailScanner/quarantine/20130522
262167 4 drwxrwx--- 2 postfix www-data 4096 May 22 11:37 /var/spool/MailScanner/quarantine/20130522/nonspam
262168 4 drwxrwx--- 2 postfix www-data 4096 May 22 11:38 /var/spool/MailScanner/quarantine/20130522/spam

The above permissions look OK, provided that the corresponding
settings in MailScanner.conf are OK... For me (my web server user is
apache, not www-data) it looks like:
Quarantine User = postfix
Quarantine Group = apache
Quarantine Permissions = 0660

Post by Alessandro Dentella
What should I check to understand why mailscanner is not storing them?

Start by running as the postfix user, and try access the quarantine
directory. Do something like:
su - postfix -s /bin/bash
cd /var/spool/MailScanner/quarantine
....
If that works, fine... But my money is on there being a "permission
problem higher up the filesystem hierarchy", or a misconfig in
MailScanner.conf.

Post by Alessandro Dentella
ii mailscanner 4.84.5-4~squeeze
ii postfix 2.7.1-1+squeeze1
Thanks in advanced
sandro
*:-)

Cheers!

--
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se

Alessandro Dentella

2013-05-23 10:04:21 UTC

Hello Glen,

Post by Alessandro Dentella
Spam Actions = store
High Scoring Spam Actions = store
Spam Actions = store deliver header "
Non Spam Actions = store deliver header "X-Spam-Status: No"

You do reaklize that you have quoted two different "Spam Actions", and
Spam Actions = store deliver header "X-Spam-Status: Yes"

Sorry, in fact I have 2 files in conf.d that resulted in 2 directives, then
I deleted part of the second one while posting. Now I just have the correct
one:

Spam Actions = store deliver header "X-Spam-Status: Yes"

Post by Glenn Steen
Start by running as the postfix user, and try access the quarantine
su - postfix -s /bin/bash
cd /var/spool/MailScanner/quarantine

root at smtp:/etc/MailScanner# su - postfix -s /bin/bash
postfix at smtp:~$ cd /var/spool/MailScanner/quarantine/
postfix at smtp:/var/spool/MailScanner/quarantine$ mkdir test
postfix at smtp:/var/spool/MailScanner/quarantine$ ls -ld test
drwxr-xr-x 2 postfix postfix 4096 May 23 11:47 test
postfix at smtp:/var/spool/MailScanner/quarantine$ rmdir test

It just looks ok to me...

BTW, I have the same problem with spam and nonspam, no message in the logs...

sandro
*:-(

Alessandro Dentella

2013-05-23 10:40:10 UTC

Post by Glenn Steen
... If you do a "MailScanner --lint", that should point you in the
right direction.

True... I run it and it finds:

Could not open file >/var/spool/MailScanner/incoming/28403/1.header: Permission denied
Cannot create + lock headers file /var/spool/MailScanner/incoming/28403/1.header, Permission denied at /usr/share/MailScanner/MailScanner/Message.pm line 523

In fact postfix does not have permission to write there. I fixed it and it
turns out as a stupid conf problem (Quarantine User = user)

Now MailScanner --lint doesn't show any other problem, but still messages
doesn't get into quarantine...

Any thoughts?

TIA
sandro
*:-)

Glenn Steen

2013-05-23 13:01:59 UTC

Post by Glenn Steen
... If you do a "MailScanner --lint", that should point you in the
right direction.

Could not open file >/var/spool/MailScanner/incoming/28403/1.header: Permission denied
Cannot create + lock headers file /var/spool/MailScanner/incoming/28403/1.header, Permission denied at /usr/share/MailScanner/MailScanner/Message.pm line 523
In fact postfix does not have permission to write there. I fixed it and it
turns out as a stupid conf problem (Quarantine User = user)
Now MailScanner --lint doesn't show any other problem, but still messages
doesn't get into quarantine...
Any thoughts?

Ok, so now we don't have any syntax errors, That's good:-).
Next over to semantics... Best is to do a debug run (this is described
in the MAQ/wiki)... Simple steps:
stop mailscanner via the init script ("service MailScanner stop", or
"/etc/init.d/MailScanner stop")

Start postfix/you MTA ... In the default MailScanner init script
there's provision for this:
service MailScanner startin
or
/etc/init.d/MailScanner startin

start the debug run via "MailScanner --debug". This will start
MailScanner without forking any children and without closing
stdin/stderr... And it will wait for exactly 1 message (or rather ...
one batch), process it and then exit... whilst spewing a bit of debug
info onto the screen.
Best is to run that as the postfix user (even though it should work
perfectly well from root... you could do two runs, one from root, one
from postfix.. The process should change user to whatever you have the
"Run User" set to... ie postfix:-).
After a bit of chatter, it'll hang, waiting for a messagebatch...
Which you need provide via normal SMTP methods.

We'll see what that gives you.

Cheers!
--
-- Glenn

Post by Alessandro Dentella
TIA
sandro
*:-)
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!

--
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se

Alessandro Dentella

2013-05-24 15:16:51 UTC

Post by Glenn Steen
... If you do a "MailScanner --lint", that should point you in the
right direction.

Ok, so now we don't have any syntax errors, That's good:-).
Next over to semantics... Best is to do a debug run (this is described
stop mailscanner via the init script ("service MailScanner stop", or
"/etc/init.d/MailScanner stop")
Start postfix/you MTA ... In the default MailScanner init script
service MailScanner startin
or
/etc/init.d/MailScanner startin
start the debug run via "MailScanner --debug". This will start
MailScanner without forking any children and without closing
stdin/stderr... And it will wait for exactly 1 message (or rather ...
one batch), process it and then exit... whilst spewing a bit of debug
info onto the screen.
Best is to run that as the postfix user (even though it should work
perfectly well from root... you could do two runs, one from root, one
from postfix.. The process should change user to whatever you have the
"Run User" set to... ie postfix:-).
After a bit of chatter, it'll hang, waiting for a messagebatch...
Which you need provide via normal SMTP methods.
We'll see what that gives you.

Runnng as root:

root at smtp:~# MailScanner --debug

In Debugging mode, not forking...
Trying to setlogsock(unix)
Building a message batch to scan...
Have a batch of 2 messages.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63, <$fh> line 4.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in unlink while running with -T switch at /usr/share/MailScanner/MailScanner/MessageBatch.pm line 630.
Insecure dependency in unlink while running with -T switch at /usr/share/MailScanner/MailScanner/MessageBatch.pm line 630.
Insecure dependency in unlink while running with -T switch at /usr/share/MailScanner/MailScanner/MessageBatch.pm line 630.
Insecure dependency in unlink while running with -T switch at /usr/share/MailScanner/MailScanner/MessageBatch.pm line 630.
Stopping now as you are debugging me.

Googling for this message, I understand is related to the perl code not to
system setup, correct?
So I don't see any interesting message...

line 630 is:
unlink @{$message->{spamarchive}}; # Wipe the spamarchive files
line 63 is:
return open($fh, IO::Handle::_open_mode_string($mode), $file);

If I run as postfix user, it complains it cannot setgid:

postfix at smtp:~$ /usr/sbin/MailScanner --debug
Can't set GID 33 at /usr/sbin/MailScanner line 1541.

once more I'm you you hands...

sandro
*;-)

PS: I'm using perl 5.10.1-27
ii perl 5.10.1-17squeeze6

Glenn Steen

2013-05-27 08:41:36 UTC

Post by Glenn Steen
... If you do a "MailScanner --lint", that should point you in the
right direction.

root at smtp:~# MailScanner --debug
In Debugging mode, not forking...
Trying to setlogsock(unix)
Building a message batch to scan...
Have a batch of 2 messages.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63, <$fh> line 4.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in open while running with -T switch at /usr/lib/perl/5.10/IO/File.pm line 63.
Insecure dependency in unlink while running with -T switch at /usr/share/MailScanner/MailScanner/MessageBatch.pm line 630.
Insecure dependency in unlink while running with -T switch at /usr/share/MailScanner/MailScanner/MessageBatch.pm line 630.
Insecure dependency in unlink while running with -T switch at /usr/share/MailScanner/MailScanner/MessageBatch.pm line 630.
Insecure dependency in unlink while running with -T switch at /usr/share/MailScanner/MailScanner/MessageBatch.pm line 630.
Stopping now as you are debugging me.
Googling for this message, I understand is related to the perl code not to
system setup, correct?
So I don't see any interesting message...

Well, the above probably indicate that any file manipulations done in
the perl code, through those "insecure" calls/dependencies, don't get
done.
Edit your MailScanner executable and change the first line from
#!/usr/bin/perl -I/usr/lib/MailScanner
to
#!/usr/bin/perl -I/usr/lib/MailScanner -U
... just to turn the tainting code (in perl) off. Restart MailScanner
after that and see if it works better...

Kind of a known issue:-).
You can find which file to edit with "which MailScanner", but it
likely is /usr/sbin/MailScanner that need be edited.

Post by Alessandro Dentella
return open($fh, IO::Handle::_open_mode_string($mode), $file);
postfix at smtp:~$ /usr/sbin/MailScanner --debug
Can't set GID 33 at /usr/sbin/MailScanner line 1541.
once more I'm you you hands...
sandro
*;-)
PS: I'm using perl 5.10.1-27
ii perl 5.10.1-17squeeze6

Cheers!

--
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se

Alessandro Dentella

2013-05-27 11:21:25 UTC

Post by Glenn Steen
start the debug run via "MailScanner --debug". This will start
MailScanner without forking any children and without closing
stdin/stderr... And it will wait for exactly 1 message (or rather ...
one batch), process it and then exit... whilst spewing a bit of debug
info onto the screen.
Best is to run that as the postfix user (even though it should work
perfectly well from root... you could do two runs, one from root, one
from postfix.. The process should change user to whatever you have the
"Run User" set to... ie postfix:-).
After a bit of chatter, it'll hang, waiting for a messagebatch...
Which you need provide via normal SMTP methods.
We'll see what that gives you.

Well, I'm getting more and more puzzled.

1. I already have -U flag in the shabang

2. the system is derived from an EFA [1] virtual machine that I modified. I
reinstalled the original EFA and that does work correctly

So I'm checking the two system side-by-side and I can't see the
differences...

Is there a way to raise the debugging level of MailScanner?

sandro
*:-)

Alessandro Dentella

2013-05-27 18:16:36 UTC

While trying to debug why MailScanner does not save messages in
/var/spool/mailScanner/qurantine I compared 2 almost identical machines.
One is working correctly (efa) and from that machine I copied MailScanner
conf to the other (smtp):

smtp# cd /etc
smtp# mv MailScanner MailScanner.orig
smtp# rsync -a efa:/etc/MailScanner .

then restarted MailScanner

While 'efa' saves correctly messages 'smtp' does not.
My uderstanding of how MailScanner works is probably wrong since otehrwise
they should both work.

The onlu difference between efa and smtp is in postfix configuration.

My understanding is that postfix puts on HOLD via the header_checks =
regexp:/etc/postfix/header_checks (/var/spool/postfix/hold/) and from there
MailScanner read the message, clean and take any needed action.

If this is correct no other agent apart from MailScanner are responsible for
copying from postfix/hold to MAilScanner/quarantine.

So, How could possibily behave different on 2 identical machines?

sandro
*:-)

Jerry Benton

2013-05-27 19:10:10 UTC

These are the primary things I check for when dealing with this problem:

1. Selinux. Put in permissive and then build custom policies. Return to
enforcing.
2. MailScanner Run As and directory ownership and permissions.
3. MailScanner store messages as user matches permissions.
4. Making sure you add the -U option to MailScanner for the newer versions
of perl.

sed -i 's:#!/usr/bin/perl -I:#!/usr/bin/perl -U -I:g' /usr/sbin/MailScanner

Also regarding permissions, I personally like to create an extra group and
then add postfix and clamav to that group. I then use that group in all
settings and use group writeable permissions. (0660)

YMMV

Jerry Benton

PS.. run MailScanner --lint and post here if the above does not help.

Post by Alessandro Dentella
While trying to debug why MailScanner does not save messages in
/var/spool/mailScanner/qurantine I compared 2 almost identical machines.
One is working correctly (efa) and from that machine I copied MailScanner
smtp# cd /etc
smtp# mv MailScanner MailScanner.orig
smtp# rsync -a efa:/etc/MailScanner .
then restarted MailScanner
While 'efa' saves correctly messages 'smtp' does not.
My uderstanding of how MailScanner works is probably wrong since otehrwise
they should both work.
The onlu difference between efa and smtp is in postfix configuration.
My understanding is that postfix puts on HOLD via the header_checks =
regexp:/etc/postfix/header_checks (/var/spool/postfix/hold/) and from there
MailScanner read the message, clean and take any needed action.
If this is correct no other agent apart from MailScanner are responsible for
copying from postfix/hold to MAilScanner/quarantine.
So, How could possibily behave different on 2 identical machines?
sandro
*:-)
--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!

--
--
Jerry Benton
Mailborder Systems
www.mailborder.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130527/050ad6a9/attachment.html

Alessandro Dentella

2013-05-29 08:51:20 UTC

Post by Jerry Benton
1. Selinux. Put in permissive and then build custom policies. Return to
enforcing.
2. MailScanner Run As and directory ownership and permissions.
3. MailScanner store messages as user matches permissions.
4. Making sure you add the -U option to MailScanner for the newer versions of
perl.?
sed -i 's:#!/usr/bin/perl -I:#!/usr/bin/perl -U -I:g' /usr/sbin/MailScanner

Today I decided to debug using strace, it resulted clearly that the file was
correctly written in quarantine but later deleted (unlink).

I also found the single line that does that, commenting it I get the mail in
quarantine:

sub RemoveInfectedSpam {
my $this = shift;

my($id, $message);

while(($id, $message) = each %{$this->{messages}}) {
#print STDERR "Message is infected\n" if $message->{infected};
# next unless $message->{infected};
next unless
MailScanner::Config::Value('keepspamarchiveclean', $message) =~ /1/;
#print STDERR "Deleting " . join(',',@{$message->{spamarchive}}) . "\n";
# unlink @{$message->{spamarchive}}; # Wipe the spamarchive files <<<<< this deletes
@{$this->{spamarchive}} = (); # Wipe the spamarchive array
}
}

So the problem is to understand why
MailScanner::Config::Value('keepspamarchiveclean', $message) =~ /1/; says
that it should be deleted.

How is that evaluated?

Thanks again for any help
sandro
*:-)

--
Sandro Dentella *:-)
http://www.reteisi.org Soluzioni libere per le scuole
http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy

Glenn Steen

2013-06-07 12:49:10 UTC

Post by Jerry Benton
1. Selinux. Put in permissive and then build custom policies. Return to
enforcing.
2. MailScanner Run As and directory ownership and permissions.
3. MailScanner store messages as user matches permissions.
4. Making sure you add the -U option to MailScanner for the newer versions of
perl.
sed -i 's:#!/usr/bin/perl -I:#!/usr/bin/perl -U -I:g' /usr/sbin/MailScanner

You've set
Keep Spam And MCP Archive Clean = yes
... So if you don't want that, then set it to "no" (should be the default:).

Post by Alessandro Dentella
Thanks again for any help
sandro
*:-)

Cheers!

--
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se

Oliveiros Peixoto

2013-05-27 23:13:49 UTC

Can you check permissions in both directory servers.

Enviado via iPhone

Glenn Steen

2013-06-07 12:37:22 UTC

Hello Sandro,

Sorry for the late reply, I've been ill for a while, so,... hence the delay.
Look below.