[Dspam-user] Understanding signatures

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[Dspam-user] Understanding signatures

Alan Chandler

I have reached a small impass with my spam retraining mechanisms, and I
think it is because I don't understand signatures properly.

My basic set up is postfix <->dspam-milter<->dspam with the hash
database as the backend. Postfix eventually deliverers via lmtp to
dovecot. using sender_bcc_maps, each user gets their sent mail saved

I think that works because
a) the hash database has stuff in it
b) my messages have X-DSPAM headers in them

this was all done before I did the following

I added a file group to /var/spool/dspam  with a single line with
dspam:shared:* in it

My rationale for this was two fold

1) There is only 3 of us with 4 accounts.  It is probably better to
share the dictionary
2) See below on sending the training messages from a different user, so
that the sender_bcc_maps don't save the training mail

I have dovecot-antispam setup to e-mail back through postfix from user
[hidden email] (a non existent user) to a a dspam
retraining address with the following taken from my postfix master.cf file

dspam-retrain unix -    n    n    -    -    pipe
   flags=Rhq user=dspam argv=/usr/bin/dspam --client --mode=teft
--class=$nexthop --source=error --user dspam

However, my logs show that dspam doesn't like something because whenever
a retraining message arrives it says

Unable to find a valid signature. Aborting.
Process message returned error -5,  dropping message


Is this because the messages where originally trained as user alan and
not dspam? or is it indicative of a real problem.

I don't quite yet know how to test this - I will probably have to
forward it off an outside address and bring them back through to check,
but I would really like to understand what is the problem first, and
sending the e-mail to outside haves the risk that I will be seen as a
spammer, so I would prefer to avoid it if possible.

Thanks

Alan Chandler








------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Wijatmoko U. Prayitno
On Thu, 06 Mar 2014 09:12:26 +0000
Alan Chandler <[hidden email]> wrote:

> My basic set up is postfix <->dspam-milter<->dspam with the hash
> database as the backend. Postfix eventually deliverers via lmtp to
> dovecot. using sender_bcc_maps, each user gets their sent mail saved
>
dspam not support milter, i think your referring content-filter postfix.
and considering to use mysql/postgresql as database backend rather than
hash.. not safe. read again file README how to do that.

--
 WUP

------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
On 06/03/14 09:33, Wijatmoko U. Prayitno wrote:

> On Thu, 06 Mar 2014 09:12:26 +0000
> Alan Chandler <[hidden email]> wrote:
>
>> My basic set up is postfix <->dspam-milter<->dspam with the hash
>> database as the backend. Postfix eventually deliverers via lmtp to
>> dovecot. using sender_bcc_maps, each user gets their sent mail saved
>>
> dspam not support milter, i think your referring content-filter postfix.
> and considering to use mysql/postgresql as database backend rather than
> hash.. not safe. read again file README how to do that.
>
There is a python dspam-milter package written by Tom Hendrix, who I
think is a regular poster to this list.  I have it set up (along with 3
other milters) in my postfix configuration.

It took my a while to get permissions and sockets correct, and I had a
problem where-by if the milter is run as anything other than root it
fails to spawn itself as a daemon, but its now definitely there and
doing the job.

The Training Emails do not go via the milter.  They are injected via
sendmail, which doesn't use smptd as its reception agent, so bypass all
the milters on that run through.

------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
On 06/03/14 10:07, Alan Chandler wrote:
>
> There is a python dspam-milter package written by Tom Hendrix, who I
>
Apologies, I see it is spelt Tom Hendrikx



------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Tom Hendrikx
In reply to this post by Alan Chandler
On 03/06/2014 11:07 AM, Alan Chandler wrote:

> On 06/03/14 09:33, Wijatmoko U. Prayitno wrote:
>> On Thu, 06 Mar 2014 09:12:26 +0000
>> Alan Chandler <[hidden email]> wrote:
>>
>>> My basic set up is postfix <->dspam-milter<->dspam with the hash
>>> database as the backend. Postfix eventually deliverers via lmtp to
>>> dovecot. using sender_bcc_maps, each user gets their sent mail saved
>>>
>> dspam not support milter, i think your referring content-filter postfix.
>> and considering to use mysql/postgresql as database backend rather than
>> hash.. not safe. read again file README how to do that.
>>
> There is a python dspam-milter package written by Tom Hendrix, who I
> think is a regular poster to this list.  I have it set up (along with 3
> other milters) in my postfix configuration.
>
> It took my a while to get permissions and sockets correct, and I had a
> problem where-by if the milter is run as anything other than root it
> fails to spawn itself as a daemon, but its now definitely there and
> doing the job.
Could you send me some details on this (off-list, github, new thread
here, whatever you like:)) ? Then I can look into it.
>
> The Training Emails do not go via the milter.  They are injected via
> sendmail, which doesn't use smptd as its reception agent, so bypass all
> the milters on that run through.
>

Tom


------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user

signature.asc (919 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Tom Hendrikx
In reply to this post by Alan Chandler
On 03/06/2014 10:12 AM, Alan Chandler wrote:

>
> I have reached a small impass with my spam retraining mechanisms, and I
> think it is because I don't understand signatures properly.
>
> My basic set up is postfix <->dspam-milter<->dspam with the hash
> database as the backend. Postfix eventually deliverers via lmtp to
> dovecot. using sender_bcc_maps, each user gets their sent mail saved
>
> I think that works because
> a) the hash database has stuff in it
> b) my messages have X-DSPAM headers in them
>
> this was all done before I did the following
>
> I added a file group to /var/spool/dspam  with a single line with
> dspam:shared:* in it
>
> My rationale for this was two fold
>
> 1) There is only 3 of us with 4 accounts.  It is probably better to
> share the dictionary
> 2) See below on sending the training messages from a different user, so
> that the sender_bcc_maps don't save the training mail
>
> I have dovecot-antispam setup to e-mail back through postfix from user
> [hidden email] (a non existent user) to a a dspam
> retraining address with the following taken from my postfix master.cf file
Any reason why you're not using the dspam support in the antispam plugin
directly? It can execute dspamc and talk to dspam directly, without
having to use email as a go-between.

>
> dspam-retrain unix -    n    n    -    -    pipe
>    flags=Rhq user=dspam argv=/usr/bin/dspam --client --mode=teft
> --class=$nexthop --source=error --user dspam
>
> However, my logs show that dspam doesn't like something because whenever
> a retraining message arrives it says
>
> Unable to find a valid signature. Aborting.
> Process message returned error -5,  dropping message
Did you look at the messages that arrive (f.i. by creating a kludge in
Postfix that sends/stores you a copy of everything that is passed off to
the retrain transport)? Does the message have the signature in the
header? The milter only adds headers, so they might disappear when you
forward stuff (not sure how dovecot-antispam does that).


>
>
> Is this because the messages where originally trained as user alan and
> not dspam? or is it indicative of a real problem.
>
> I don't quite yet know how to test this - I will probably have to
> forward it off an outside address and bring them back through to check,
> but I would really like to understand what is the problem first, and
> sending the e-mail to outside haves the risk that I will be seen as a
> spammer, so I would prefer to avoid it if possible.
>
> Thanks
>
> Alan Chandler




------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user

signature.asc (919 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
In reply to this post by Tom Hendrikx
On 06/03/14 11:24, Tom Hendrikx wrote:
>
> Could you send me some details on this (off-list, github, new thread
> here, whatever you like:)) ? Then I can look into it.
> Tom
>
>
I'll start a new thread.

Alan

------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
In reply to this post by Tom Hendrikx
On 06/03/14 11:31, Tom Hendrikx wrote:

> On 03/06/2014 10:12 AM, Alan Chandler wrote:
>> I have reached a small impass with my spam retraining mechanisms, and I
>> think it is because I don't understand signatures properly.
>>
>> My basic set up is postfix <->dspam-milter<->dspam with the hash
>> database as the backend. Postfix eventually deliverers via lmtp to
>> dovecot. using sender_bcc_maps, each user gets their sent mail saved
>>
>> I think that works because
>> a) the hash database has stuff in it
>> b) my messages have X-DSPAM headers in them
>>
>> this was all done before I did the following
>>
>> I added a file group to /var/spool/dspam  with a single line with
>> dspam:shared:* in it
>>
>> My rationale for this was two fold
>>
>> 1) There is only 3 of us with 4 accounts.  It is probably better to
>> share the dictionary
>> 2) See below on sending the training messages from a different user, so
>> that the sender_bcc_maps don't save the training mail
>>
>> I have dovecot-antispam setup to e-mail back through postfix from user
>> [hidden email] (a non existent user) to a a dspam
>> retraining address with the following taken from my postfix master.cf file
> Any reason why you're not using the dspam support in the antispam plugin
> directly? It can execute dspamc and talk to dspam directly, without
> having to use email as a go-between.
>
>
I tried it and I couldn't make dspam run.  (nor could I find anything in
the log to say why)  All that would happen is that the attempt to move a
mail from my inbox to my junk box would fail in thunderbird

So I gave up.  I assumed at the time it was because antispam was calling
it with uid 10001 (which doesn't have an account).  I can't remember the
exact message that made me think that, and looking back at syslog and
mail.log at the time this happened I can't find any entries at all for dspam

Just looking back at what I discovered since, it might be because dspam
was having read problems with dspam.conf (permissions 660 with root.root
as the owner)

I will try this again this evening because I would much prefer to do it
this way rather than all the other issues around non existent e-mail
addresses.





------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
In reply to this post by Alan Chandler
On 06/03/14 11:33, Alan Chandler wrote:

> On 06/03/14 11:24, Tom Hendrikx wrote:
>> Could you send me some details on this (off-list, github, new thread
>> here, whatever you like:)) ? Then I can look into it.
>> Tom
>>
>>
> I'll start a new thread.
>
> Alan
>

No need - I figured out the problem.  What happens on the system is that
/run (symlinked to from /var/run) is owned by root and has permissions 755

When run as a non root user it can't write the pid file and then
silently exits.

Can I tell the milter not to create the pid file and let start stop
daemon do it instead?


Alan


------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Tom Hendrikx
On 03/06/2014 01:28 PM, Alan Chandler wrote:

> On 06/03/14 11:33, Alan Chandler wrote:
>> On 06/03/14 11:24, Tom Hendrikx wrote:
>>> Could you send me some details on this (off-list, github, new thread
>>> here, whatever you like:)) ? Then I can look into it.
>>> Tom
>>>
>>>
>> I'll start a new thread.
>>
>> Alan
>>
>
> No need - I figured out the problem.  What happens on the system is that
> /run (symlinked to from /var/run) is owned by root and has permissions 755
>
> When run as a non root user it can't write the pid file and then
> silently exits.
>
> Can I tell the milter not to create the pid file and let start stop
> daemon do it instead?
>
Exiting silently when pidfile creation fails, should not happen (but
that probably why the default is in /tmp right now ;>). It should at
least put something in the logs and have an exit code indicating issues.
I'll look into that.

As far as not creating a pid file: that would be possible.

Proper way would probably be to have a directory (/var/run/dspam-milter
f.i.) owned by the user running the milter, and put the pidfile in
there. The directory would have to be managed by the init script, i.e.
distro-specific.

I created an issue for this:
https://github.com/whyscream/dspam-milter/issues/16

Tom


------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user

signature.asc (919 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
On 06/03/14 12:51, Tom Hendrikx wrote:
> Proper way would probably be to have a directory
> (/var/run/dspam-milter f.i.) owned by the user running the milter, and
> put the pidfile in there. The directory would have to be managed by
> the init script, i.e. distro-specific.
I have set this up, and now dspam-milter is running nicely as user dspam

Thanks

Alan



------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Tom Hendrikx
In reply to this post by Alan Chandler
On 03/06/2014 12:58 PM, Alan Chandler wrote:

> On 06/03/14 11:31, Tom Hendrikx wrote:
>> On 03/06/2014 10:12 AM, Alan Chandler wrote:
>>> I have reached a small impass with my spam retraining mechanisms, and I
>>> think it is because I don't understand signatures properly.
>>>
>>> My basic set up is postfix <->dspam-milter<->dspam with the hash
>>> database as the backend. Postfix eventually deliverers via lmtp to
>>> dovecot. using sender_bcc_maps, each user gets their sent mail saved
>>>
>>> I think that works because
>>> a) the hash database has stuff in it
>>> b) my messages have X-DSPAM headers in them
>>>
>>> this was all done before I did the following
>>>
>>> I added a file group to /var/spool/dspam  with a single line with
>>> dspam:shared:* in it
>>>
>>> My rationale for this was two fold
>>>
>>> 1) There is only 3 of us with 4 accounts.  It is probably better to
>>> share the dictionary
>>> 2) See below on sending the training messages from a different user, so
>>> that the sender_bcc_maps don't save the training mail
>>>
>>> I have dovecot-antispam setup to e-mail back through postfix from user
>>> [hidden email] (a non existent user) to a a dspam
>>> retraining address with the following taken from my postfix master.cf file
>> Any reason why you're not using the dspam support in the antispam plugin
>> directly? It can execute dspamc and talk to dspam directly, without
>> having to use email as a go-between.
>>
>>
> I tried it and I couldn't make dspam run.  (nor could I find anything in
> the log to say why)  All that would happen is that the attempt to move a
> mail from my inbox to my junk box would fail in thunderbird
>
> So I gave up.  I assumed at the time it was because antispam was calling
> it with uid 10001 (which doesn't have an account).  I can't remember the
> exact message that made me think that, and looking back at syslog and
> mail.log at the time this happened I can't find any entries at all for dspam
>
> Just looking back at what I discovered since, it might be because dspam
> was having read problems with dspam.conf (permissions 660 with root.root
> as the owner)
The dspam binary should be installed with setgid bit enabled (the 's' in
ls output: -rwxr-sr-x 1 dspam dspam 105792 May 30  2013 /usr/bin/dspam)
which means that dspam.conf should be readable for the dspam binary when
dspam.conf is readable by group 'dspam'.

To be able to retrain, the username of the calling user should probably
also be configured as trusted user ('Trust ...' in dspam.conf).

>
> I will try this again this evening because I would much prefer to do it
> this way rather than all the other issues around non existent e-mail
> addresses.
>
>
>
>
>
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works.
> Faster operations. Version large binaries.  Built-in WAN optimization and the
> freedom to use Git, Perforce or both. Make the move to Perforce.
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Dspam-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/dspam-user
>


------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user

signature.asc (919 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Rick Leir
In reply to this post by Alan Chandler
On Thu, 2014-03-06 at 12:28 +0000, Alan Chandler wrote:
No need - I figured out the problem.  What happens on the system is that 
/run (symlinked to from /var/run) is owned by root and has permissions 755

When run as a non root user it can't write the pid file and then 
silently exits.

Can I tell the milter not to create the pid file and let start stop 
daemon do it instead?
Alan
Can you run dspam as a content filter?

# ls -la /var/run/dspam
total 4
drwxr-xr-x  2 dspam root   60 Dec 11 16:58 .
drwxr-xr-x 12 root  root  380 Mar  6 07:53 ..
-rw-rw----  1 dspam dspam   4 Dec 11 16:58 dspam.pid

# ps -aef |grep dsp
dspam      237     1  0  2013 ?        00:07:37 /usr/bin/dspam --daemon

# head /etc/postfix/master.cf
smtp      inet  n       -       -       -       -       smtpd
  -o syslog_name=postfix25
  -o content_filter=lmtp:unix:/dspam/dspam.sock

Rick Leir, Senior Developer
http://CirrusComputing.com/ 


------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
In reply to this post by Tom Hendrikx
On 06/03/14 13:14, Tom Hendrikx wrote:

> On 03/06/2014 12:58 PM, Alan Chandler wrote:
>> I tried it and I couldn't make dspam run.  (nor could I find anything in
>> the log to say why)  All that would happen is that the attempt to move a
>> mail from my inbox to my junk box would fail in thunderbird
>>
>> So I gave up.  I assumed at the time it was because antispam was calling
>> it with uid 10001 (which doesn't have an account).  I can't remember the
>> exact message that made me think that, and looking back at syslog and
>> mail.log at the time this happened I can't find any entries at all for dspam
>>
>> Just looking back at what I discovered since, it might be because dspam
>> was having read problems with dspam.conf (permissions 660 with root.root
>> as the owner)
> The dspam binary should be installed with setgid bit enabled (the 's' in
> ls output: -rwxr-sr-x 1 dspam dspam 105792 May 30  2013 /usr/bin/dspam)
> which means that dspam.conf should be readable for the dspam binary when
> dspam.conf is readable by group 'dspam'.
>
> To be able to retrain, the username of the calling user should probably
> also be configured as trusted user ('Trust ...' in dspam.conf).
>
>

I have tried to set dovecot-antispam up again to run the dspam backend.
but it still isn't working

Here is the part of syslog where antispam starts up, The crux of it
seems to be dspam saying "Unable to determine the runtime user"

I think antispam will be running as user uid 10001 for which there is no
system account.  --user dspam, is set because I have a shared group
called dspam

Mar  6 17:59:12 piserver imap: antispam: plugin initialising (2.0-notgit)
Mar  6 17:59:12 piserver imap: antispam: verbose debug enabled
Mar  6 17:59:12 piserver imap: antispam: "Junk" is exact match spam folder
Mar  6 17:59:12 piserver imap: antispam: no unsure folders
Mar  6 17:59:12 piserver imap: antispam: "Trash" is exact match trash folder
Mar  6 17:59:12 piserver imap: antispam: dspam binary set to /usr/bin/dspam
Mar  6 17:59:12 piserver imap: antispam: dspam extra arg --deliver=
Mar  6 17:59:12 piserver imap: antispam: dspam extra arg --user
Mar  6 17:59:12 piserver imap: antispam: dspam extra arg dspam
Mar  6 17:59:12 piserver imap: antispam: signature header line is
"X-DSPAM-Signature"
Mar  6 17:59:18 piserver imap: antispam: mailbox_is_unsure(Junk): 0
Mar  6 17:59:18 piserver imap: antispam: mailbox_is_trash(INBOX): 0
Mar  6 17:59:18 piserver imap: antispam: mailbox_is_trash(Junk): 0
Mar  6 17:59:18 piserver imap: antispam: mail copy: from trash: 0, to
trash: 0
Mar  6 17:59:18 piserver imap: antispam: mailbox_is_spam(INBOX): 0
Mar  6 17:59:18 piserver imap: antispam: mailbox_is_spam(Junk): 1
Mar  6 17:59:18 piserver imap: antispam: mailbox_is_unsure(INBOX): 0
Mar  6 17:59:18 piserver imap: antispam: mail copy: src spam: 0, dst
spam: 1, src unsure: 0
Mar  6 17:59:18 piserver imap: antispam: /usr/bin/dspam --source=error
--class=spam --signature=531890a5293641874011248 --deliver= --user dspam
Mar  6 17:59:18 piserver dspam[30868]: Unable to determine the runtime user






------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Tom Hendrikx
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 06-03-14 19:04, Alan Chandler wrote:

> On 06/03/14 13:14, Tom Hendrikx wrote:
>> On 03/06/2014 12:58 PM, Alan Chandler wrote:
>>> I tried it and I couldn't make dspam run.  (nor could I find
>>> anything in the log to say why)  All that would happen is that
>>> the attempt to move a mail from my inbox to my junk box would
>>> fail in thunderbird
>>>
>>> So I gave up.  I assumed at the time it was because antispam
>>> was calling it with uid 10001 (which doesn't have an account).
>>> I can't remember the exact message that made me think that, and
>>> looking back at syslog and mail.log at the time this happened I
>>> can't find any entries at all for dspam
>>>
>>> Just looking back at what I discovered since, it might be
>>> because dspam was having read problems with dspam.conf
>>> (permissions 660 with root.root as the owner)
>> The dspam binary should be installed with setgid bit enabled (the
>> 's' in ls output: -rwxr-sr-x 1 dspam dspam 105792 May 30  2013
>> /usr/bin/dspam) which means that dspam.conf should be readable
>> for the dspam binary when dspam.conf is readable by group
>> 'dspam'.
>>
>> To be able to retrain, the username of the calling user should
>> probably also be configured as trusted user ('Trust ...' in
>> dspam.conf).
>>
>>
>
> I have tried to set dovecot-antispam up again to run the dspam
> backend. but it still isn't working
>
> Here is the part of syslog where antispam starts up, The crux of it
>  seems to be dspam saying "Unable to determine the runtime user"
>
> I think antispam will be running as user uid 10001 for which there
> is no system account.  --user dspam, is set because I have a shared
> group called dspam

Dspam needs to know who is calling it, so it can check if it's a
trusted user. Real question is of course why you are trying to run it
as a numeric uid that does not belong to any system account :)

If you have a valid reason for doing that (can't really think of any,
but it's your call), then try wrapping the call to dspam in the
antispam config with sudo: run the command (passwordless) as an
existing system user/group.

>
> Mar  6 17:59:12 piserver imap: antispam: plugin initialising
> (2.0-notgit) Mar  6 17:59:12 piserver imap: antispam: verbose debug
> enabled Mar  6 17:59:12 piserver imap: antispam: "Junk" is exact
> match spam folder Mar  6 17:59:12 piserver imap: antispam: no
> unsure folders Mar  6 17:59:12 piserver imap: antispam: "Trash" is
> exact match trash folder Mar  6 17:59:12 piserver imap: antispam:
> dspam binary set to /usr/bin/dspam Mar  6 17:59:12 piserver imap:
> antispam: dspam extra arg --deliver= Mar  6 17:59:12 piserver imap:
> antispam: dspam extra arg --user Mar  6 17:59:12 piserver imap:
> antispam: dspam extra arg dspam Mar  6 17:59:12 piserver imap:
> antispam: signature header line is "X-DSPAM-Signature" Mar  6
> 17:59:18 piserver imap: antispam: mailbox_is_unsure(Junk): 0 Mar  6
> 17:59:18 piserver imap: antispam: mailbox_is_trash(INBOX): 0 Mar  6
> 17:59:18 piserver imap: antispam: mailbox_is_trash(Junk): 0 Mar  6
> 17:59:18 piserver imap: antispam: mail copy: from trash: 0, to
> trash: 0 Mar  6 17:59:18 piserver imap: antispam:
> mailbox_is_spam(INBOX): 0 Mar  6 17:59:18 piserver imap: antispam:
> mailbox_is_spam(Junk): 1 Mar  6 17:59:18 piserver imap: antispam:
> mailbox_is_unsure(INBOX): 0 Mar  6 17:59:18 piserver imap:
> antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Mar  6
> 17:59:18 piserver imap: antispam: /usr/bin/dspam --source=error
> --class=spam --signature=531890a5293641874011248 --deliver= --user
> dspam Mar  6 17:59:18 piserver dspam[30868]: Unable to determine
> the runtime user
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTGOc5AAoJEJPfMZ19VO/1saQQAKZt84ZiXiHQZQ1dhV1qzLRg
zQ5EPVzjBE/3APaLWljpRCzLisPhUMc6c/QTAVspOtI7DIWQyCq8w4qaaFIbScKk
mpBdg1qpauB3c51+xX+3+/GFX7YNFGOlzrE7b6Qi0CQUjcQE12b9+oMXoZ3sRnC6
aMnBRxqO6UKdSFQTnxL020fr2LlvRPGwATATWdiRrLIuenbRBwy7RwKRBPpyZHnO
aByltSJCGYsfViRnpJufQHRAC4c+p71iyWYOW/ielEjCrM2ox2uGNdNTQup3lF0J
SUiifqeM43cdKim28Irtl1V1vCHArxQ217Gv2FYED5KjsSv+iVLPXM5otMCsuECT
oGwlSfV2BeCvsIwSZpxF6pcdswdsNMGwf19neyTRKg20uiAMllF8yISviNBM+F0u
aXO4jJpqtSOo06QKcpY0aN6PAKMiAhgPpukR08lL3PEnDSu6mg2KrB6ZX3GGExpC
pKMfxyvR7/1xztiKhOLrACeGfOZMe9eyQ5tHRGeOv762dEeiJlXaJOdKG6GUjkO6
zzYfxivnLQ6vsT1wwGywCU38sfj5bkk8twkFzc6VqOi4DSgn1xo1XWT5nPFgBYNh
W2nCpRzBzOybPdE2WGBLPw9Wx0vD8EcyeW5tRpq3B+vlY1Y09dw4hpyywsS7hoe2
49uAkX9pgSP/J/UPE2Wn
=GX0s
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
On 06/03/14 21:23, Tom Hendrikx wrote:
>
> Dspam needs to know who is calling it, so it can check if it's a
> trusted user. Real question is of course why you are trying to run it
> as a numeric uid that does not belong to any system account :)

Well, because I suppose that is the architecture of Dovecot and the
Antispam plugin.  It runs it in the context of the uid of the users
mailbox.

Why = when I started this project, although I only have 4 users, I
decided it was all about doing what someone with a virtual mailboxes
with 1000s of users
would do, so I tried to design the system so that I didn't need a system
user.

I came across a similar post to mine about 4 years old - someone had hit
this problem and couldn't find the correct answer.  He didn't seem to
find the answer then, and went down the mail forwarding approach

It looks though this discussion is going the same way.  Dovecot antispam
and a dspam backend is not the right way when you have virtual
mailboxes.  Instead, use antispam's ability to pipe the mail to sendmail
and postfix's ability to call dspam with a solid user account.  And as
someone said in that other thread - makes sense to not have all the
users creating lots of threads as they move chunks of messages to and
out of their spam folder - but rather use postfix's queuing mechanism to
have a more controlled approach to such a load.








------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Tom Hendrikx
On 03/07/2014 12:26 AM, Alan Chandler wrote:

> On 06/03/14 21:23, Tom Hendrikx wrote:
>>
>> Dspam needs to know who is calling it, so it can check if it's a
>> trusted user. Real question is of course why you are trying to run it
>> as a numeric uid that does not belong to any system account :)
>
> Well, because I suppose that is the architecture of Dovecot and the
> Antispam plugin.  It runs it in the context of the uid of the users
> mailbox.
>
> Why = when I started this project, although I only have 4 users, I
> decided it was all about doing what someone with a virtual mailboxes
> with 1000s of users
> would do, so I tried to design the system so that I didn't need a system
> user.
The regular approach to that is that you run and store everything under
a single uid, typically named 'vmail' [1]. Right now you have a 'virtual
accounts' setup that borrows privilege separation from the 'system
accounts' design, but with crippled/broken system accounts.

[1] http://wiki2.dovecot.org/HowTo/SimpleVirtualInstall

>
> I came across a similar post to mine about 4 years old - someone had hit
> this problem and couldn't find the correct answer.  He didn't seem to
> find the answer then, and went down the mail forwarding approach
>
> It looks though this discussion is going the same way.  Dovecot antispam
> and a dspam backend is not the right way when you have virtual
> mailboxes.  Instead, use antispam's ability to pipe the mail to sendmail
> and postfix's ability to call dspam with a solid user account.  And as
> someone said in that other thread - makes sense to not have all the
> users creating lots of threads as they move chunks of messages to and
> out of their spam folder - but rather use postfix's queuing mechanism to
> have a more controlled approach to such a load.
>
Do you have any number that back this up? I guess that the load caused
by an incoming spam run creates a larger peak in IMAP activities (i.e.
dovecots LDA storing the messages) than the users that will use the
anti-spam plugin to move those messages around: the delivery happens
within the hour, the users will do their work in the 24 or so hours
after that...

Anyway: if you want to go down the sendmail approach, you need to look
into the way that messages are formatted when they are sent, and if the
headers are preserved.

Tom


------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user

signature.asc (919 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Understanding signatures

Alan Chandler
In reply to this post by Alan Chandler
On 06/03/14 09:12, Alan Chandler wrote:

> I have dovecot-antispam setup to e-mail back through postfix from user
> [hidden email] (a non existent user) to a a dspam
> retraining address with the following taken from my postfix master.cf file
>
> dspam-retrain unix -    n    n    -    -    pipe
>     flags=Rhq user=dspam argv=/usr/bin/dspam --client --mode=teft
> --class=$nexthop --source=error --user dspam
>
> However, my logs show that dspam doesn't like something because whenever
> a retraining message arrives it says
>
> Unable to find a valid signature. Aborting.
> Process message returned error -5,  dropping message
>
>
This thread went off in a side direction which I explored and got lost
in.  I want to bring it back to the real issue, which I have now.

I have reinstated dovecot-antispam sending the e-mails to the training
address when I move them from my inbox to Junk (or vica versa).

They arrive. and I get the error message as shown above.

I have a file /var/spool/dspam/group with the contents

dspam:shared:*

and that is the user that I use in the postfix dspam-retrain command
(see above).

I had a poke around in the hash database. (or at least I presume that is
what it is).

There are directories for the domain the mail was sent to, and then
inside the directory for the domain recipient there is the name of the
user as alan.chandler (ie me) that the mail was sent to.
inside that directory is a file called alan.chandler.log

Inside that file was the signature I was looking for (I looked at the
header X-DPAM_Signature of the original mail)

ALSO

there is a directory "local" with a directory inside that called "dspam"
and inside that a directory called "dspam.sig" and inside that there is
a with the name of the signature that I was looking for with a .sig
extension.  The content of this file is binary.


So I am wondering - in this retraining do I have to somehow use the name
"local"?





------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user