[Dspam-user] Dspam and the Dovecot Antispam plugin retraining messages

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Dspam-user] Dspam and the Dovecot Antispam plugin retraining messages

David Mehler
Hello,

 I'm having an issue where spam messages are delivered to a Spam
mailbox via sieve, but to retrain I have to move them to the inbox or
another folder. Well, the subject ***spam*** header is still there and
the message headers are still showing a status of spam and in the
message the text identified as spam is appended. I was thinking the
headers would be rewritten on a retrain and that text message would
also be redone. I'm concluding my retraining is not working. If I need
to provide logs let me know.

Thanks.
Dave.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Dspam-user] Dspam and the Dovecot Antispam plugin retraining messages

Martin Wheldon
Hi Dave,

The antispam plugin only moves the message between the imap folders
spam/other folder and optionally sends a copy of the message
to another process to do the training i.e dspam.

Your training may well be working, you can see if this is the case be
either:
1. Turning on debugging in DSPAM and retraining a message
2. Checking the users stats with dspam_stats before and after
retraining a message.

Best Regards

Martin

On 2015-02-26 21:23, David Mehler wrote:

> Hello,
>
>  I'm having an issue where spam messages are delivered to a Spam
> mailbox via sieve, but to retrain I have to move them to the inbox or
> another folder. Well, the subject ***spam*** header is still there
> and
> the message headers are still showing a status of spam and in the
> message the text identified as spam is appended. I was thinking the
> headers would be rewritten on a retrain and that text message would
> also be redone. I'm concluding my retraining is not working. If I
> need
> to provide logs let me know.
>
> Thanks.
> Dave.
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your
> hub for all
> things parallel software development, from weekly thought leadership
> blogs to
> news, videos, case studies, tutorials and more. Take a look and join
> the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Dspam-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/dspam-user
>
> !DSPAM:41,54ef8fc9249111276915234!


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Dspam-user] Dspam and the Dovecot Antispam plugin retraining messages

Tom Hendrikx
In reply to this post by David Mehler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 26-02-15 22:23, David Mehler wrote:

> Hello,
>
> I'm having an issue where spam messages are delivered to a Spam
> mailbox via sieve, but to retrain I have to move them to the inbox
> or another folder. Well, the subject ***spam*** header is still
> there and the message headers are still showing a status of spam
> and in the message the text identified as spam is appended. I was
> thinking the headers would be rewritten on a retrain and that text
> message would also be redone. I'm concluding my retraining is not
> working. If I need to provide logs let me know.

The message cannot be rewritten. this is IMAP protocol which says that
once a message has been delivered, no changes may be made to it.
There is also no reason for rewriting: the message is already
delivered, so the headers won't be used anymore for filtering or
delivering.

If you want to know whether training worked, try this:

- - Find a message you want to retrain (and didn't retrain before)
- - Run dspam_stats for the user you're training
- - Move the message so dovecot antispam will run your retrain command
- - Run dspam_stats again: it should show the change by decreasing a
counter (one of true positive/negative) and increase another (one of
false positive/negative).
- - The DSPAM system or user log (if enabled) should also have an entry
appended that mentions Retrained

If it failed, your regular dspam logging should contain an error message.

Tom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJU75jpAAoJEJPfMZ19VO/1J7cQAMXuPO/K2lC+9bMi/+fHdvg4
4eIdDP0m3ynnJQF14kbI73fw6JJhWCEC8Qe7ACb5ngar7DnzzQdNOhdTRxsnBY8m
qY/0rr41EtQml5Vrlwv+BS6f/JxL5vX75jgpG2rzeCxl9ebs58BEbe7RpW/SoMqH
Yx9V7wPiKdzpAf506LRMs8q09xseYOrTk1t3Jh5VV6HFrBYwAw57g4AkSnah6TFh
Ssnjy+b+oxgWjzBK+x2O4kHcbHbeOqikeAWy3TtEyOUx8iK4PWUg6LIx5YOwqR7s
02XvlEDa9xnxGkstkd4W4u2iD0Skbfn6h9g5bRLPeU5fgbFc0LygdbuXuwQTh+0I
eTXHjWzkwf9MxqCimGQ9FdjPzPYqtGmZsoZ1d8TgjnSzOIn919iYZWrWrqzhIcyd
wN4nrfkJ5lqE+cPXnxWoC7oZHPfi3VwE8l03XUYptV8eNLiUYYQUsR5AF077F2D1
HOsulcS6dxgLvm7NHiXvFu4xlrBuOg899DasBXM9/Phktz8azZC0TFbTNGI5XT+z
dprPQLmKnm4US4QU4b5g3kYoYQuNziMxtoPqDraIJjIjMbk9eIo+2hpNj+92bdZv
jByevO1ymNxeC2jZi5MHBwOaqc9Cw3vyt4oR96GNvQkU9nNixOuQxPn4ahncGP1Y
9st3N1afRpeJzjrIot6F
=8WQG
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Dspam-user] Dspam and the Dovecot Antispam plugin retraining messages

Eric Broch
Hi Tom,

I have a client who uses MS Outlook 2013. It strips some of the header when moving email between IMAP folders. Have you heard of this? Leave it to Microsoft to break RFC protocol.

Eric Broch

On 2/26/2015 3:06 PM, Tom Hendrikx wrote:
> On 26-02-15 22:23, David Mehler wrote:
> > Hello,
>
> > I'm having an issue where spam messages are delivered to a Spam
> > mailbox via sieve, but to retrain I have to move them to the inbox
> > or another folder. Well, the subject ***spam*** header is still
> > there and the message headers are still showing a status of spam
> > and in the message the text identified as spam is appended. I was
> > thinking the headers would be rewritten on a retrain and that text
> > message would also be redone. I'm concluding my retraining is not
> > working. If I need to provide logs let me know.
>
> The message cannot be rewritten. this is IMAP protocol which says that
> once a message has been delivered, no changes may be made to it.
> There is also no reason for rewriting: the message is already
> delivered, so the headers won't be used anymore for filtering or
> delivering.
>
> If you want to know whether training worked, try this:
>
> - Find a message you want to retrain (and didn't retrain before)
> - Run dspam_stats for the user you're training
> - Move the message so dovecot antispam will run your retrain command
> - Run dspam_stats again: it should show the change by decreasing a
> counter (one of true positive/negative) and increase another (one of
> false positive/negative).
> - The DSPAM system or user log (if enabled) should also have an entry
> appended that mentions Retrained
>
> If it failed, your regular dspam logging should contain an error message.
>
> Tom
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Dspam-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/dspam-user




------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Dspam-user] Dspam and the Dovecot Antispam plugin retraining messages

Tom Hendrikx
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Eric,

Does the change also happen on the IMAP server, or only in the
client's local cache? Which IMAP server is that?

I'm no expert, but AFAIK the client tells the server to move the
message around. There is no way for the client to tell the server to
remove or change message data. But it might download, edit, upload
again, then delete old copy, but in IMAP terms this would be
completely new message.

Tom

On 26-02-15 23:52, Eric Broch wrote:

> Hi Tom,
>
> I have a client who uses MS Outlook 2013. It strips some of the
> header when moving email between IMAP folders. Have you heard of
> this? Leave it to Microsoft to break RFC protocol.
>
> Eric Broch
>
> On 2/26/2015 3:06 PM, Tom Hendrikx wrote:
>> On 26-02-15 22:23, David Mehler wrote:
>>> Hello,
>>
>>> I'm having an issue where spam messages are delivered to a
>>> Spam mailbox via sieve, but to retrain I have to move them to
>>> the inbox or another folder. Well, the subject ***spam***
>>> header is still there and the message headers are still showing
>>> a status of spam and in the message the text identified as spam
>>> is appended. I was thinking the headers would be rewritten on a
>>> retrain and that text message would also be redone. I'm
>>> concluding my retraining is not working. If I need to provide
>>> logs let me know.
>>
>> The message cannot be rewritten. this is IMAP protocol which says
>> that once a message has been delivered, no changes may be made to
>> it. There is also no reason for rewriting: the message is
>> already delivered, so the headers won't be used anymore for
>> filtering or delivering.
>>
>> If you want to know whether training worked, try this:
>>
>> - Find a message you want to retrain (and didn't retrain before)
>> - Run dspam_stats for the user you're training - Move the message
>> so dovecot antispam will run your retrain command - Run
>> dspam_stats again: it should show the change by decreasing a
>> counter (one of true positive/negative) and increase another (one
>> of false positive/negative). - The DSPAM system or user log (if
>> enabled) should also have an entry appended that mentions
>> Retrained
>>
>> If it failed, your regular dspam logging should contain an error
>> message.
>>
>> Tom
>>
>>
> ------------------------------------------------------------------------------
>>
>
Dive into the World of Parallel Programming The Go Parallel Website,

> sponsored
>> by Intel and developed in partnership with Slashdot Media, is
>> your hub
> for all
>> things parallel software development, from weekly thought
>> leadership
> blogs to
>> news, videos, case studies, tutorials and more. Take a look and
>> join the conversation now. http://goparallel.sourceforge.net/ 
>> _______________________________________________ Dspam-user
>> mailing list [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/dspam-user
>
>
>
>
> ------------------------------------------------------------------------------
>
>
Dive into the World of Parallel Programming The Go Parallel Website,
sponsored

> by Intel and developed in partnership with Slashdot Media, is your
> hub for all things parallel software development, from weekly
> thought leadership blogs to news, videos, case studies, tutorials
> and more. Take a look and join the conversation now.
> http://goparallel.sourceforge.net/
>
>
>
> _______________________________________________ Dspam-user mailing
> list [hidden email]
> https://lists.sourceforge.net/lists/listinfo/dspam-user
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJU76juAAoJEJPfMZ19VO/1nM4P/2er+eF9h1Zt1YJXQv9AmnOK
jHr7Mqw49zylyjAggmTHuS/di7sY9QMXyfMIHiTC9wI71jD8hLWwZ7SaLDM7s1pu
akpAEPTudjvDEURLhfT31DdhgicXCmb+XMOOpB/wmQxLvKzUpaVZRx6kbn+wBXT1
joTj7VJtiQVejH+I15FBxT5EfwiXk+l4guppKs6ybikAdSSJ2R4rX10b7PuFRWlB
GRWKCEhiGlpeACvOOEpRndDoxmQ12c1Xzjw4MxBs9w6pnrUE0f5YCP6RaaHnDW1b
WgOgubscs5uD0IUcf6JP8GBIdUezivfDjJsrHGxBqSKk53TK+1iaZPrrMYDq9Ntp
Y5rT/Z+ql0n4e631m5HWcX0dsgQtV4vLmbIw+5LyTuLExva1Y5gxrMuRFOHVXZHk
ZZ+gbSG9pcoRaCmjH/bp/jfKI2fO1AAF1y6RqMraLXdrxZNmxDHuH5TERkOTVzOl
Zk3EIMLGs/I5yyt37tAC37IPOtnPTyB9rxhRq5nATfjtSazTq3KeKQCU0g6RFJV+
PcU1jsXJw9YMXkXNvRQCKzDIwa8jE9b3YS6B+bTduefexK+VPvjM6NCPS6n5WJk4
n/0OmetX7KJrWsr0aOARXocdR+xXW0WyrIWeBAqJuB1N/fBLdyoLfhlBjPRS9uaM
nKsh42cElkfLokqpBm5A
=kyvc
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Dspam-user] Dspam and the Dovecot Antispam plugin retraining messages

Eric Broch
Hi Tom,

We use Dovecot, but its not Dovecot changing the message header its MS Outlook 2013. It's not just a local copy (pst file) that's being changed; the server copy is being changed by Outlook as well. We ran into this when trying to train DSPAM on messages moved from the inbox to a spam folder and I noticed that the messages had most of the header removed. Subsequently we had to use Thunderbird to move message into the spam folder from the inbox to train DPSAM. This is just one of the links where this subject is discussed:

https://social.msdn.microsoft.com/Forums/en-US/8eafe714-8a8c-44bc-9228-d6a68731494f/outlook-2013-behavior-as-an-imap-client?forum=os_exchangeprotocols

Eric

 
On 2/26/2015 4:14 PM, Tom Hendrikx wrote:
Hi Eric,

Does the change also happen on the IMAP server, or only in the
client's local cache? Which IMAP server is that?

I'm no expert, but AFAIK the client tells the server to move the
message around. There is no way for the client to tell the server to
remove or change message data. But it might download, edit, upload
again, then delete old copy, but in IMAP terms this would be
completely new message.

Tom

On 26-02-15 23:52, Eric Broch wrote:
> Hi Tom,

> I have a client who uses MS Outlook 2013. It strips some of the
> header when moving email between IMAP folders. Have you heard of
> this? Leave it to Microsoft to break RFC protocol.

> Eric Broch

> On 2/26/2015 3:06 PM, Tom Hendrikx wrote:
>> On 26-02-15 22:23, David Mehler wrote:
>>> Hello,
>>
>>> I'm having an issue where spam messages are delivered to a
>>> Spam mailbox via sieve, but to retrain I have to move them to
>>> the inbox or another folder. Well, the subject ***spam***
>>> header is still there and the message headers are still showing
>>> a status of spam and in the message the text identified as spam
>>> is appended. I was thinking the headers would be rewritten on a
>>> retrain and that text message would also be redone. I'm
>>> concluding my retraining is not working. If I need to provide
>>> logs let me know.
>>
>> The message cannot be rewritten. this is IMAP protocol which says
>> that once a message has been delivered, no changes may be made to
>> it. There is also no reason for rewriting: the message is
>> already delivered, so the headers won't be used anymore for
>> filtering or delivering.
>>
>> If you want to know whether training worked, try this:
>>
>> - Find a message you want to retrain (and didn't retrain before)
>> - Run dspam_stats for the user you're training - Move the message
>> so dovecot antispam will run your retrain command - Run
>> dspam_stats again: it should show the change by decreasing a
>> counter (one of true positive/negative) and increase another (one
>> of false positive/negative). - The DSPAM system or user log (if
>> enabled) should also have an entry appended that mentions
>> Retrained
>>
>> If it failed, your regular dspam logging should contain an error
>> message.
>>
>> Tom
>>
>>
> ------------------------------------------------------------------------------
>>

Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
>> by Intel and developed in partnership with Slashdot Media, is
>> your hub
> for all
>> things parallel software development, from weekly thought
>> leadership
> blogs to
>> news, videos, case studies, tutorials and more. Take a look and
>> join the conversation now. http://goparallel.sourceforge.net/
>> _______________________________________________ Dspam-user
>> mailing list [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/dspam-user




> ------------------------------------------------------------------------------


Dive into the World of Parallel Programming The Go Parallel Website,
sponsored
> by Intel and developed in partnership with Slashdot Media, is your
> hub for all things parallel software development, from weekly
> thought leadership blogs to news, videos, case studies, tutorials
> and more. Take a look and join the conversation now.
> http://goparallel.sourceforge.net/



> _______________________________________________ Dspam-user mailing
> list [hidden email]
> https://lists.sourceforge.net/lists/listinfo/dspam-user


>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Dspam-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/dspam-user




------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Loading...