[Dspam-user] The correct way of DSPAM training

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[Dspam-user] The correct way of DSPAM training

Орхан Касумов
A colleague of mine said that the way I'm training DSPAM is not correct.
In my setup, when a user moves a message from "Inbox" folder to "DSPAM" folder or vice versa,
DSPAM client installed on the same VM as the IMAP server connects to the DSPAM server (on a separate VM) and retrains DSPAM.
Although it works, my colleague says that DSPAM client should not connect back to the DSPAM server;
instead it should connect to the MySQL server (yet another separate VM) and change database records classifying the message as either "Spam" or "Innocent".

Frankly, I've never thought that such a setup could be possible, but after some thought, this started to seem logical to me.
Is it a feasible way of DSPAM training? Has anyone already implemented it in that way? Please share.

Thanks in advance!

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

ktm@rice.edu
On Tue, Mar 31, 2015 at 04:00:03PM +0300, Орхан Касумов wrote:

>  A colleague of mine said that the way I'm training DSPAM is not correct.
> In my setup, when a user moves a message from "Inbox" folder to "DSPAM" folder or vice versa,
> DSPAM client installed on the same VM as the IMAP server connects to the DSPAM server (on a separate VM) and retrains DSPAM.
> Although it works, my colleague says that DSPAM client should not connect back to the DSPAM server;
> instead it should connect to the MySQL server (yet another separate VM) and change database records classifying the message as either "Spam" or "Innocent".
>
> Frankly, I've never thought that such a setup could be possible, but after some thought, this started to seem logical to me.
> Is it a feasible way of DSPAM training? Has anyone already implemented it in that way? Please share.
>
> Thanks in advance!


Hi,

The DSPAM training involves checking for the existance of the DSPAM signature
in the DB, then pulling all of the tokens from it and adjusting their counts
in the backend DB. Can you code this for direct DB access? Yes. Should you
try to duplicate the DSPAM client functionality? No. You are doing your training
completely correctly. The client is light-weight since the server does the work.
Also, if you have a problem, you are completely on your own. You can write your
own web browser too. That doesn't make it a good idea. :)

Regards,
Ken

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

Darac Marjal
In reply to this post by Орхан Касумов
On Tue, Mar 31, 2015 at 04:00:03PM +0300, Орхан Касумов wrote:

>    A colleague of mine said that the way I'm training DSPAM is not correct.
>    In my setup, when a user moves a message from "Inbox" folder to "DSPAM"
>    folder or vice versa,
>    DSPAM client installed on the same VM as the IMAP server connects to the
>    DSPAM server (on a separate VM) and retrains DSPAM.
>    Although it works, my colleague says that DSPAM client should not connect
>    back to the DSPAM server;
>    instead it should connect to the MySQL server (yet another separate VM)
>    and change database records classifying the message as either "Spam" or
>    "Innocent".
This sounds logical, but requires the client to have knowledge of the
storage backend used and the credentials used to access that. The server
already knows all that, though. It's actually simpler / more secure for the
dspam client to connect to the dspam server. This keeps all the
database access code out of the client, for example.

>
>    Frankly, I've never thought that such a setup could be possible, but after
>    some thought, this started to seem logical to me.
>    Is it a feasible way of DSPAM training? Has anyone already implemented it
>    in that way? Please share.
>
>    Thanks in advance!

> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/

> _______________________________________________
> Dspam-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/dspam-user


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

Орхан Касумов
The problem is, I can't achieve full redundancy and avoid the single point of failure.
Currently my setup consists of all separate VMs grouped in two chains:

MX1=> DSPAM1 => IMAP1
MX2=> DSPAM2 => IMAP2

Any node in one chain can inter-operate with any NEXT node in its own or the other chain,
so, there's no single point of failure when it comes to message delivery, BUT...

A single point of failure still exists when it comes to DSPAM training. DSPAM client on each IMAP VM
interacts with only one DSPAM server specified as "ClientHost <IP address>" in dspam.conf.
If that DSPAM server is unavailable, the training won't work.

However, in the same dspam.conf file, the database server can be specified by providing a DNS name,
and the DNS name can then resolve to several IP addresses for failover purposes. The problem is, DSPAM server
listening on some port uses the "auto" mode to accept both standard LMTP messages from MX servers and
DSPAM-specific LMTP messages from DSPAM client. But MySQL server listens on a dedicated port
only for MySQL-specific messages. This is the most confusing part for me: how can DSPAM client inter-operate
with MySQL server to do training?

My colleague who offered this idea (unfortunately, he can't help me with configs personally)
says he did that before using some "training mode" setting in the standard dspam.conf file. Is it possible?
Can DSPAM client talk MySQL language to MySQL server (for "Spam"/"Innocent" reclassification purposes)
by changing some setting in the dspam.conf file?



Вторник, 31 марта 2015, 14:06 +01:00 от Darac Marjal <[hidden email]>:
On Tue, Mar 31, 2015 at 04:00:03PM +0300, Орхан Касумов wrote:
> A colleague of mine said that the way I'm training DSPAM is not correct.
> In my setup, when a user moves a message from "Inbox" folder to "DSPAM"
> folder or vice versa,
> DSPAM client installed on the same VM as the IMAP server connects to the
> DSPAM server (on a separate VM) and retrains DSPAM.
> Although it works, my colleague says that DSPAM client should not connect
> back to the DSPAM server;
> instead it should connect to the MySQL server (yet another separate VM)
> and change database records classifying the message as either "Spam" or
> "Innocent".

This sounds logical, but requires the client to have knowledge of the
storage backend used and the credentials used to access that. The server
already knows all that, though. It's actually simpler / more secure for the
dspam client to connect to the dspam server. This keeps all the
database access code out of the client, for example.

>
> Frankly, I've never thought that such a setup could be possible, but after
> some thought, this started to seem logical to me.
> Is it a feasible way of DSPAM training? Has anyone already implemented it
> in that way? Please share.
>
> Thanks in advance!

> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/

> _______________________________________________
> Dspam-user mailing list
> Dspam-user@...
> https://lists.sourceforge.net/lists/listinfo/dspam-user

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
Dspam-user@...
https://lists.sourceforge.net/lists/listinfo/dspam-user


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

ktm@rice.edu
On Tue, Mar 31, 2015 at 06:53:24PM +0300, Орхан Касумов wrote:

>  The problem is, I can't achieve full redundancy and avoid the single point of failure.
> Currently my setup consists of all separate VMs grouped in two chains:
>
> MX1=> DSPAM1 => IMAP1
> MX2=> DSPAM2 => IMAP2
>
> Any node in one chain can inter-operate with any NEXT node in its own or the other chain,
> so, there's no single point of failure when it comes to message delivery, BUT...
>
> A single point of failure still exists when it comes to DSPAM training. DSPAM client on each IMAP VM
> interacts with only one DSPAM server specified as "ClientHost <IP address>" in dspam.conf.
> If that DSPAM server is unavailable, the training won't work.
>
> However, in the same dspam.conf file, the database server can be specified by providing a DNS name,
> and the DNS name can then resolve to several IP addresses for failover purposes. The problem is, DSPAM server
> listening on some port uses the "auto" mode to accept both standard LMTP messages from MX servers and
> DSPAM-specific LMTP messages from DSPAM client. But MySQL server listens on a dedicated port
> only for MySQL-specific messages. This is the most confusing part for me: how can DSPAM client inter-operate
> with MySQL server to do training?
>
> My colleague who offered this idea (unfortunately, he can't help me with configs personally)
> says he did that before using some "training mode" setting in the standard dspam.conf file. Is it possible?
> Can DSPAM client talk MySQL language to MySQL server (for "Spam"/"Innocent" reclassification purposes)
> by changing some setting in the dspam.conf file?
>

Use mail alias based training to avoid the single-point of failure.

Regards,
Ken

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

Орхан Касумов
Good day.

Still working on SPAM filtering system described previously in an attempt to add redundancy/failover capabilities to it.
What I did is: on each IMAP server, local DSPAM instance now works as server IN ADDITION to being called as client.
These local DSPAM servers are not used for initial spam filtering (it's done by dedicated DSPAM servers between the SMTP servers and IMAP servers),
they will only be used for Spam/Innocent reclassification purposes (they connect to the same MySQL database as dedicated DSPAM servers).

Intended behavior is: no matter which IMAP server a user is connected to, when the user drags a mail from "Inbox" folder to "DSPAM" folder,
  1) on IMAP server, Dovecot calls local DSPAM as a client,
  2) then DSPAM client talks to local DSPAM server,
  3) then DSPAM server talks to MySQL server to reclassify the message.

The problem is, due to either incorrect arguments passed by Dovecot's antispam plugin, or some missing setting in local dspam.conf,
local DSPAM server on IMAP server can't properly talk to MySQL: user is not found is MySQL database because the path is incorrect:
(DSPAM debug output fragment)


input args: dspam --client --source=error --signature=<some value> --class=innocent
processing user [hidden email]
uid = 0, euid = 0, gid = 0, egid = 0
loading preferences for user [hidden email]
default preferences empty. reverting to dspam.conf preferences.
-> found attribute 'Home' with value '/var/db/dspam'
-> read attribute 'Home' with value '/var/db/dspam'
using /var/db/dspam/opt-in/[hidden email] as path
using /var/db/dspam/opt-out/[hidden email] as path
_mysql_drv_getpwnam: returning NULL for query on name: [hidden email]
_mysql_drv_set_spamtotals: unable to _mysql_drv_getpwnam([hidden email])



Please advise, what arguments could be passed by Dovecot's antispam plugin, so that the path for a user is not constructed as:

  /var/db/dspam/opt-in/[hidden email]

but constructed as:

  /<mail storage directory>/<example.xyz>/<[hidden email]

Am I missing something obvious?
Thanks in advance for attention!


Вторник, 31 марта 2015, 11:19 -05:00 от "[hidden email]" <[hidden email]>:
On Tue, Mar 31, 2015 at 06:53:24PM +0300, Орхан Касумов wrote:
> The problem is, I can't achieve full redundancy and avoid the single point of failure.
> Currently my setup consists of all separate VMs grouped in two chains:
>
> MX1=> DSPAM1 => IMAP1
> MX2=> DSPAM2 => IMAP2
>
> Any node in one chain can inter-operate with any NEXT node in its own or the other chain,
> so, there's no single point of failure when it comes to message delivery, BUT...
>
> A single point of failure still exists when it comes to DSPAM training. DSPAM client on each IMAP VM
> interacts with only one DSPAM server specified as "ClientHost <IP address>" in dspam.conf.
> If that DSPAM server is unavailable, the training won't work.
>
> However, in the same dspam.conf file, the database server can be specified by providing a DNS name,
> and the DNS name can then resolve to several IP addresses for failover purposes. The problem is, DSPAM server
> listening on some port uses the "auto" mode to accept both standard LMTP messages from MX servers and
> DSPAM-specific LMTP messages from DSPAM client. But MySQL server listens on a dedicated port
> only for MySQL-specific messages. This is the most confusing part for me: how can DSPAM client inter-operate
> with MySQL server to do training?
>
> My colleague who offered this idea (unfortunately, he can't help me with configs personally)
> says he did that before using some "training mode" setting in the standard dspam.conf file. Is it possible?
> Can DSPAM client talk MySQL language to MySQL server (for "Spam"/"Innocent" reclassification purposes)
> by changing some setting in the dspam.conf file?
>

Use mail alias based training to avoid the single-point of failure.

Regards,
Ken

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user


------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

Eric Broch
I know this may not be the best measure, but you can synchronize MySQL databases across servers.

On 4/8/2015 1:32 AM, Орхан Касумов wrote:
Good day.

Still working on SPAM filtering system described previously in an attempt to add redundancy/failover capabilities to it.
What I did is: on each IMAP server, local DSPAM instance now works as server IN ADDITION to being called as client.
These local DSPAM servers are not used for initial spam filtering (it's done by dedicated DSPAM servers between the SMTP servers and IMAP servers),
they will only be used for Spam/Innocent reclassification purposes (they connect to the same MySQL database as dedicated DSPAM servers).

Intended behavior is: no matter which IMAP server a user is connected to, when the user drags a mail from "Inbox" folder to "DSPAM" folder,
  1) on IMAP server, Dovecot calls local DSPAM as a client,
  2) then DSPAM client talks to local DSPAM server,
  3) then DSPAM server talks to MySQL server to reclassify the message.

The problem is, due to either incorrect arguments passed by Dovecot's antispam plugin, or some missing setting in local dspam.conf,
local DSPAM server on IMAP server can't properly talk to MySQL: user is not found is MySQL database because the path is incorrect:
(DSPAM debug output fragment)


input args: dspam --client --source=error --signature=<some value> --class=innocent
processing user [hidden email]
uid = 0, euid = 0, gid = 0, egid = 0
loading preferences for user [hidden email]
default preferences empty. reverting to dspam.conf preferences.
-> found attribute 'Home' with value '/var/db/dspam'
-> read attribute 'Home' with value '/var/db/dspam'
using /var/db/dspam/opt-in/[hidden email] as path
using /var/db/dspam/opt-out/[hidden email] as path
_mysql_drv_getpwnam: returning NULL for query on name: [hidden email]
_mysql_drv_set_spamtotals: unable to _mysql_drv_getpwnam([hidden email])



Please advise, what arguments could be passed by Dovecot's antispam plugin, so that the path for a user is not constructed as:

  /var/db/dspam/opt-in/[hidden email]

but constructed as:

  /<mail storage directory>/<example.xyz>/<[hidden email]

Am I missing something obvious?
Thanks in advance for attention!


Вторник, 31 марта 2015, 11:19 -05:00 от [hidden email] [hidden email]:
On Tue, Mar 31, 2015 at 06:53:24PM +0300, Орхан Касумов wrote:
> The problem is, I can't achieve full redundancy and avoid the single point of failure.
> Currently my setup consists of all separate VMs grouped in two chains:
>
> MX1=> DSPAM1 => IMAP1
> MX2=> DSPAM2 => IMAP2
>
> Any node in one chain can inter-operate with any NEXT node in its own or the other chain,
> so, there's no single point of failure when it comes to message delivery, BUT...
>
> A single point of failure still exists when it comes to DSPAM training. DSPAM client on each IMAP VM
> interacts with only one DSPAM server specified as "ClientHost <IP address>" in dspam.conf.
> If that DSPAM server is unavailable, the training won't work.
>
> However, in the same dspam.conf file, the database server can be specified by providing a DNS name,
> and the DNS name can then resolve to several IP addresses for failover purposes. The problem is, DSPAM server
> listening on some port uses the "auto" mode to accept both standard LMTP messages from MX servers and
> DSPAM-specific LMTP messages from DSPAM client. But MySQL server listens on a dedicated port
> only for MySQL-specific messages. This is the most confusing part for me: how can DSPAM client inter-operate
> with MySQL server to do training?
>
> My colleague who offered this idea (unfortunately, he can't help me with configs personally)
> says he did that before using some "training mode" setting in the standard dspam.conf file. Is it possible?
> Can DSPAM client talk MySQL language to MySQL server (for "Spam"/"Innocent" reclassification purposes)
> by changing some setting in the dspam.conf file?
>

Use mail alias based training to avoid the single-point of failure.

Regards,
Ken

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user



------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF


_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user


------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

Phil Stracchino
On 04/08/15 08:13, Eric Broch wrote:
> I know this may not be the best measure, but you can synchronize MySQL
> databases across servers.

Especially for a high-update-rate application like DSpam, though, I
would NOT recommend doing so using MySQL native asynchronous
replication.  Consider deploying Percona XtraDB Cluster instead.  It is
more robust and capable of much higher and, more importantly, much more
synchronous replication throughput.


--
  Phil Stracchino
  Babylon Communications
  [hidden email]
  [hidden email]
  Landline: 603.293.8485

------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

Christoph Langguth
In reply to this post by Орхан Касумов
Hi,

I'm not an expert, so I may be wrong on this, but I think that your
problem is not caused by a wrong path, but is only on the MySQL side.
specifically, it seems like this query:

select uid from dspam_virtual_uids where
username='[hidden email]';

fails (returns nothing) when run on your dspam server against whatever
DB is configured there. I think you can focus on fixing this (maybe some
DB synchronization issue? Others already gave hints about that), and
that the other stuff you mention ("setting Home...") is irrelevant for
this problem. But as said, I might be wrong.

HTH
Chris

On 08/04/15 09:32, Орхан Касумов wrote:

> Good day.
>
> Still working on SPAM filtering system described previously in an
> attempt to add redundancy/failover capabilities to it.
> What I did is: on each IMAP server, local DSPAM instance now works as
> server IN ADDITION to being called as client.
> These local DSPAM servers are not used for initial spam filtering (it's
> done by dedicated DSPAM servers between the SMTP servers and IMAP servers),
> they will only be used for Spam/Innocent reclassification purposes (they
> connect to the same MySQL database as dedicated DSPAM servers).
>
> Intended behavior is: no matter which IMAP server a user is connected
> to, when the user drags a mail from "Inbox" folder to "DSPAM" folder,
>    1) on IMAP server, Dovecot calls local DSPAM as a client,
>    2) then DSPAM client talks to local DSPAM server,
>    3) then DSPAM server talks to MySQL server to reclassify the message.
>
> The problem is, due to either incorrect arguments passed by Dovecot's
> antispam plugin, or some missing setting in local dspam.conf,
> local DSPAM server on IMAP server can't properly talk to MySQL: user is
> not found is MySQL database because the path is incorrect:
> (DSPAM debug output fragment)
>
>
> input args: dspam --client --source=error --signature=<some value>
> --class=innocent
> processing user [hidden email]
> uid = 0, euid = 0, gid = 0, egid = 0
> loading preferences for user [hidden email]
> default preferences empty. reverting to dspam.conf preferences.
> -> found attribute 'Home' with value '/var/db/dspam'
> -> read attribute 'Home' with value '/var/db/dspam'
> using /var/db/dspam/opt-in/[hidden email] as path
> using /var/db/dspam/opt-out/[hidden email] as path
> _mysql_drv_getpwnam: returning NULL for query on name:
> [hidden email]
> _mysql_drv_set_spamtotals: unable to
> _mysql_drv_getpwnam([hidden email])
>
>
>
> Please advise, what arguments could be passed by Dovecot's antispam
> plugin, so that the path for a user is not constructed as:
>
>    /var/db/dspam/opt-in/[hidden email]
>
> but constructed as:
>
>    /<mail storage directory>/<example.xyz>/<[hidden email]>?
> <mailto:[hidden email]%3E?>
>
> Am I missing something obvious?
> Thanks in advance for attention!
>


------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

Eric Broch
In reply to this post by Phil Stracchino
Thanks, Phil! I'm going to look into this myself.

On 4/8/2015 6:20 AM, Phil Stracchino wrote:

> On 04/08/15 08:13, Eric Broch wrote:
>> I know this may not be the best measure, but you can synchronize MySQL
>> databases across servers.
> Especially for a high-update-rate application like DSpam, though, I
> would NOT recommend doing so using MySQL native asynchronous
> replication.  Consider deploying Percona XtraDB Cluster instead.  It is
> more robust and capable of much higher and, more importantly, much more
> synchronous replication throughput.
>
>


------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] The correct way of DSPAM training

Орхан Касумов
In reply to this post by Christoph Langguth
Dear Chris,

Thank you for properly reading and understanding my question. The problem was solved.
The problem had two causes. One of them was related to DSPAM's home directory, as indicated by this message:

                using /var/db/dspam/opt-out/orkhan.gasimov@... as path

and the other one was related to MySQL DB, as indicated by the message:

                mysql_drv_getpwnam: returning NULL for query on name: [hidden email]

The second problem happened because arguments passed by Dovecot's antispam plugin to DSPAM client included "%u" for the user.
So, DSPAM tried to find specific user (in my case, "[hidden email]") in the "dspam_virtual_uids" table in MySQL DB.
But the record for that user didn't exist, as (for test purposes) DSPAM was configured to do filtering for "globalgroup", not for individual users.
As soon as I changed the arguments passed by Dovecot's antispam plugin to include "globalgroup" instead of "%u",
and corrected the first problem, retraining worked.

So, it seems like FOR NOW I've done with this redundant and failover-viable mail system with spam filter.
Thanks to everyone for willing to help.


Среда, 8 апреля 2015, 14:58 +02:00 от Christoph Langguth <[hidden email]>:
Hi,

I'm not an expert, so I may be wrong on this, but I think that your
problem is not caused by a wrong path, but is only on the MySQL side.
specifically, it seems like this query:

select uid from dspam_virtual_uids where
username='[hidden email]';

fails (returns nothing) when run on your dspam server against whatever
DB is configured there. I think you can focus on fixing this (maybe some
DB synchronization issue? Others already gave hints about that), and
that the other stuff you mention ("setting Home...") is irrelevant for
this problem. But as said, I might be wrong.

HTH
Chris

On 08/04/15 09:32, Орхан Касумов wrote:
> Good day.
>
> Still working on SPAM filtering system described previously in an
> attempt to add redundancy/failover capabilities to it.
> What I did is: on each IMAP server, local DSPAM instance now works as
> server IN ADDITION to being called as client.
> These local DSPAM servers are not used for initial spam filtering (it's
> done by dedicated DSPAM servers between the SMTP servers and IMAP servers),
> they will only be used for Spam/Innocent reclassification purposes (they
> connect to the same MySQL database as dedicated DSPAM servers).
>
> Intended behavior is: no matter which IMAP server a user is connected
> to, when the user drags a mail from "Inbox" folder to "DSPAM" folder,
> 1) on IMAP server, Dovecot calls local DSPAM as a client,
> 2) then DSPAM client talks to local DSPAM server,
> 3) then DSPAM server talks to MySQL server to reclassify the message.
>
> The problem is, due to either incorrect arguments passed by Dovecot's
> antispam plugin, or some missing setting in local dspam.conf,
> local DSPAM server on IMAP server can't properly talk to MySQL: user is
> not found is MySQL database because the path is incorrect:
> (DSPAM debug output fragment)
>
>
> input args: dspam --client --source=error --signature=<some value>
> --class=innocent
> processing user orkhan.gasimov@...
> uid = 0, euid = 0, gid = 0, egid = 0
> loading preferences for user orkhan.gasimov@...
> default preferences empty. reverting to dspam.conf preferences.
> -> found attribute 'Home' with value '/var/db/dspam'
> -> read attribute 'Home' with value '/var/db/dspam'
> using /var/db/dspam/opt-in/orkhan.gasimov@... as path
> using /var/db/dspam/opt-out/orkhan.gasimov@... as path
> _mysql_drv_getpwnam: returning NULL for query on name:
> orkhan.gasimov@...
> _mysql_drv_set_spamtotals: unable to
> _mysql_drv_getpwnam([hidden email])
>
>
>
> Please advise, what arguments could be passed by Dovecot's antispam
> plugin, so that the path for a user is not constructed as:
>
> /var/db/dspam/opt-in/orkhan.gasimov@...
>
> but constructed as:
>
> /<mail storage directory>/<example.xyz>/<orkhan.gasimov@...>?
> <mailto:[hidden email]%3E?>
>
> Am I missing something obvious?
> Thanks in advance for attention!
>


------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Dspam-user mailing list
Dspam-user@...
https://lists.sourceforge.net/lists/listinfo/dspam-user


------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user