[Dspam-user] Adding a feature

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[Dspam-user] Adding a feature

Phil Stracchino
On the subject of Dspam and its status ... I've wished for a long time
for a feature to manually whitelist a sender via the web interface.  I'm
starting to look at the code to see if i can figure out how to implement
this, but I'd appreciate it if someone with a good level of familiarity
with the codebase and how DSpam works internally would contact me and
perhaps give some advice on a general approach of how it *should* best
be implemented.  I'd like to implement a good and solid feature, not a
"this looks like it ought to work" hack.


--
  Phil Stracchino
  Babylon Communications
  [hidden email]
  [hidden email]
  Landline: 603.293.8485

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Adding a feature

ktm@rice.edu
On Thu, Dec 04, 2014 at 10:20:45AM -0500, Phil Stracchino wrote:

> On the subject of Dspam and its status ... I've wished for a long time
> for a feature to manually whitelist a sender via the web interface.  I'm
> starting to look at the code to see if i can figure out how to implement
> this, but I'd appreciate it if someone with a good level of familiarity
> with the codebase and how DSpam works internally would contact me and
> perhaps give some advice on a general approach of how it *should* best
> be implemented.  I'd like to implement a good and solid feature, not a
> "this looks like it ought to work" hack.
>
>
> --
>   Phil Stracchino
>   Babylon Communications
>   [hidden email]
>   [hidden email]
>   Landline: 603.293.8485
>

Hi Phil,

The whitelist happens when the appropriate token based on the senders
entire From: address line is found in the database. It should be relatively
simple to do that using the information in the History or Qurantine tabs.

Regards,
Ken

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Adding a feature

Stevan Bajić
In reply to this post by Phil Stracchino
Hello Phil,

DSPAM does not whitelist a sender but instead adds a special token with
the whole FROM line.
So "From: [hidden email]" will be translated to "From*[hidden email]".
The problem with a whitelisting functionality in the WebUI is that the
from line could be:
From: "Stevan Bajić" <[hidden email]>   -> Token: From*"Stevan Bajić"
<[hidden email]>
From: [hidden email]  -> Token: From*[hidden email]
From: <[hidden email]>  -> Token: From*<[hidden email]>
and and and....

To make the manual whitelisting work, you would need to weaken the
current mechanism and normalize the sender address and add that
normalized sender address as a whitelist token to the storage engine.
Then you could add a whitelist functionality to the WebUI. But with the
current whitelisting functionality adding a option to whitelist one
email address is practically pointless.

Usually the whitelist threshold is 10. If you want your DSPAM instance
to whitelist faster, then try lowering the whitelistThreshold preference.

--
  Kind Regards from Switzerland,

  Stevan Bajić


On 04.12.2014 16:20, Phil Stracchino wrote:

> On the subject of Dspam and its status ... I've wished for a long time
> for a feature to manually whitelist a sender via the web interface.  I'm
> starting to look at the code to see if i can figure out how to implement
> this, but I'd appreciate it if someone with a good level of familiarity
> with the codebase and how DSpam works internally would contact me and
> perhaps give some advice on a general approach of how it *should* best
> be implemented.  I'd like to implement a good and solid feature, not a
> "this looks like it ought to work" hack.
>
>


------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Adding a feature

Phil Stracchino
On 12/04/14 11:03, Stevan Bajić wrote:

> Hello Phil,
>
> DSPAM does not whitelist a sender but instead adds a special token with
> the whole FROM line.
> So "From: [hidden email]" will be translated to "From*[hidden email]".
> The problem with a whitelisting functionality in the WebUI is that the
> from line could be:
> From: "Stevan Bajić" <[hidden email]>   -> Token: From*"Stevan Bajić"
> <[hidden email]>
> From: [hidden email]  -> Token: From*[hidden email]
> From: <[hidden email]>  -> Token: From*<[hidden email]>
> and and and....
>
> To make the manual whitelisting work, you would need to weaken the
> current mechanism and normalize the sender address and add that
> normalized sender address as a whitelist token to the storage engine.
> Then you could add a whitelist functionality to the WebUI. But with the
> current whitelisting functionality adding a option to whitelist one
> email address is practically pointless.
>
> Usually the whitelist threshold is 10. If you want your DSPAM instance
> to whitelist faster, then try lowering the whitelistThreshold preference.

Stevan,
I considered just lowering the threshold.  The problem I'm trying to
solve is that I have two specific senders whom Dspam *almost always*
misclassifies as spam, so the sender never gets the chance to reach the
whitelisting threshold before another misclassification resets the
counter.  And retraining missed messages does not appear to count
towards satisfying the whitelist threshold.


--
  Phil Stracchino
  Babylon Communications
  [hidden email]
  [hidden email]
  Landline: 603.293.8485

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Adding a feature

Stevan Bajić
On 04.12.2014 18:04, Phil Stracchino wrote:

> On 12/04/14 11:03, Stevan Bajić wrote:
>> Hello Phil,
>>
>> DSPAM does not whitelist a sender but instead adds a special token with
>> the whole FROM line.
>> So "From: [hidden email]" will be translated to "From*[hidden email]".
>> The problem with a whitelisting functionality in the WebUI is that the
>> from line could be:
>> From: "Stevan Bajić" <[hidden email]>   -> Token: From*"Stevan Bajić"
>> <[hidden email]>
>> From: [hidden email]  -> Token: From*[hidden email]
>> From: <[hidden email]>  -> Token: From*<[hidden email]>
>> and and and....
>>
>> To make the manual whitelisting work, you would need to weaken the
>> current mechanism and normalize the sender address and add that
>> normalized sender address as a whitelist token to the storage engine.
>> Then you could add a whitelist functionality to the WebUI. But with the
>> current whitelisting functionality adding a option to whitelist one
>> email address is practically pointless.
>>
>> Usually the whitelist threshold is 10. If you want your DSPAM instance
>> to whitelist faster, then try lowering the whitelistThreshold preference.
> Stevan,
> I considered just lowering the threshold.  The problem I'm trying to
> solve is that I have two specific senders whom Dspam *almost always*
> misclassifies as spam, so the sender never gets the chance to reach the
> whitelisting threshold before another misclassification resets the
> counter.  And retraining missed messages does not appear to count
> towards satisfying the whitelist threshold.
>
>
Hello Phi,

you could try to delete their whitelist token in your storage backend or
bump their innocent counter. When I look at my instance and check for
your name then I have this:

hermes ~ # dspam_dump [hidden email] "From*Phil Stracchino
<[hidden email]>"
4102437411936826873  S: 00000  I: 00003  P: 0.4000
hermes ~ #


So far I got 3 times a innocent hit and none spam hit. My
whitelistThreshold is 10:
hermes ~ # dspam_admin aggregate preference [hidden email] | grep
whitelistThreshold
whitelistThreshold=10
hermes ~ #

So far you are not whitelisted but I could go and change in my storage
backend the token to be over 10:

mysql> select * from dspam_token_data where uid=4 and
token=4102437411936826873;
+-----+---------------------+-----------+---------------+------------+
| uid | token               | spam_hits | innocent_hits | last_hit |
+-----+---------------------+-----------+---------------+------------+
|   4 | 4102437411936826873 |         0 |             3 | 2014-12-04 |
+-----+---------------------+-----------+---------------+------------+
1 row in set (0.00 sec)

mysql>

Let's change it to 30:
mysql> update dspam_token_data set innocent_hits=30 where uid=4 and
token=4102437411936826873;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql>


Let's check on the command line with dspam_dump:
hermes ~ # dspam_dump [hidden email] "From*Phil Stracchino
<[hidden email]>"
4102437411936826873  S: 00000  I: 00030  P: 0.0100
hermes ~ #



Bingo :)


--
  Kind Regards from Switzerland,

  Stevan Bajić


------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Adding a feature

Phil Stracchino
On 12/04/14 12:38, Stevan Bajić wrote:
> you could try to delete their whitelist token in your storage backend or
> bump their innocent counter. When I look at my instance and check for
> your name then I have this:

Zeroing the spam_hits for one of the two users APPEARS to have worked.
Observing to make sure...   :)


--
  Phil Stracchino
  Babylon Communications
  [hidden email]
  [hidden email]
  Landline: 603.293.8485

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user