[Dspam-user] Can't Begin Using Dspam with hash_drv in TOE Mode

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[Dspam-user] Can't Begin Using Dspam with hash_drv in TOE Mode

Sabahattin Gucukoglu
Hello,

Hope someone can help.  I’m running into a very frustrating problem with a new dspam configuration.  When using the hash driver with a virtual testing user, I get mail delivered to my trusted MDA (for testing, /bin/cat), without X-Dspam* headers, but only in TOE mode; in TEFT mode everything works, the initial extents are generated, and delivery begins for the new user from the start.  It’s important for me that the first email that arrives on this appliance should be classified, because the signatures will be needed to retrain it.  I have to conclude that there’s something wrong with dspam.  Similar comments apply when classifying on an empty hash file with the —classify option.

I run this (root is trusted):
# dspam —debug —user dspam —deliver=spam,nonspam <msg.txt

msg.txt is just a silly test email, headers and body, reflected to the MDA without change:

From: me
To: you
Date: today
Subject: carry on

etc
and so on
and so forth

And this appears in mail.log:
Jun 17 16:08:11 mail dspam[11933]: bailing on error -2
Jun 17 16:08:11 mail dspam[11933]: received invalid result (!DSR_ISSPAM && !DSR_ISINNOCENT): -2
Jun 17 16:08:11 mail dspam[11933]: process_message returned error -5.  delivering.

Here’s what’s in /var/log/dspam/dspam.debug (Debian package):

11933: [06/17/2014 16:08:11] No QuarantineAgent option found. Using standard quarantine.
11933: [06/17/2014 16:08:11] DSPAM Instance Startup
11933: [06/17/2014 16:08:11] input args: dspam --debug --user dspam --deliver=spam,nonspam
11933: [06/17/2014 16:08:11] pass-thru args: /bin/cat
11933: [06/17/2014 16:08:11] processing user dspam
11933: [06/17/2014 16:08:11] uid = 0, euid = 0, gid = 0, egid = 121
11933: [06/17/2014 16:08:11] loading preferences for user dspam
11933: [06/17/2014 16:08:11] default preferences empty. reverting to dspam.conf preferences.
11933: [06/17/2014 16:08:11] Loading preferences from dspam.conf
11933: [06/17/2014 16:08:11] using /var/spool/dspam/opt-in/local/dspam.dspam as path
11933: [06/17/2014 16:08:11] using /var/spool/dspam/opt-out/local/dspam.nodspam as path
11933: [06/17/2014 16:08:11] sedation level set to: 0
11933: [06/17/2014 16:08:11] creating directory '/var/spool/dspam/data/local'
11933: [06/17/2014 16:08:11] creating directory '/var/spool/dspam/data/local/dspam'
11933: [06/17/2014 16:08:11] Loading 4 BNR patterns
11933: [06/17/2014 16:08:11] [burton] [0.500000] and+so (2frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] and+so (2frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so (2frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so (2frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] and (2frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] and (2frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so+#+#+so+forth (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] and+#+on+and+so (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so+#+and+so+forth (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+and+so+#+and (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+#+#+on+and (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+#+so+#+and (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so+on+#+#+forth (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+and+so+on+and (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] and+so+#+and+so (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so+on+#+so+forth (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+#+so+on+and (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] and+#+#+and+so (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+#+#+#+and (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so+on+and+#+forth (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+and+#+#+and (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so+#+#+#+forth (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so+#+and+#+forth (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] so+on+and+so+forth (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] and+#+on+#+so (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] and+so+on+and+so (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+and+#+on+and (1frq, 0s, 0i)
11933: [06/17/2014 16:08:11] Burton-Bayesian Probability: 0.500000 Samples: 27
11933: [06/17/2014 16:08:11] no factors specified; using default
11933: [06/17/2014 16:08:11] Result Confidence: 0.50
11933: [06/17/2014 16:08:11] _ds_setall_spamrecords() failed
11933: [06/17/2014 16:08:11] total processing time: 0.00211s
11933: [06/17/2014 16:08:11] delivering message
11933: [06/17/2014 16:08:11] Opening pipe to LDA: /bin/cat
11933: [06/17/2014 16:08:11] LDA returned success
11933: [06/17/2014 16:08:11] DSPAM Instance Shutdown.  Exit Code: 0

Here’s the output of dspam —version:

DSPAM Anti-Spam Suite 3.10.2 (agent/library)

Copyright (C) 2002-2012 DSPAM Project
http://dspam.sourceforge.net.

DSPAM may be copied only under the terms of the GNU Affero General Public
License, a copy of which can be found with the DSPAM distribution kit.

Configuration parameters:  '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--build=x86_64-linux-gnu' '--host=x86_64-linux-gnu' '--sysconfdir=/etc/dspam' '--disable-dependency-tracking' '--enable-split-configuration' '--enable-static' '--enable-external-lookup' '--enable-syslog' '--with-logdir=/var/log/dspam/' '--with-dspam-home=/var/spool/dspam' '--enable-domain-scale' '--with-delivery-agent=/usr/bin/procmail' '--enable-daemon' '--with-mysql-includes=/usr/include/mysql' '--with-pgsql-includes=/usr/include/postgresql' '--with-storage-driver=hash_drv,mysql_drv,pgsql_drv,sqlite3_drv' '--enable-debug' '--enable-virtual-users' '--enable-preferences-extension' '--enable-clamav' 'build_alias=x86_64-linux-gnu' 'host_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,defs -Wl,--as-needed' 'CPPFLAGS=-D_FORTIFY_SOURCE=2’

As promised the /var/spool/dspam/data now contains the local/dspam directory, but there is no .sig directory, only a lock file and the .css file.  The css file is 37M (64-bit machine).

And finally the configuration, with blank lines and comments removed (empty preferences, no files in /var/spool/dspam at start, no default prefs; all overridden):

Home /var/spool/dspam
StorageDriver /usr/lib/x86_64-linux-gnu/dspam/libhash_drv.so
TrustedDeliveryAgent "/bin/cat"
FallbackDomains off
EnablePlusedDetail off
PlusedCharacter -
PlusedUserLowercase off
OnFail unlearn
Trust root
Trust vmail
TrainingMode toe
TestConditionalTraining on
Feature noise
Feature tb=0
Algorithm burton
Tokenizer sbph
PValue markov
WebStats off
ImprobabilityDrive on
Preference "trainingMode=TOE" # { TOE | TUM | TEFT | NOTRAIN } -> default:teft
Preference "spamAction=deliver" # { quarantine | tag | deliver } -> default:quarantine
Preference "spamSubject=[SPAM]" # { string } -> default:[SPAM]
Preference "statisticalSedation=0" # { 0 - 10 } -> default:0
Preference "enableBNR=on" # { on | off } -> default:off
Preference "enableWhitelist=off" # { on | off } -> default:on
Preference "signatureLocation=headers" # { message | headers } -> default:message
Preference "tagSpam=off" # { on | off }
Preference "tagNonspam=off" # { on | off }
Preference "showFactors=off" # { on | off } -> default:off
Preference "optIn=on" # { on | off }
Preference "optOut=off" # { on | off }
Preference "whitelistThreshold=10" # { Integer } -> default:10
Preference "makeCorpus=off" # { on | off } -> default:off
Preference "storeFragments=off" # { on | off } -> default:off
Preference "processorBias=off" # { on | off } -> default:on
Preference "fallbackDomain=off" # { on | off } -> default:off
Preference "trainPristine=off" # { on | off } -> default:off
Preference "optOutClamAV=off" # { on | off } -> default:off
Preference "ignoreRBLLookups=off" # { on | off } -> default:off
Preference "RBLInoculate=off" # { on | off } -> default:off
Preference "notifications=off" # { on | off } -> default:off
RBLInoculate off
Notifications off
PurgeSignatures 30 # Stale signatures
PurgeNeutral 90 # Tokens with neutralish probabilities
PurgeUnused 90 # Unused tokens
PurgeHapaxes 30 # Tokens with less than 5 hits (hapaxes)
PurgeHits1S 15 # Tokens with only 1 spam hit
PurgeHits1I 15 # Tokens with only 1 innocent hit
LocalMX 127.0.0.1
SystemLog off
UserLog off
TrainPristine off
Opt out
TrackSources spam nonspam virus
ParseToHeaders off
ChangeModeOnParse off
ChangeUserOnParse off
MaxMessageSize 134217728
ProcessorWordFrequency occurrence
ProcessorURLContext on
ProcessorBias off
StripRcptDomain on
ExtLookup off # Turns on/off external lookup
HashRecMax 1572869
HashAutoExtend on
HashMaxExtents 1000
HashExtentSize 1572869
HashMaxSeek 134217728
HashConcurrentUser user
HashConnectionCache 10

So there it is.  I tried looking for answers, but got lost somewhere about four levels of undocumented error codes into the referenced functions.  But from what I’ve seen so far, it’s the “Unknown” error -2, whatever that is. :)

Any help much appreciated!

Cheers,
Sabahattin


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Can't Begin Using Dspam with hash_drv in TOE Mode

Tom Hendrikx
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 17-06-14 19:46, Sabahattin Gucukoglu wrote:

> Hello,
>
> Hope someone can help.  I’m running into a very frustrating problem
> with a new dspam configuration.  When using the hash driver with a
> virtual testing user, I get mail delivered to my trusted MDA (for
> testing, /bin/cat), without X-Dspam* headers, but only in TOE mode;
> in TEFT mode everything works, the initial extents are generated,
> and delivery begins for the new user from the start.  It’s
> important for me that the first email that arrives on this
> appliance should be classified, because the signatures will be
> needed to retrain it.  I have to conclude that there’s something
> wrong with dspam.  Similar comments apply when classifying on an
> empty hash file with the —classify option.
>
> I run this (root is trusted): # dspam —debug —user dspam
> —deliver=spam,nonspam <msg.txt
>
> msg.txt is just a silly test email, headers and body, reflected to
> the MDA without change:
>
> From: me To: you Date: today Subject: carry on
>
> etc and so on and so forth
>
> And this appears in mail.log: Jun 17 16:08:11 mail dspam[11933]:
> bailing on error -2 Jun 17 16:08:11 mail dspam[11933]: received
> invalid result (!DSR_ISSPAM && !DSR_ISINNOCENT): -2 Jun 17 16:08:11
> mail dspam[11933]: process_message returned error -5.  delivering.
>
> Here’s what’s in /var/log/dspam/dspam.debug (Debian package):
>
> 11933: [06/17/2014 16:08:11] No QuarantineAgent option found. Using
> standard quarantine. 11933: [06/17/2014 16:08:11] DSPAM Instance
> Startup 11933: [06/17/2014 16:08:11] input args: dspam --debug
> --user dspam --deliver=spam,nonspam 11933: [06/17/2014 16:08:11]
> pass-thru args: /bin/cat 11933: [06/17/2014 16:08:11] processing
> user dspam 11933: [06/17/2014 16:08:11] uid = 0, euid = 0, gid = 0,
> egid = 121 11933: [06/17/2014 16:08:11] loading preferences for
> user dspam 11933: [06/17/2014 16:08:11] default preferences empty.
> reverting to dspam.conf preferences. 11933: [06/17/2014 16:08:11]
> Loading preferences from dspam.conf 11933: [06/17/2014 16:08:11]
> using /var/spool/dspam/opt-in/local/dspam.dspam as path 11933:
> [06/17/2014 16:08:11] using
> /var/spool/dspam/opt-out/local/dspam.nodspam as path 11933:
> [06/17/2014 16:08:11] sedation level set to: 0 11933: [06/17/2014
> 16:08:11] creating directory '/var/spool/dspam/data/local' 11933:
> [06/17/2014 16:08:11] creating directory
> '/var/spool/dspam/data/local/dspam' 11933: [06/17/2014 16:08:11]
> Loading 4 BNR patterns 11933: [06/17/2014 16:08:11] [burton]
> [0.500000] and+so (2frq, 0s, 0i) 11933: [06/17/2014 16:08:11]
> [burton] [0.500000] and+so (2frq, 0s, 0i) 11933: [06/17/2014
> 16:08:11] [burton] [0.500000] so (2frq, 0s, 0i) 11933: [06/17/2014
> 16:08:11] [burton] [0.500000] so (2frq, 0s, 0i) 11933: [06/17/2014
> 16:08:11] [burton] [0.500000] and (2frq, 0s, 0i) 11933: [06/17/2014
> 16:08:11] [burton] [0.500000] and (2frq, 0s, 0i) 11933: [06/17/2014
> 16:08:11] [burton] [0.500000] so+#+#+so+forth (1frq, 0s, 0i) 11933:
> [06/17/2014 16:08:11] [burton] [0.500000] and+#+on+and+so (1frq,
> 0s, 0i) 11933: [06/17/2014 16:08:11] [burton] [0.500000]
> so+#+and+so+forth (1frq, 0s, 0i) 11933: [06/17/2014 16:08:11]
> [burton] [0.500000] etc+and+so+#+and (1frq, 0s, 0i) 11933:
> [06/17/2014 16:08:11] [burton] [0.500000] etc+#+#+on+and (1frq, 0s,
> 0i) 11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+#+so+#+and
> (1frq, 0s, 0i) 11933: [06/17/2014 16:08:11] [burton] [0.500000]
> so+on+#+#+forth (1frq, 0s, 0i) 11933: [06/17/2014 16:08:11]
> [burton] [0.500000] etc+and+so+on+and (1frq, 0s, 0i) 11933:
> [06/17/2014 16:08:11] [burton] [0.500000] and+so+#+and+so (1frq,
> 0s, 0i) 11933: [06/17/2014 16:08:11] [burton] [0.500000]
> so+on+#+so+forth (1frq, 0s, 0i) 11933: [06/17/2014 16:08:11]
> [burton] [0.500000] etc+#+so+on+and (1frq, 0s, 0i) 11933:
> [06/17/2014 16:08:11] [burton] [0.500000] and+#+#+and+so (1frq, 0s,
> 0i) 11933: [06/17/2014 16:08:11] [burton] [0.500000] etc+#+#+#+and
> (1frq, 0s, 0i) 11933: [06/17/2014 16:08:11] [burton] [0.500000]
> so+on+and+#+forth (1frq, 0s, 0i) 11933: [06/17/2014 16:08:11]
> [burton] [0.500000] etc+and+#+#+and (1frq, 0s, 0i) 11933:
> [06/17/2014 16:08:11] [burton] [0.500000] so+#+#+#+forth (1frq, 0s,
> 0i) 11933: [06/17/2014 16:08:11] [burton] [0.500000]
> so+#+and+#+forth (1frq, 0s, 0i) 11933: [06/17/2014 16:08:11]
> [burton] [0.500000] so+on+and+so+forth (1frq, 0s, 0i) 11933:
> [06/17/2014 16:08:11] [burton] [0.500000] and+#+on+#+so (1frq, 0s,
> 0i) 11933: [06/17/2014 16:08:11] [burton] [0.500000]
> and+so+on+and+so (1frq, 0s, 0i) 11933: [06/17/2014 16:08:11]
> [burton] [0.500000] etc+and+#+on+and (1frq, 0s, 0i) 11933:
> [06/17/2014 16:08:11] Burton-Bayesian Probability: 0.500000
> Samples: 27 11933: [06/17/2014 16:08:11] no factors specified;
> using default 11933: [06/17/2014 16:08:11] Result Confidence: 0.50
> 11933: [06/17/2014 16:08:11] _ds_setall_spamrecords() failed

This isdicates that something went wrong while writing to the css
file. I don't know what might be the problem with the css stuff, as I
use database backends only.

Could you try switching to the mysql or even the sqlite backend?
Even if you fix this issue, the hash driver has other longstanding
issues, that probably won't get fixed soon.

> 11933: [06/17/2014 16:08:11] total processing time: 0.00211s 11933:
> [06/17/2014 16:08:11] delivering message 11933: [06/17/2014
> 16:08:11] Opening pipe to LDA: /bin/cat 11933: [06/17/2014
> 16:08:11] LDA returned success 11933: [06/17/2014 16:08:11] DSPAM
> Instance Shutdown.  Exit Code: 0
>
> Here’s the output of dspam —version:
>
> DSPAM Anti-Spam Suite 3.10.2 (agent/library)
>
> Copyright (C) 2002-2012 DSPAM Project
> http://dspam.sourceforge.net.
>
> DSPAM may be copied only under the terms of the GNU Affero General
> Public License, a copy of which can be found with the DSPAM
> distribution kit.
>
> Configuration parameters:  '--prefix=/usr'
> '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
> '--infodir=${prefix}/share/info' '--sysconfdir=/etc'
> '--localstatedir=/var' '--libdir=${prefix}/lib/x86_64-linux-gnu'
> '--libexecdir=${prefix}/lib/x86_64-linux-gnu'
> '--disable-maintainer-mode' '--build=x86_64-linux-gnu'
> '--host=x86_64-linux-gnu' '--sysconfdir=/etc/dspam'
> '--disable-dependency-tracking' '--enable-split-configuration'
> '--enable-static' '--enable-external-lookup' '--enable-syslog'
> '--with-logdir=/var/log/dspam/'
> '--with-dspam-home=/var/spool/dspam' '--enable-domain-scale'
> '--with-delivery-agent=/usr/bin/procmail' '--enable-daemon'
> '--with-mysql-includes=/usr/include/mysql'
> '--with-pgsql-includes=/usr/include/postgresql'
> '--with-storage-driver=hash_drv,mysql_drv,pgsql_drv,sqlite3_drv'
> '--enable-debug' '--enable-virtual-users'
> '--enable-preferences-extension' '--enable-clamav'
> 'build_alias=x86_64-linux-gnu' 'host_alias=x86_64-linux-gnu'
> 'CFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat
> -Werror=format-security' 'LDFLAGS=-Wl,-Bsymbolic-functions
> -Wl,-z,relro -Wl,-z,defs -Wl,--as-needed'
> 'CPPFLAGS=-D_FORTIFY_SOURCE=2’
>
> As promised the /var/spool/dspam/data now contains the local/dspam
> directory, but there is no .sig directory, only a lock file and the
> .css file.  The css file is 37M (64-bit machine).
>
> And finally the configuration, with blank lines and comments
> removed (empty preferences, no files in /var/spool/dspam at start,
> no default prefs; all overridden):
>
> Home /var/spool/dspam StorageDriver
> /usr/lib/x86_64-linux-gnu/dspam/libhash_drv.so TrustedDeliveryAgent
> "/bin/cat" FallbackDomains off EnablePlusedDetail off
> PlusedCharacter - PlusedUserLowercase off OnFail unlearn Trust
> root Trust vmail TrainingMode toe TestConditionalTraining on
> Feature noise Feature tb=0 Algorithm burton Tokenizer sbph PValue
> markov WebStats off ImprobabilityDrive on Preference
> "trainingMode=TOE" # { TOE | TUM | TEFT | NOTRAIN } ->
> default:teft Preference "spamAction=deliver" # { quarantine | tag
> | deliver } -> default:quarantine Preference "spamSubject=[SPAM]"
> # { string } -> default:[SPAM] Preference "statisticalSedation=0" #
> { 0 - 10 } -> default:0 Preference "enableBNR=on" # { on | off }
> -> default:off Preference "enableWhitelist=off" # { on | off } ->
> default:on Preference "signatureLocation=headers" # { message |
> headers } -> default:message Preference "tagSpam=off" # { on | off
> } Preference "tagNonspam=off" # { on | off } Preference
> "showFactors=off" # { on | off } -> default:off Preference
> "optIn=on" # { on | off } Preference "optOut=off" # { on | off
> } Preference "whitelistThreshold=10" # { Integer } -> default:10
> Preference "makeCorpus=off" # { on | off } -> default:off
> Preference "storeFragments=off" # { on | off } -> default:off
> Preference "processorBias=off" # { on | off } -> default:on
> Preference "fallbackDomain=off" # { on | off } -> default:off
> Preference "trainPristine=off" # { on | off } -> default:off
> Preference "optOutClamAV=off" # { on | off } -> default:off
> Preference "ignoreRBLLookups=off" # { on | off } -> default:off
> Preference "RBLInoculate=off" # { on | off } -> default:off
> Preference "notifications=off" # { on | off } -> default:off
> RBLInoculate off Notifications off PurgeSignatures 30 # Stale
> signatures PurgeNeutral 90 # Tokens with neutralish probabilities
> PurgeUnused 90 # Unused tokens PurgeHapaxes 30 # Tokens with less
> than 5 hits (hapaxes) PurgeHits1S 15 # Tokens with only 1 spam hit
> PurgeHits1I 15 # Tokens with only 1 innocent hit LocalMX 127.0.0.1
> SystemLog off UserLog off TrainPristine off Opt out TrackSources
> spam nonspam virus ParseToHeaders off ChangeModeOnParse off
> ChangeUserOnParse off MaxMessageSize 134217728
> ProcessorWordFrequency occurrence ProcessorURLContext on
> ProcessorBias off StripRcptDomain on ExtLookup off # Turns
> on/off external lookup HashRecMax 1572869 HashAutoExtend on
> HashMaxExtents 1000 HashExtentSize 1572869 HashMaxSeek
> 134217728 HashConcurrentUser user HashConnectionCache 10
>
> So there it is.  I tried looking for answers, but got lost
> somewhere about four levels of undocumented error codes into the
> referenced functions.  But from what I’ve seen so far, it’s the
> “Unknown” error -2, whatever that is. :)
>
> Any help much appreciated!
>
> Cheers, Sabahattin
>
>
> ------------------------------------------------------------------------------
>
>
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
> Find What Matters Most in Your Big Data with HPCC Systems Open
> Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages
> Graph Analysis for Fast Processing & Easy Data Exploration
> http://p.sf.net/sfu/hpccsystems 
> _______________________________________________ Dspam-user mailing
> list [hidden email]
> https://lists.sourceforge.net/lists/listinfo/dspam-user
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJToJBMAAoJEJPfMZ19VO/1KD8QAJMzGWkgIRZM76NNMqLZ3/vE
lZsOdVlBUwIg3H/yc39ZQVoD4fUd9L/h2Vvb5mBYRZmBw8KPtW2tJwZZ1x1r/JDb
gNsSq8LTGZD4aNUHMVX9FFq11A8kVb9n0L7JwuCcGm8OBlG6clS437seVVXDNgE6
NlyKOyAFWuK2asHUUcDu/rzD0n3rDaVVVJN31z6uZxzyruGQ6fuXNjv04UxhLuzY
NpGXYRxQ7ZfCZbLnZRMu1C0aeWVGocKuVFYzXPvvETknlsKbNdO48ZkGdt4fmza8
wh+qRLrDYsFa226rQK6fziC/f7hpddP4i93Cw/Zp+4aOd749sUNsQ/k70WBD6wFs
cJL8sz0DjNNmbneHrhcvFDT4QiCzY0VndDJKnnhH3F4wBQTz+NU34TpCkvnRfkdn
BM7fqss6hJkJKKREkDZ8X+81ZEUdTvbrJ8Jvrh2XMJ9dWcydWQtH9FHEcLBdHu8+
ziFW1/5ggLDQMaZYlvDRhJXWHObF7L/PGCmUHc1isUPkah0VK5Y9rs8D7W7n/P5+
vhSAavCDTYCr9IrAyDZ2pRm193+8PjH7SsNz59xhKEzFZ/zvER2ip4iNHZdgV7mj
xWrZjPWk9LBkCVZtevX8gvd5OoJ0mU7lrpColqWtbUifcnb2y6qTR0VAljCYBxjT
59acTzi3vuKLccTFnj6H
=+ZMb
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Can't Begin Using Dspam with hash_drv in TOE Mode

Sabahattin Gucukoglu
On 17 Jun 2014, at 20:00, Tom Hendrikx <[hidden email]> wrote:

> On 17-06-14 19:46, Sabahattin Gucukoglu wrote:
>> [06/17/2014 16:08:11] Burton-Bayesian Probability: 0.500000
>> Samples: 27 11933: [06/17/2014 16:08:11] no factors specified;
>> using default 11933: [06/17/2014 16:08:11] Result Confidence: 0.50
>> 11933: [06/17/2014 16:08:11] _ds_setall_spamrecords() failed
>
> This isdicates that something went wrong while writing to the css
> file. I don't know what might be the problem with the css stuff, as I
> use database backends only.
>
> Could you try switching to the mysql or even the sqlite backend?
> Even if you fix this issue, the hash driver has other longstanding
> issues, that probably won't get fixed soon.

Ah, thanks.  I certainly took well to the sqlite3 driver, and it does happen that the machine has MariaDB on it.  Would that do?

However the reason I selected hash was sbph+markov.  That’s not going to be supported with sqlite3/mysql, is it?  I *suppose* I could live with this. :)

Cheers,
Sabahattin


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Can't Begin Using Dspam with hash_drv in TOE Mode

ktm@rice.edu
On Tue, Jun 17, 2014 at 08:46:51PM +0100, Sabahattin Gucukoglu wrote:

> On 17 Jun 2014, at 20:00, Tom Hendrikx <[hidden email]> wrote:
> > On 17-06-14 19:46, Sabahattin Gucukoglu wrote:
> >> [06/17/2014 16:08:11] Burton-Bayesian Probability: 0.500000
> >> Samples: 27 11933: [06/17/2014 16:08:11] no factors specified;
> >> using default 11933: [06/17/2014 16:08:11] Result Confidence: 0.50
> >> 11933: [06/17/2014 16:08:11] _ds_setall_spamrecords() failed
> >
> > This isdicates that something went wrong while writing to the css
> > file. I don't know what might be the problem with the css stuff, as I
> > use database backends only.
> >
> > Could you try switching to the mysql or even the sqlite backend?
> > Even if you fix this issue, the hash driver has other longstanding
> > issues, that probably won't get fixed soon.
>
> Ah, thanks.  I certainly took well to the sqlite3 driver, and it does happen that the machine has MariaDB on it.  Would that do?
>
> However the reason I selected hash was sbph+markov.  That’s not going to be supported with sqlite3/mysql, is it?  I *suppose* I could live with this. :)
>
> Cheers,
> Sabahattin
>

I believe they are in the current release. We are using PostgreSQL with osb+markov
now. By the way, I would recommend osb over sbph. It performs less DB work and is
generally more accurate.

Regards,
Ken

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user
Reply | Threaded
Open this post in threaded view
|

Re: [Dspam-user] Can't Begin Using Dspam with hash_drv in TOE Mode

Sabahattin Gucukoglu
On 17 Jun 2014, at 20:54, [hidden email] wrote:
On Tue, Jun 17, 2014 at 08:46:51PM +0100, Sabahattin Gucukoglu wrote:

>> On 17 Jun 2014, at 20:00, Tom Hendrikx <[hidden email]> wrote:
>>> On 17-06-14 19:46, Sabahattin Gucukoglu wrote:
>>>> [06/17/2014 16:08:11] Burton-Bayesian Probability: 0.500000
>>>> Samples: 27 11933: [06/17/2014 16:08:11] no factors specified;
>>>> using default 11933: [06/17/2014 16:08:11] Result Confidence: 0.50
>>>> 11933: [06/17/2014 16:08:11] _ds_setall_spamrecords() failed
>>>
>>> This isdicates that something went wrong while writing to the css
>>> file. I don't know what might be the problem with the css stuff, as I
>>> use database backends only.
>>>
>>> Could you try switching to the mysql or even the sqlite backend?
>>> Even if you fix this issue, the hash driver has other longstanding
>>> issues, that probably won't get fixed soon.
>>
>> Ah, thanks.  I certainly took well to the sqlite3 driver, and it does happen that the machine has MariaDB on it.  Would that do?
>>
>> However the reason I selected hash was sbph+markov.  That’s not going to be supported with sqlite3/mysql, is it?  I *suppose* I could live with this. :)
>
> I believe they are in the current release. We are using PostgreSQL with osb+markov
> now. By the way, I would recommend osb over sbph. It performs less DB work and is
> generally more accurate.

I just wanted to say that this configuration works, using sqlite3.  Thanks!  Evidently, the hash driver needs obsolescence.

Cheers,
Sabahattin


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Dspam-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dspam-user